Skip to Content

Human Experience and Design

Microsoft Research

Assessing Multiple Sclerosis with Kinect

  • System in use
  • System in use

Multiple Sclerosis is a neuro-degenerative condition that causes physical and cognitive impairment over an unpredictable timeframe, making the ability to detect disease change very important. Currently disease tracking is done through clinical examination, in which a neurologist watches a patient perform a number of exercises, such as touching the finger to the nose, and then rates the level of motor abnormality. However, the high variance in judgements between neurologists, or the same neurologist at different time points, makes reliably tracking disease progression problematic. The high variance in current tools requires large number of patients to be included in clinical trials, making the discovery and evaluation of new treatments a slow and expensive process.

ASSESS MS addresses this by providing a consistent, quantified measure of motor ability using Kinect that is both inexpensive and non-invasive. Using Kinect, the system captures depth videos of neuro-assessment movements performed by patients in a clinical setting with the support of the health professional. The videos are then processed offline and a classification of specific aspects of motor ability is provided based on the Expanded Disability Severity Score (EDSS). These are then presented back to the health professional through a series of visualisations that compare data over time. The project is a collaboration between the Human Experience & Design group and the Machine Learning and Perception group at Microsoft Research Cambridge, as well as Novartis Pharma AG (Basel, Switzerland) and three large research hospitals in Europe (Basel, Bern, Amsterdam). 

  • Touch screen
  • Screens and prototype one
  • Stills from Demo Videos

The Human Experience & Design team has contributed to the project through ensuring the system works appropriately in a clinical setting. A key challenge has been the standardization of data need for optimal machine learning. This includes movement standardization, such as starting a Finger Nose test with arm outstretched to the side versus the front, as well as image standardization, such as ensuring the patient is centred in the image without any clutter behind. This project has demonstrated that standardization is a slippery concept that is highly dependent on the particularities that arise in real-world use. It is created through trade-offs between technical possibility, human interaction, and clinical need. Key contributions from our research are documented in the paper: Assessing Multiple Sclerosis with Kinect: Designing computer vision systems for real-world use.

  • UI and Testing System
  • UI Screens
  • Depth video Drawing a Square

We designed a prototype system, both physical and software, to enable effective recording of depth videos in a standardized manner that is sensitive to the needs of those who use it – both patients and health professionals. An evaluation of the system, published in the paper: Usability and acceptability of ASSESS MS: a system to support the assessment of motor dysfunction in Multiple Sclerosis using depth-sensing computer vision has shown excellent results. The system is usable and supports the acquisition of highly standardized data. We are currently carrying out further research on the visualisation of the machine learning results for health professionals. We are particularly interested in how temporal representations can enable health professionals to gain new insights into the motor ability of their patients.