Adaptify: Flexible games to support physiotherapy and rehabilitation

Interview with Jan Smeddinck, Workgroup Digital Media, Center for Computing and Communication Technologies, TZI, University of Bremen

Personalized approaches in medicine are very popular. Treatment with customized therapies is intended to achieve better results and be less stressful on the patients. This could soon also be the case for physiotherapy and rehabilitation since every person is not just different - he also moves differently.


Photo: Jan Smeddinck

Jan Smeddinck; © Jan Smeddinck/ Frederic Pollmann spoke with Jan Smeddinck about playful exercise programs that are meant to adapt to the user, and the important role a sensor mat plays with back exercises. Also, find out how older users can be addressed with digital games.

Mr. Smeddinck, you are involved in the "Adaptify" project. What does the name stand for?

Jan Smeddinck: The name is derived from the English verb "to adapt", which means to conform and adjust to something. This is an important point for us in playful exercise programs that support physiotherapy, rehabilitation, and prevention. They are designed to adapt to a user’s individual skills and needs.

The project's objective is to personalize the programs for these applications. We are currently addressing problems of the lower back. Ultimately, the goal is for persons with chronic back pain to perform a greater variety of exercises than otherwise possible.

What does it look like when a program adapts to a user?

Smeddinck: Of course, mobility is different with every person, especially in therapeutic application. Needless to say, we are not able to utilize difficulty levels such as "easy - medium - hard" as we know them from commercial games; we want to be more precise. We are developing a user interface where therapists can specify how well a patient is actually still able to move and the performance goals that should be achieved after a few months.
However, automated methods are also meant to project the user's development and present individual training plans. In this case, heuristic techniques collaborate with developmental trajectories known from medicine and individually collected user-related data that is evaluated using machine learning methods. This partial automation can subsequently support therapists in their work.
Photo: Older man and physical therapist during exercise

Older people can get excited about digital games, too. Developers of games for medical use need to reach out to the target audience somewhat more than developers of commerical games; ©Frederic Pollmann/ Universität Bremen, TZI

Does data protection also play a role in these types of programs?

Smeddinck: Absolutely. We want to gather mobility data from many users and different user groups. Medical data is also being incurred in a therapeutic context. This is why it is very important to properly educate users about what happens here and for us to limit ourselves to a minimum in terms of data analysis. A legal institute is a member of our project consortium to address data protection issues from the start.

The system includes two hardware components, a sensor mat, and a camera system. How are they integrated and what do they record?

Smeddinck: With modern tracking methods such as the one best known from Microsoft Kinect, an infrared camera generates a depth image of the room. It identifies a human being in it and is essentially able to offer a good assessment about the position and posture of a person's body in space.

With a certain number of exercises, you can already identify whether the user moves correctly. However, back exercises are often done near ground level, for instance in a quadruped position, that is on your hands and knees, or lying down. In this case, it is very difficult for cameras to determine the exact body posture. That is how we came up with the idea of fusing the camera image with the input from a sensor mat. This is a type of yoga or exercise mat with embedded pressure sensors that subsequently shows how the user stands or lies down. This allows us to find out more about the skeleton' posture and the balance distribution in the body.

What feedback about their progress can users receive from the program?

Smeddinck: There are three types of feedback. First, you get a direct visual feedback during the game, of course. Then there is feedback about the quality of the performed exercises. The user is meant to see whether he is doing the exercise correctly, is in a tilted position and compromises his balance or has turned his back in a bad position. Thirdly, the programs are also meant to record the user’s performance over time and thus determine basic parameters such as the level of physical mobility. In doing so, we can obtain a very different, objective image of the treatment process and developmental trajectories and show the user medium- and long-term trends.
Photo: Collage from four pictures showing physiotherapy

The pictures show an employee of a mobile social service and care provider who is also project partner at Adaptify. She demonstrates early concepts of the devices developed by Adaptify with the help of an older couple; © Frederic Pollmann/Universität Bremen, TZI

All of this is reminiscent of the classic computer and video games. Can you also reach older people who are supposed to perform these types of exercises?

Smeddinck: Older people also like games, of course. There are the classic card, board or puzzle games on the one hand. On the other hand, they can also get excited about digital games. At least, this is what tests revealed with prototypes from older projects that we conducted with groups of seniors that meet in the afternoons. After a while, the majority of the group has a positive attitude about it.

Of course, we also need to reach out to the target audience. When we are older, our perception system changes, for instance, which is why our games have clearly visible graphics and line-doubled audio and visual output. Overall, they aim at a significantly lower pace, deliberately allow for breaks during the game and are better scalable in terms of difficulty.

The project will run until 2018. Can you outline its further course?

Smeddinck: Right now, we are at a point where we produce early prototypes. Over the next few months, we will start to work with users and request feedback. We subsequently want to develop programs for various user scenarios in multiple iterations; in part, to use in practices and in part, for use at home. This is actually a large stumbling block you need to remove early on: even though we need great playful mobility programs, they also need to make sense when integrating them into the complex user scenario of different target groups. At the same time, we are also refining the sensory components for the mat and work on fusing the camera image and sensor data. In about one year, we are going to move into open, practical testing and try to prepare two to three studies.

The interview was conducted by Timo Roth and translated from German by Elena O'Meara.