05.03.2024

Previous robotic prostheses only respond to their users' movement intentions to a limited extent. A new control method being developed at the TU Munich could help: A network of 128 sensors and the use of artificial intelligence could allow people to control their prosthetic hands more intuitively and naturally.

Image: Smiling woman with long, brown hair, grey-white blazer and dotted blouse – Prof. Cristina Piazza; Copyright: Andreas Heddergott/TU Munich

Prof. Cristina Piazza, TU Munich

In an interview with MEDICA-tradefair.com, Prof. Cristina Piazza explains how a more intuitive operation for prostheses can be achieved and what possibilities this could open up for the future.

Prof. Piazza, could you provide more insights into your approach?

Prof. Cristina Piazza: We are aiming for a user-centered approach, involving subjects with limb loss in our research activities, to identify their needs and the limitations of current technologies.

In the past twenty years, we observed a technological advancement in artificial hands and upper limb devices, but there are often problems with the control strategies. So far, one or two sensors on the skin measures the degree of muscle contraction in the residual limb, but only from large muscles. This allows to control the closure and reopening of the prosthesis.

Current strategies for controlling the more advanced level of dexterity of these prostheses often require a high level of cognitive effort, for instance the simultaneous activation of agonist-antagonist muscles. We were looking for a control method in robotics that comes closer to human motor control and can be more natural for the user.

How exactly does your control system work?

Piazza: We use a system consisting of two foils, each with 64 sensors, which are attached to the front and back of the forearm. This allows the system to obtain an extensive amount of data and process it with an AI to detect the user intention of movement. We analyze the different contributions of various muscles of the forearm for each hand gesture.

When a gesture is performed, our algorithm recognizes the different areas of muscle activation. This gives us spatial information about the forearm muscle activation, and we can convert this into the user's intention to perform a specific hand movement.

Unlike most current commercial systems, users do not have to use an app on their smartphone to select which gesture the bionic hand should perform, but simply contract their muscles in the most natural way. They think about rotating their wrist, the sensors extrapolate the corresponding muscle activation, the AI algorithm recognizes this intention and translates it into the prosthesis movement.

Products and exhibitors around this topic

Discover more interesting products and exhibitors in the database of MEDICA:

Photo gallery about the prosthesis control

Image: A hand holding an apple and a second hand making a gasping movement. Behind them is a hand prosthesis with fingers that are also bend to grip something; Copyright: Andreas Heddergott/TU Munich

The idea behind the prosthesis control: 128 sensors read signals of muscle activation in remaining muscles, an AI recognizes the intention behind the activation and implements a corresponding movement of the prosthesis.

Image: A woman with long blond hair, Johanna Happold, is holding a hand prosthesis that is gripping an orange; Copyright: Andreas Schmitz/TU München

The hand prosthesis should be able to move more flexible and enable a more natural movement.

Image: A woman is holding a hand prosthesis that holds an orange. A sensor is attached to one of her arms, the sensor is connected to the prosthesis with a cable; Copyright: Andreas Schmitz/TU Munich

Until now, hand prostheses have used information from two sensors on the forearm to close and open the prosthesis. In future, a network of 128 sensors and the use of AI should enable finer and more intuitive control.

Image: A woman with glasses and long dark hair, Dr. Patricia Capsi-Morales, is showing a hand prosthesis together with a foil that has black dots on it; Copyright: Andreas Schmitz/TU Munich

A total of 128 sensors are used for the control system, which are attached with two carrier foils on the forearm. This allows more muscle activity to be recorded in order to recognize the wearer's intention to move.

Image: Three women – Dr. Patricia Capsi-Morales, Prof. Cristina Piazza, graduate student Johanna Happold – present components of their prosthesis control; Copyright: Andreas Schmitz/TU Munich

The team behind the prostheses control: Dr. Patricia Capsi-Morales (left), Prof. Cristina Piazza (center) and graduate student Johanna Happold.

What is the advantage of this control system?

Piazza: We do not want the user to adapt to the system, but to tailor the system to align with the user. We try to understand which muscle groups are activated for each user and then we train the control strategies to be intuitive and match the requirements of each patient.

How long does it take to learn these controls?

Piazza: We tested this system with healthy people, and they were able to learn it in less than ten minutes. If the learning process was longer, users would probably lose motivation. Now we want to carry out the same study with participants with limb loss.

What are other possible use cases?

Piazza: One futuristic goal is to achieve individual finger movements with a natural control strategy. However, we also want to transfer our work to other areas and, for example, investigate the use of virtual reality environments for the rehabilitation of stroke patients. This is possible because these sensors can also detect reduced muscle activation and can allow stroke patients to control the movement of virtual limbs.

Another possibility is to decode signals from these sensors to extract information at the brain level. This would enable studies on human motor control, with potential application to many fields, such as the use of supernumerary limbs or exoskeletons.

Where are the limits?

Piazza: We want to develop compact wearable prostheses for everyday use. AI requires large computing capacities, which is why we need microcontrollers that can be built directly into the prosthesis.