DAIOR research project: Distributed AI improves telesurgery

In the DAIOR research project, doctors and technicians are working on the evaluation of surgical data from multimodal sources using artificial intelligence (AI). One of the project goals is to support robot-assisted telesurgery in real time.

In an interview with MEDICA-tradefair.com, Johannes Horsch describes how cross-location data is used and how this results in AI applications that can help with operations in the future.

What does the collaboration for evaluating the data look like?

Johannes Horsch: The use of AI and telesurgery is becoming increasingly important for operations. However, valuable data from hospitals is tied to the locations of the clinics for data protection reasons alone.

In order to use this knowledge from different locations, three partners are working together in an interdisciplinary consortium in the DAIOR (Distributed Artificial Intelligence for the Operating Room) project: Fraunhofer IPA in Mannheim is focusing on clinical healthcare technologies and robotics. The IHU Strasbourg is researching innovative image-guided therapies. And the Bosch Digital Innovation Hub (KTBW) in Stuttgart aims to increase the implementation rate of research projects and innovations and is optimally networked with the institutions in Mannheim and Strasbourg.

We use data from different locations and in different formats to train an AI model.

How do you use the data to facilitate robot-assisted surgery?

Horsch: We train the algorithm of the AI application with data from various sources. Simulation models can also be used to generate synthetic data. One approach to developing AI models is federated learning.

It can be compared to a doctor who completes his or her training in Mannheim, gains experience there and then goes through further stages of development at other locations. We also use data from different locations and sources and in different formats when training the AI. We then transfer the model to the next location and the data remains at the old location in compliance with data protection regulations.

How does the data actually improve operations?

Horsch: We differentiate between two use cases in the DAIOR project: on the one hand, we collect and analyze data such as images, videos, vital data and surgical reports, extract the knowledge and bring it together in a compact form that can be interpreted by the doctors. On the other hand, there is the robotic use case of telerobotics, for which we use the fast 5G communication structure as a basis.

Our aim is to carry out telesurgery operations across location boundaries. During an operation in Strasbourg, for example, the surgeon in Mannheim can operate the robotic system.

Due to the long distance, delays are possible or data packets are lost. With the AI model that we have trained with data from different locations, we can now compensate for such delays or dropouts by using this wealth of experience to predict certain reactions.

Robots in telesurgery can also provide haptic feedback. In contrast to audio or video signals, our haptic senses react much more sensitively to latencies. Only with this calculated prediction of a reaction do operators receive an immersive impression in their mixed-reality world, which promotes the clinical workflow.

The quality of the algorithms increases through training. This results in better predictions of how the sensor will react, and therefore also better operation of telesurgery through refined assistance systems.

Are partial automations also possible as a result of these improvements?

Johannes Horsch: Repetitive tasks will increasingly be automated in this way, but doctors will retain decision-making authority. For example, if a catheter needs to be navigated autonomously through the vascular system after a stroke, if a suture needs to be placed or if maneuvering around a vessel is required, appropriately trained assistance systems can take over such subtasks.