Medical AIs: increasing reliability

The reliability of AI applications in healthcare still comes with a question mark: even sophisticated medical models may produce outdated results or even hallucinate, struggle with limited data in highly specialized fields or sub-fields, or be too resource-hungry to deploy where they’re needed most.

In this article, we highlight three current works in research that address these challenges.

A research team at Massachusetts Institute of Technology (MIT) has developed the FTTE (Federated Tiny Training Engine) with the goal to improve the efficiency of federated learning.

In federated learning, networked devices collaboratively train a shared AI model without transferring raw user data to a central server. Thus, this approach is highly relevant for privacy-sensitive applications like healthcare.

Federated learning normally assumes uniform device capability and stable connectivity. These assumptions fail as healthcare-relevant devices form very heterogeneous networks which comprise patient smartwatches, patient-worn sensors, and smartphones on the lower end of the performance-scale. FTTE addresses this by transmitting only a targeted subset of model parameters to each device, using an asynchronous update process, and weighing updates by recency.

HelixAI, a spin-off co-founded by IRB Barcelona, ICREA, the Universitat Politècnica de Catalunya (UPC), and Napptilus, targets another reliability problem: biomedical datasets are often high-dimensional and heterogeneous, yet limited in sample size. Under these conditions, standard AI models struggle.

HelixAI's approach uses Graph Foundation Models to detect complex relationships across biological data types from transcriptomics, epigenomics, and metabolomics, among others. Its clinical platform focuses on oncology, integrating multi-omic data with the aim to support diagnosis and prognosis.

A current review examines retrieval-augmented generation (RAG) as a strategy to improve AI output accuracy in medical settings. Standard LLMs rely solely on training data, which can lead to outdated or incorrect responses. And this obviously is a significant concern in healthcare.

RAG systems add a retrieval step. This means they query trusted sources such as clinical guidelines and research databases before they actually generate a response. Early applications, currently focusing on oncology, include clinical decision support (CDS), clinical trial matching, patient communication, and imaging interpretation. Across these use cases, RAG-enhanced systems showed improved accuracy and closer alignment with medical expert recommendations.

Challenges that remain are source quality, technical complexity, and safe integration into healthcare workflows. The review positions RAG as a purely supportive tool, while decision-making authority should remain with clinicians.

The three developments address AI reliability from different angles: deployment efficiency and data privacy, model robustness under data scarcity, and output accuracy through evidence grounding. None of them represents a definitive solution to the question of reliability, and each of them carries its own implementation challenges. However, they still reflect an active field where researchers are working to close the gap between AI's potential in medicine and its practical use.

Sources: MIT - Massachusetts Institute of Technology, Institute for Research in Biomedicine (IRB Barcelona), AI in Precision Oncology