Human-AI collaboration enhances diagnostic accuracy in medicine

Advanced AI systems, particularly large language models (LLMs) such as ChatGPT-4, Gemini, or Claude 3, are increasingly being explored for their potential to support medical diagnosis. These systems offer rapid access to a wide range of medical knowledge, yet they also pose risks, including hallucinated responses, biased outputs, and hard-to-interpret errors. 

The research team used data from the Human Diagnosis Project, which includes more than 2,100 realistic medical case vignettes along with validated diagnoses. They simulated different types of diagnostic teams: 

• Individual medical professionals 

• Groups of medical professionals 

• Individual AI systems 

• Groups of AI models 

• Mixed human-AI collectives 

Each of the more than 40,000 diagnoses was classified using the SNOMED CT medical standard and evaluated for accuracy. 

The study found that diagnostic quality improved as more AI models were added to a group. AI-only teams outperformed 85 percent of individual human diagnosticians. However, in many scenarios, human experts still outperformed AI. 

The key insight: hybrid teams of humans and AI models consistently achieved the best diagnostic results. “Our results show that cooperation between humans and AI models has great potential to improve patient safety,” said lead author Nikolas Zöller, postdoctoral researcher at the Max Planck Institute. 

Even the simple act of adding a single AI model to a group of physicians, or vice versa, led to significantly better outcomes. The strongest results were achieved when multiple humans and AIs collaborated. 

The success of hybrid collectives stems from error complementarity: AI and humans tend to make different types of mistakes. When AI fails, humans often succeed, and vice versa. This mutual compensation leads to higher overall accuracy. 

“It’s not about replacing humans with machines. Rather, we should view artificial intelligence as a complementary tool that unfolds its full potential in collective decision-making,” said co-author Stefan Herzog. 

The study, part of the EU-funded Hybrid Human Artificial Collective Intelligence in Open-Ended Decision Making (HACID) project, focused on diagnosis, not treatment, and used only text-based vignettes, not real-life clinical settings. Researchers see potential applications in underserved regions, where hybrid human–AI systems could improve access to medical expertise. Co-author Vito Trianni added: “The approach can also be transferred to other critical areas – such as the legal system, disaster response, or climate policy – anywhere that complex, high-risk decisions are needed.” 

MEDICA-tradefair.com; Source: Max-Planck-Institut für Bildungsforschung