Microsoft Develops AI Tool That Can Diagnose 'Sequentially' Like a Doctor

 

Microsoft researchers have developed an advanced AI tool that mirrors a physician’s step-by-step diagnostic process.

According to a study published in Science, this sequential approach allowed the model to outperform general physicians and drastically improve accuracy when tackling complex medical cases.

Key Highlights & Performance Findings

• Outperformed Human Doctors: Tasked with diagnosing over 300 rare and complex medical cases, the AI model was more than twice as accurate as general physicians who had access to internet resources and medical textbooks. 
• Emergency Department Success: The tool also surpassed human doctors when evaluated against real-world scenarios from a major database of emergency room admissions. 
• Cost-Effective Work-ups: The research showed that diagnostic plans designed by the AI cost less than those conducted by human doctors due to more judicious test selection. 

How the "Diagnostic Orchestrator" Works

Traditional medical AI systems evaluate a patient's symptoms, laboratory results, and imaging studies all at once. Microsoft’s new tool instead uses an orchestrator that coordinates a simulated panel of collaborating clinical experts to tackle cases sequentially:

1. Patient History & Physical Exam: The AI first determines the most critical questions to ask regarding patient history and identifies key physical exam findings. 

2. Targeted Testing: Based on initial findings, it orders specific medical tests, reviews the results, and determines if additional testing is required. 

3. Dynamic Hypotheses: Throughout the entire work-up, the orchestrator continuously updates its working diagnostic hypotheses and treatment plans. 

Moving Beyond Multiple-Choice Testing

Standard assessments for medical AI often rely on multiple-choice board exams—a environment where older AI models already achieve high scores. However, real-world medicine is far less structured.

"AI has become very good at getting a 100% pass rate on those... But of course, that's not real-world. No patient ever came to see me with a nice summary." 

— Dr. Dominic King, Corporate VP of Microsoft AI Health 

Harsha Nori, Head of AI at Microsoft AI Health, emphasized that the evaluation measured not only raw accuracy, but also how carefully and judiciously the AI conducts the overall diagnostic process.

Next Steps: Real-World Triage Testing

Microsoft is collaborating with Beth Israel Deaconess Medical Center in Boston to evaluate the tool's capabilities in a clinical setting. The next phase of testing will focus on using the AI system to triage patients waiting in emergency department reception areas.