> ## Content Index
> Fetch the complete content index at: https://www.skimwire.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Microsoft's diagnostic AI beat generalist doctors on hard simulated cases in a Science study
- URL: https://www.skimwire.com/microsoft-mai-dxo-diagnostic-ai-science-study/
- Published: 2026-10-22T19:48:11.000Z
- Updated: 2026-10-22T19:48:11.000Z
- Description: MAI-DxO scored 83% on complex NEJM cases vs. 36% for generalist doctors in simulations published in Science. A real hospital trial is next.
- Author: Jeremy Moon
- Tags: AI

## The skim

- Microsoft's diagnostic AI, MAI-DxO, got 83% of complex New England Journal of Medicine cases right in a simulation, versus 36% for generalist doctors using reference books and the internet, in a study published today in Science.
- On more typical emergency-room cases it scored 80% versus 47% for physicians, and Microsoft says it got there with fewer tests, at half to a third of the cost.
- The big caveats: these were simulated cases, the doctors were generalists rather than specialists, and much of the gain comes from the underlying model (OpenAI's GPT-5 and GPT-5.5). A real-world hospital trial in Boston is next.

## What Microsoft found

Medical AI is usually graded on licensing-exam questions, where every fact is handed over up front. Real diagnosis doesn't work that way. Doctors ask questions, order tests and change their minds as results come in. So Microsoft built a benchmark that does the same, called SDBench, from 304 complex cases published in The New England Journal of Medicine and a set of everyday emergency-department cases from the MIMIC-IV-ED dataset. Each case starts with limited information, and the AI or doctor has to decide what to ask or order next. It's scored on accuracy and on the estimated cost of the tests.

MAI-DxO works "like a panel of virtual physicians," Microsoft says in its [announcement](https://microsoft.ai/news/building-medical-superintelligence/?ref=skimwire.com). Several AI agents propose diagnoses, pick tests and challenge each other over several rounds. The results:

- **Complex NEJM cases:** 83% accuracy, versus 22% for physicians without reference resources and 36% with them.
- **Emergency-department cases:** 80% versus 47%.
- **Cost:** one cost-focused setup hit 81% on the NEJM cases at less than half the cost of comparable systems, Microsoft says.

Microsoft first showed an early version of this work as a preprint in mid-2025\. Today's paper is the peer-reviewed version, and Microsoft is open-sourcing the benchmark code for a selection of the NEJM cases so others can check it.

## The fine print

[TechCrunch's read](https://techcrunch.com/2026/10/22/microsofts-mustafa-suleyman-says-its-ai-outperforms-doctors-on-hard-cases-as-he-chases-medical-superintelligence/?ref=skimwire.com) of the study is worth knowing. Cases that hard are normally handled by specialists or teams of doctors, and none were in the comparison. Much of the improvement comes from stronger underlying models, with the orchestration adding gains on top. GPT-5 alone with a basic prompt already scored 78.9% on the complex cases. With the newer GPT-5.5, MAI-DxO lifted that model's own complex-case score from 72% to nearly 87%, but on emergency cases GPT-5.5 scored about 85% with or without it. The system helped weaker models most: Google's Gemini 2.5 Flash went from 46.4% to nearly 69%.

## Why it matters

Microsoft joins Google and OpenAI in claiming its AI can match or beat doctors, and it's betting health care is where it can stand out. Microsoft AI CEO Mustafa Suleyman called the result a step toward "medical superintelligence" and told TechCrunch that Microsoft is working with Mayo Clinic on a health foundation model planned for next year. Microsoft already sells clinical tools such as Dragon Copilot dictation and PowerScribe radiology reporting.

The real test is next. In a new partnership with Beth Israel Deaconess Medical Center in Boston, patients who consent will use the system alongside their normal care, and Microsoft will compare its recommendations with the doctors' own assessments. Until results like that come in, MAI-DxO is a strong benchmark score, not a doctor.

*Sources:* [*Microsoft AI: Building Medical Superintelligence*](https://microsoft.ai/news/building-medical-superintelligence/?ref=skimwire.com) *and* [*TechCrunch*](https://techcrunch.com/2026/10/22/microsofts-mustafa-suleyman-says-its-ai-outperforms-doctors-on-hard-cases-as-he-chases-medical-superintelligence/?ref=skimwire.com)*.*