Can AI Match a Doctor's Clinical Reasoning?

The prospect of artificial intelligence (AI) stepping into roles traditionally held by medical professionals has sparked much debate. As AI technologies continue to advance, they propose a myriad of applications within healthcare. This ongoing evolution raises a pivotal question: Can AI, with its current capabilities, undertake clinical reasoning tasks on par with human doctors? Understanding the essence of clinical reasoning—a fundamental skill in medical practice that involves gathering, analyzing, and interpreting clinical information to make informed decisions—is crucial to answering this question.

---

Exploring AI's Capabilities

A recent study published in JAMA Internal Medicine sought to delve into this question by comparing the clinical reasoning capabilities of a Generative AI Model, GPT-4, against human physicians, specifically targeting internal medicine residents and attending physicians. Conducted over two months in 2023 at two Boston-based academic medical centers, the study utilized 20 clinical scenarios to mimic the sequential process of gathering clinical data. Participants, including the AI model, were tasked with developing a problem representation and a prioritized differential diagnosis, supported by justifications across four stages of each case. The Revised-IDEA (R-IDEA) score, a validated 10-point scale evaluates four core domains of clinical reasoning documentation as follows:

• Interpretative summary

• Differential diagnosis

• Explanation of the lead diagnosis

• Alternative diagnosis explained

---

Key Findings

R-IDEA Performance
- The AI model showcased superior performance in R-IDEA scores, achieving a median score of 10 (on a 10-point scale), compared to 9 for attending physicians and 8 for residents. This suggests a higher efficacy in synthesizing and representing clinical data.

Comparative Outcomes
- Diagnostic accuracy, correct clinical reasoning, and inclusion of cannot-miss diagnoses were similar between the AI model and human physicians.
- The AI model had more instances of incorrect clinical reasoning (13.8% of the time) compared to residents (2.8%), though it was worth noting it was similar to attending physicians (12.5%) pointing to an area for potential refinement.

---

Implications

The study's results highlight the potential of AI to support and enhance the clinical reasoning process, particularly in synthesizing complex clinical data into coherent problem representations. This capability could augment human clinicians' ability to diagnose and treat patients, especially in situations where quick synthesis of vast amounts of data is required.

However, the study also highlights challenges and limitations that need to be addressed. The increased instances of incorrect clinical reasoning by the AI model compared to human residents emphasize the need for cautious integration of AI into clinical workflows. Though it’s worth noting the AI was only incorrect a little bit more than attending physicians. It's crucial that AI's clinical reasoning capabilities are continuously evaluated and refined to minimize errors. Furthermore, the study conducted simulations with clinical data rather than real patient interactions, indicating that further research is needed to understand how AI models perform in real-world clinical settings.

---

Summary

So will AI replace doctors? The same question was asked of computers when they first appeared. It is more likely that doctors who use AI will replace doctors that do not - just as pretty much every doctor today depends on a computer.

But the speed at which AI capabilities are improving is impressive. It is both exciting and sometimes scary. As we navigate this journey, ethical considerations and the development of best practices will be paramount to harness AI's potential responsibly and effectively.

---