Computation Without Consequence

Psychology Today

by John Nosta

March 2, 2026

AI-Generated Deep Dive Summary

A recent study from the Icahn School of Medicine at Mount Sinai assessed a health-focused version of ChatGPT using 60 clinician-authored patient scenarios. The findings revealed that while AI performed well in routine cases and textbook emergencies, it struggled in ambiguous situations where human judgment is critical. In 52% of cases requiring emergency care, physicians agreed on the need for immediate action, but ChatGPT failed to recommend it. This highlights a key limitation of AI in handling "gray zone" scenarios, where subtle clinical signs and the potential consequences of misjudgment are significant. The study underscores that AI systems, like ChatGPT, operate based on patterns and statistical coherence from training data. While this approach excels in structured, clear-cut cases, it lacks the human clinician's ability to weigh the potential consequences of being wrong. For example, a patient with abdominal pain or fever of unknown origin may not present as an obvious emergency, but clinicians often err on the side of caution due to the real-world impact of missed diagnoses. This "leaning toward consequence" is a critical aspect of human judgment that AI currently cannot replicate. The findings also raise important questions about the role of AI in healthcare decision-making. While AI can support routine tasks and provide valuable insights, it should not be relied upon for high-stakes decisions where ambiguity and clinical intuition are essential. The study emphasizes the need to recognize the distinct strengths of computational models versus human judgment, which operates by redefining frames and considering context beyond predefined categories. Ultimately, the research suggests that AI has the potential to complement, but not replace, human clinicians. By understanding these limitations and leveraging AI's computational power alongside human judgment, healthcare providers can create a synergistic approach that optimizes both efficiency and patient safety. This distinction is crucial for readers interested in health, as it highlights the importance of balancing technology with human expertise in medicine.

Verticals

healthmental-health

Originally published on Psychology Today on 3/2/2026