AccScience Publishing / AIH / Online First / DOI: 10.36922/AIH026250061
Cite this article
4
Download
136
Views
Related Info Links
More by Authors Links
Journal Browser
Volume | Year
Issue
Search
News and Announcements
View All
PERSPECTIVE ARTICLE

From P(doom) to P(harm) in healthcare: An operational surveillance framework for accumulative clinical risk

Arturo Loaiza-Bonilla1,2* Nikhil G. Thaker3,4
Show Less
1 Massive Bio, Boca Raton, Florida, United States of America
2 St. Luke’s University Health Network, Lewis Katz School of Medicine, Temple University, Bethlehem, Pennsylvania, United States of America
3 Capital Health Medical Center, Pennington, New Jersey, United States of America
4 Bayta Systems, Newtown, Pennsylvania, United States of America
Received: 15 June 2026 | Revised: 7 July 2026 | Accepted: 7 July 2026 | Published online: 16 July 2026
© 2026 by the Author(s). This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution 4.0 International License ( https://creativecommons.org/licenses/by/4.0/ )
Abstract

Debates about artificial intelligence (AI) safety often reach for P(doom), a shorthand in AI-safety discussions for the probability of catastrophic AI risk. Healthcare needs a nearer and more testable companion: P(harm), a proposed surveillance framework for cumulative clinical risk. We define accumulative risk as harm that arises from repeated small shifts in diagnosis, triage, clinician behavior, training, or workflow rather than a single catastrophic failure. We operationalize P(harm) as a family of workflow-specific, severity-weighted conditional estimates: in a declared workflow and model version, it is the estimated probability that an independently reviewed AI-exposed encounter has an adverse outcome of at least a prespecified severity within a prespecified time horizon, after stratifying or adjusting for acuity, case mix, and local workflow covariates. In pilot use, the estimator should be prespecified, for example, a risk-adjusted empirical incidence with confidence or credible intervals or a hierarchical logistic/Bayesian model; P(harm) is not a causal-attributable fraction unless paired with a valid comparison design. The framework is organized around four local pathways—AI–clinician discordance in complex patients, automation bias, skill erosion and never-skilling, and consumer AI triage failure—plus a fifth, less mature pathway for correlated vendor or foundation-model failures across institutions. The motivating evidence is early and heterogeneous, often retrospective or vignette-based, so each pathway is framed as a surveillance hypothesis rather than proof of generalized harm. The purpose is not to slow beneficial AI, which may reduce errors and administrative burden, but to make clinical drift visible early enough for health systems to adjudicate, recalibrate, roll back, or retrain before repeated small failures accumulate.

Graphical abstract
Keywords
Artificial intelligence
Patient safety
Clinical decision support
Automation bias
Causal inference
Accumulative risk
Health-system governance
Clinical AI surveillance
Funding
None.
Conflict of interest
Arturo Loaiza-Bonilla is a co-founder and chief medical AI officer of Massive Bio and a co-founder and chief scientific officer of AEQUIS AI, and holds equity in both companies; he is an inventor on four AI-related patents (issued and/ or pending); and he holds unpaid leadership roles as a Champion of the ASCO Artificial Intelligence Community of Practice, Program Chair for NeurIPS 2025, and a founding member of CancerX (US Cancer Moonshot). He is employed by St. Luke’s University Health Network and holds a faculty appointment at the Lewis Katz School of Medicine at Temple University. Nikhil G. Thaker is affiliated with Bayta Systems and Capital Health. The authors declare these relationships, affiliations, equity holdings, patents, and leadership roles as potential competing interests. The authors declare no other competing interests.
References
  1. Ord T. The Precipice: Existential Risk and the Future of Humanity. Grand Central Publishing; 2020.
  2. Kasirzadeh A. Two types of AI existential risk: decisive and accumulative. Philos Stud. 2025;182(7):1975-2003. doi: 10.1007/s11098-025-02301-3
  3. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med. 2019;25(1):44- 56. doi: 10.1038/s41591-018-0300-7
  4. Chatziisaak D, Burri P, Sparn M, Hahnloser D, Steffen T, Bischofberger S. Concordance of ChatGPT artificial intelligence decision-making in colorectal cancer multidisciplinary meetings: retrospective study. BJS Open. 2025;9(3):zraf040. doi: 10.1093/bjsopen/zraf040
  5. Zeltzer D, Kugler Z, Hayat L, et al. Comparison of initial artificial intelligence and final physician recommendations in AI-assisted virtual urgent care visits. Ann Intern Med. 2025;178(4):498-506. doi: 10.7326/ANNALS-24-03283
  6. Goh E, Gallo R, Hom J, et al. Large language model influence on diagnostic reasoning: a randomized clinical trial. JAMA Netw Open. 2024;7(10):e2440969. doi: 10.1001/jamanetworkopen.2024.40969
  7. Dratsch T, Chen X, Mehrizi MR, et al. Automation bias in mammography: the impact of artificial intelligence BI-RADS suggestions on reader performance. Radiology. 2023;307(4):e222176. doi: 10.1148/radiol.222176
  8. Budzyń K, Romańczyk M, Kitala D, et al. Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: a multicentre, observational study. Lancet Gastroenterol Hepatol. 2025;10(10):896-903. doi: 10.1016/S2468-1253(25)00133-5
  9. Hough J, Culley N, Erganian C, Alahdab F. Potential risks of GenAI on medical education. BMJ Evid Based Med. 2025;30(6):406-408. doi: 10.1136/bmjebm-2025-114339
  10. Abdulnour REE, Gin B, Boscardin CK. Educational strategies for clinical supervision of artificial intelligence use. N Engl J Med. 2025;393(8):786-797. doi: 10.1056/NEJMra2503232
  11. Ke Y, Jin L, Ong JCL, et al. AI-induced never-skilling in medical education. Nat Med. 2026;32(6):1997-2006. doi: 10.1038/s41591-026-04438-y
  12. Ramaswamy A, Tyagi A, Hugo H, et al. ChatGPT Health performance in a structured test of triage recommendations. Nat Med. 2026;32(5):1671-1675. doi: 10.1038/s41591-026-04297-7
  13. HTI-1 Final Rule. Office of the National Coordinator for Health Information Technology. Accessed June 24, 2026. https://healthit.gov/regulations/hti-rules/hti-1-final-rule/
  14. Clinical Decision Support Software: Guidance for Industry and Food and Drug Administration Staff. US Food and Drug Administration. Published January 2026. Updated January 29, 2026. Accessed June 24, 2026. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/ clinical-decision-support-software
  15. Mitigating automation bias in physician-LLM diagnostic reasoning using behavioral nudges. ClinicalTrials.gov. Accessed June 24, 2026. https://clinicaltrials.gov/study/NCT07328815
  16. Loaiza-Bonilla A, Penberthy S. Harnessing Moravec’s paradox in health care: a new era of collaborative intelligence. NEJM AI. 2025;2(5):AIp2500005. doi: 10.1056/AIp2500005
Share
Back to top
Artificial Intelligence in Health, Electronic ISSN: 3029-2387 Print ISSN: 3041-0894, Published by AccScience Publishing