Humanizing Algorithms: Clinician Insights into Ethical and Practical Challenges in AI-Based Survival Prediction

Abstract: Artificial intelligence (AI), machine-learning (ML), and algorithm-based prediction tools for survival and mortality are on the rise in healthcare. These tools offer potentially more objective and precise estimates, intended both to supplement physicians’ clinical recommendations and increase patient understanding of different therapeutic options. Given their novelty, few studies have empirically examined the ethical and practical challenges faced by clinicians when implementing AI/ML/algorithm-based prediction tools into clinical practice. As part of an AHRQ-funded multi-site study to develop and implement an algorithmic personalized risk calculator (Heart Mate 3 Risk Score) for patients with advanced heart failure who are considering left ventricular assist device (LVAD) therapy, we present qualitative findings from clinician interviews regarding the implementation of the risk prediction calculator in specific clinical settings. Grounded in our previously published exploration of clinicians’ ethical views and values around the use of AI/ML-based personalized risk scores, we trace how normative perspectives around the use of AI/ML in clinical care meet clinicians’ actual preferences towards the ethical design of the tool itself, including preferred interface features, characteristics, and communication of results. Finally, we examine the ways in which contextual, pragmatic, and logistical challenges further shape how clinicians implement this algorithmic tool in practice at each of our six respective partnering sites. Our findings offer a glimpse into how normative preferences around AI/ML meet real-world conditions for deployment.

Kristin Kostick-Quenet, PhD – Assistant Professor, Center for Medical Ethics and Health Policy, Baylor College of Medicine; Benjamin Lang, MA – Doctoral student, Department of Philosophy, Oxford University; Meghan Hurley, MA – Research assistant, Center for Medical Ethics and Health Policy, Baylor College of Medicine; Jared Smith, PhD – Post-doctoral fellow, Center for Medical Ethics and Health Policy, Baylor College of Medicine; Jennifer Blumenthal-Barby, PhD, MA – Cullen Professor of Medical Ethics & Associate Director, Center for Medical Ethics and Health Policy, Baylor College of Medicine