Patient and Physician Perspectives on AI Transparency

Abstract: Disclosure and informed consent for non-research clinical uses of artificial intelligence (AI) exist in an ambiguous regulatory landscape that can be difficult for physicians and patients to navigate. Patients often lack awareness of AI in their health care, as many of these technologies are designed for physicians as end-users. For instance, an AI-enabled hemodynamic monitor may assist a care team during a hospital stay unbeknownst to a patient. We conducted case-based focus groups with 18 clinicians, interviews with 18 patients, and focus groups with 42 patients to explore their perspectives on AI information needs, including disclosure and consent, in cardiac care. We found mixed views among physicians regarding disclosing and obtaining informed consent for AI systems involved in patient care. Some physicians thought it was important to disclose AI systems given their potential role in informing clinical decision-making. However, others thought providing this level of transparency may not be feasible given the likely ubiquity of clinical AI in the future. Patients varied considerably in their information preferences; some wanted to be made aware of and provide their consent every time AI was used in their health care. Other patients did not expect any notice and expected their physician to act in their best interest using any tools necessary. Our research highlights opportunities to enhance the transparency of AI systems, including explicit disclosure and consent practices in alignment with physician and patient values. It also underscores the need for strategies aimed at addressing stakeholder information preferences.

Xuan Zhu, PhD, MS – Mayo Clinic; Sarah Minteer, PhD – Mayo Clinic; Dong Whi Yoo, PhD – Kent State University; Maryam Mooghali, MD, MSc – Yale School of Medicine; Jennifer Miller, PhD – Yale School of Medicine; Barbara Barry – Mayo Clinic