Insights from a Delphi Study on Ethical Considerations Emerging with Clinical Trials of Autonomous AI

Abstract: As artificial intelligence (AI) is being integrated into clinical practice through software tools and digital devices, ethical challenges have emerged regarding evaluation of these tools. We convened a Delphi panel to better understand diverse experts’ views about the ethical implications of a clinical trial for autonomous AI detecting diabetic retinopathy (DR) in pediatric patients with diabetes. A panel of fourteen experts in AI and data science, ophthalmology, public health policy, law, bioethics, and patient advocacy arrived at consensus recommendations through an iterative process involving two surveys with open- and closed-ended questions and a final virtual meeting. The panelists focused on clinical trial design for AI for DR, but also addressed wider issues such as the potential use of clinical AI tools to mitigate treatment disparities and the weight given to cost-effectiveness in risk-benefit analysis. While many ethical concerns concerning clinical AI tools may be generally comparable to the those for non-AI clinical tools in several ways (e.g. bias, equity, and access), the panel noted that the advent of AI tools presents an important opportunity for larger systemic changes to help address such concerns. In particular, the panel addressed ways that autonomous AI could be evaluated and used in terms of mitigating health disparities. Furthermore, there was consensus around the importance of transparency regarding the characteristics of the training data population in relation to the intended patient population. These findings are used to inform a set of ethical recommendations for how to consider evaluation of autonomous AI clinical trials.

Alaa Youssef – Stanford Center for Artificial Intelligence in Medicine and Imaging; Risa Wolf – Johns Hopkins University School of Medicine; Danton Char – Anesthesiology – Stanford School of Medicine; Nicole Martinez-Martin – Stanford Center for Biomedical Ethics