Can Healthcare Nudging Algorithms Avoid Becoming Bullshit Bots? Truth and Power in the Age of AI

Abstract: Artificial intelligence (AI) has a proclivity for bullshit, though the more polite term is ‘hallucinate.’ When tasked to persuade humans, large language models like Chat GPT are not above fabricating sources or fudging arguments. While AI lacks the internal motivations to lie, it will seek to persuade without regard to the truth, the textbook definition of bullshit according to Harry Frankfurt’s influential book On Bullshit. AI bullshit is particularly problematic when it comes to the development of healthcare nudging algorithms, which seek to steer patients' behaviors by subtly harnessing biases inherent within human judgment. Using Frankfurt's other book On Truth, this presentation will demonstrate how the twin dangers of AI bullshit and algorithmic nudging denigrate truth and threaten the iatric relationship. This presentation will argue that truth is important in the clinic because falsehoods like bullshit are an attempt to impose our will onto others which, when detected, makes the deceived question who they ought to trust and doubt their ability to distinguish between the real world and a concocted imaginary one. For Frankfurt, the deceiver's problematic motive is his desire to transmit an imaginary world. In every instance it has been deployed, nudging software has been used by the powerful to transmit their will over the powerless--whether that be social media conglomerates over users, corporations over workers, or physicians over patients. This presentation will conclude by describing a more moral path for algorithmic nudging to avoid AI bullshit, by turning the power of nudging AI onto physicians themselves.