Liability Risks of Ambient Clinical Workfows With Artifcial Intelligence for Clinicians, Hospitals, and Manufacturers

Policy and Practice Liability Risks of Ambient Clinical Workflows With Artificial Intelligence for Clinicians, Hospitals, and Manufacturers Sara Gerke, Dipl-Jur Univ, MA 1 ; David A. Simon, PhD, JD, LLM 2 ; and Benjamin R. Roman, MD, MBA, MSHP 3 DOI https://doi.org/10.1200/OP-24-01060 In August 2024, the nation’s largest nonprofit integrated health care provider, Kaiser Permanente, announced that clinicians would have access to an ambient clinical documentation scribe: an assisted clinical documentation tool that uses artificial intelligence (AI) to securely summarize relevant medical information from spoken, natural conversations (also called ambient clinical documentation or AI scribes). 1 After automatically summarizing the encounter, the AI scribe sends the summary to the clinician for review. Ambient clinical documentation scribes are now offered by some of the fastest-growing AI companies in health care, with significant venture capital funding and an impressive roster of health system customers. Technologies such as ambient clinical documentation and other generative AI tools may improve care and lessen clinician burnout by reducing documentation burdens. But they also raise the question of who is responsible when AI-generated patient information is inaccurate, especially when those errors cause injury to a patient. This question is particularly acute in cancer care, where there is a unique set of terminology for each of the more than 400 types of cancer, leading to an increased chance of documentation error, and where decisions on the basis of the assumption of information accuracy can be life-altering. 2 AI transcription tools in their current versions are not considered regulated medical devices under the US Federal Food, Drug, and Cosmetic Act. 3 Unless this changes, the responsibility falls to stakeholders other than the US Food and Drug Administration (FDA) to ensure the technology’s safety and efficacy. In this article, we analyze the AI governance responsibilities and potential tort liability for clinicians, hospitals, and manufacturers using AI for clinical notetaking and suggest several potential ways to address them. TYPES OF ERRORS Hospitals and clinicians are likely to face an evolving landscape of AI clinical note-taking tools with powerful capabilities and functions, but also the potential for making mistakes. The current gold standard of human-written clinical notes of course also has shortcomings. 3 According to a recent estimate, much of the content in patients’medical records is duplicated, resulting from copy-pasting previous clinical notes, potentially leading to medical errors. 4 Ambient clinical documentation technologies can make three types of errors. First, it may omit information by failing to recognize, transcribe, and move information to a draft note. Second, it may incorrectly document information, such as the wrong name or dose of a medication. Third, it can hallucinate information, fabricating new and inaccurate information. Although these errors are similar to human errors made without AI scribes, the mechanism for addressing them will be different. Although clinicians may have a better chance of addressing errors made by AI scribes because they will have a transcript of the encounter, this remains unproven and at the very least, clinicians will have to adopt new workflows. The potential errors of AI scribes are nontrivial. For example, one transcription tool has recently come under criticism for inventing several sentences of text, including the patient’s medical treatment, race, and medication. 4 Documentation errors are particularly problematic in highly specialized fields such as oncology, where clinicians rely on accurate information about the patient’s medical history, medication or other treatments, as well as adverse events and Accepted June 20, 2025 Published August 1, 2025 JCO Oncol Pract 00:1-5 © 2025 by American Society of Clinical Oncology View Online Article ascopubs.org/journal/op | Volume nnn, Issue nnn |1 Downloaded from ascopubs.org by 37.228.250.230 on November 24, 2025 from 037.228.250.230 Copyright © 2025 American Society of Clinical Oncology. All rights reserved. response to therapy for life-and-death treatment decisions. Errors in this context may create significant liability risks. HUMANS IN THE LOOP During certain development phases of the technology, manufacturers have humans in the loop to review accuracy and note quality, including carrying out major updates to their AI models. However, once the models reach clinical practice, manufacturers do not use a third-party clinician reviewer—the responsibility falls to the clinician reviewing and finalizing the note. For example, AI tools offered by companies such as Microsoft and Abridge record and transcribe the entire patient-clinician conversation and summarize the transcript into a draft note. 5 The note is then sent to the clinician, who is responsible for reviewing the draft, making changes, and finalizing the note. On the one hand, such technology may actually improve clinical documentation, capturingmoreorbetterinformation than the clinician alone, such as adverse events for patients receiving drugs in oncology clinical trials. On the other hand, this technology may lead to new or different mistakes by clinicians. For example, a clinician may not recognize an error in the AI-drafted note for a variety of reasons, including language/cultural differences, recording quality, or simple interpretation. And because clinicians have high caseloads and significant paperwork obligations, they risk automation bias 6 : as heavy reliance on technology reduces cognitive load, physicians may be less likely to read every word of the draft note before finalizing it. Clinicians may be sensitive to the risk of automation bias during implementation and likely would not use the tool at all if they did not trust it. However, once a clinician gets used to using the technology, the risk of automation bias increases. In the future, efficiency demands may drive hospitals and clinicians to implement AI clinical note-taking tools that skip clinician review altogether. Although it is unclear whether sufficient trust in this technology could ever drive adoption of fully automated documentation, its theoretical use could exacerbate the existing risks to patient care. 18 LIABILITY RISKS FOR CLINICIANS, HOSPITALS, AND MANUFACTURERS Consider the following vignette: A 68-year-old man with metastatic non–small cell lung cancer had his chemotherapy treatment notes transcribed using ambient clinical documentation. The technology incorrectly documented that he had failed first-line and second-line therapies, when in fact he had only failed first-line therapies, and the second-line therapies were being discussed. The oncologist reviewed the note, failed to notice the mistake, and finalized it in the patient’s record. The patient then switched to a new oncologist, who relied on these notes and initiated third-line therapy with immunotherapy prematurely. Instead of receiving a more appropriate second-line therapy, this third-line therapy resulted in severe immune-related pneumonitis and hospitalization. This error deprived him of the full benefitofsecond-linetherapyandworsened his prognosis. Who is likely to be held liable for the patient’s harm? The clinicians, the hospital, and/or the AI manufacturer? The basic principles of tort law would likely apply, but the precise scope of the liability risks for each actor is determined by a number of factors. Clinician Liability Clinicians are liable when their unreasonable actions cause injury to patients. Reasonableness is determined by the existing standard of care, which can be influenced by various factors, including the nature of the mistake, how difficult it would be to spot it, and the risk it posed. For example, in the vignette case in which the first medical oncologist reviewed and approved the AI-generated note, they could be held liable under a negligence theory for the patient’s injury if a reasonable clinician would have noticed the note’s inaccuracy. Importantly, what is reasonable for the physician is often determined by what is customary in that clinician’s particular specialty (which is here medical oncology). To defend the patient’s medical malpractice claim, the first medical oncologist could argue that the subsequent treating oncologist was liable because a reasonable clinician would confirm information with a patient before treating them. In other words, a reasonable oncologist would have verified what drugs the patient was previously treated with before proceeding with third-line therapy. Successfully proving such a defense, however, would require the first medical oncologist to introduce evidence that the second oncologist knew or should have known the note was inaccurate— perhaps through inconsistencies in the notes or the information the first medical oncologist or patient provided to the second medical oncologist directly. Even if this argument succeeds, the first oncologist could still be liable for some or all of the injury because his negligence was sufficient to cause it. Finally, the oncologists might argue that the cause of harm was the plaintiff’s preexisting cancer, not their negligence. While this argument cannot defeat a patient’s case by itself, it can block some claims and reduce damages for others. 19 For example, some courts require that the plaintiff must show that the probability of a good outcome without negligence was greater than 50 percent. If they can, they are entitled to all damages, including from the underlying cancer. Other jurisdictions allow claims as long as the plaintiff can show any reduced recovery but limit damages only to the proportionate loss. Whatever requirements courts impose, the fact that a physician made a patient worse is not a per se bar to a viable tort claim. 8 2| © 2025 by American Society of Clinical Oncology Gerke, Simon, and Roman Downloaded from ascopubs.org by 37.228.250.230 on November 24, 2025 from 037.228.250.230 Copyright © 2025 American Society of Clinical Oncology. All rights reserved. Hospital Liability The hospital where the first medical oncologist or the second medical oncologist practices may also be liable on at least two theories. First, the hospital could be liable if the medical oncologists were negligent. Under a doctrine known as respondeat superior, employers are liable for the torts of their employees committed within the scope of employment or with the employer’s property. 7 If the clinicians were employees, the hospital would be liable for their negligence since the first medical oncologist used the AI tool and the second medical oncologist relied on its notes within the scope of their employment. 20 Second, even if neither the first medical oncologist nor the second medical oncologist was negligent, the hospital where the first medical oncologist practices could still be liable if it failed to meet its duty to exercise reasonable care in adopting, implementing, or monitoring the use of the AI tool. A hospital could breach its duty of care, for example, if it adopts an AI clinical note-taking tool with a higher error rate than its current practice (ie, human note-taking). However, properly validated systems that reduce the risk associated with the current practice may make the AI tool reasonable to implement. The hospital may also be liable if it validated it in one practice area (eg, internal medicine) and implemented it in other areas (eg, oncology), or if it did not properly train clinicians on how to use it. For example, if the hospital in the vignette described above did not instruct the first medical oncologist using the AI tool about the potential for mistakes or the type of mistakes likely to arise, the hospital may be liable for harm caused by this failure. Hospitals may defend these liability claims by showing that the AI tool was properly validated, that the manufacturer did not provide adequate training before adoption, or that the tool was defective for some reason, such as not meeting the manufacturer’s own specifications. However, any defense that asserts the AI manufacturer’s liability may already be limited by contract law in two unrelated ways. First, contract law imposes product liability on the producer of a good but not a service; if a court finds AI is a service, then the AI manufacturer will be immune from contract-based product liability suits. Second, manufacturers may limit their own liability through contracts with hospitals that use their products. 8 Manufacturer Liability Manufacturers of AI clinical note-taking tools could also face liability for injuries to patients in scenarios such as the ones in the vignette under negligence or products liability law. Like hospitals, the manufacturer can be negligent in training the clinicians who use the tool. Unlike hospitals and clinicians, however, manufacturers could be liable even if their practices matched an industry custom. Suppose, for example, that the manufacturer hired a third-party firm located in a foreign country that employed non-native English speakers to conduct the initial quality review of their tool, before deployment to clinicians. If a reasonable manufacturer had hired a native English speaker or provided appropriate training to the first medical oncologist in the vignette that would have resulted in them catching the error, then the manufacturer could be liable. Manufacturers could also be liable if their products are defective. Failing to provide adequate warnings or make the AI clinical note-taking tool safe for the intended uses of the product, for example, may create liability for the manufacturer. 9 Warnings must be provided to consumers of the product, in this case, to the first oncologist—and possibly even to the second oncologist who reviews the completed patient history and notes. Manufacturers who fail to disclose risks, validation techniques, and other information could face liability if a patient is injured from their use. 10 For example, the manufacturer could be liable if it does not inform the hospital or the first medical oncologist using the AI of its risks, including the risk that the tool may not work as well in certain settings or regarding information from certain specialties, such as oncology, that differ from those used to train or validate it. Likewise, manufacturers that do not properly validate their tools can face liability. Because many AI clinical note-taking tools are not likely to be classified as FDA-regulated devices, what will be sufficient validation in the vignette will depend on either user (eg, health care provider) expectations about the accuracy of the tool or the potential benefit of the tool relative to its risks, as well as whether any alternate safer designs were available. 11 MITIGATING ERRORS AND LIABILITY Clinicians and Hospitals Hospitals and the clinicians they employ who use AI clinical note-taking tools can address liability risks in several ways. Clinicians and hospitals should work closely with AI manufacturers to ensure that clinicians are properly trained to use the technology across different treatment settings. Training should help clinicians to spot common AI mistakes or those that look correct but are instead incorrect or ambiguous. This could reduce the risk that the clinician approves a note that contains an error. Importantly, training should not be a one-off: clinicians should be retrained when manufacturers update their underlying AI models, the user interface, or workflows that affects documentation. Clinicians need to trust that the technology is safe and effective; this can be queried with a validated clinician survey, such as the Theory of trust and acceptance of artificial intelligence technology (TRrAAIT). 12 Hospitals should also be methodical when deciding on and implementing AI technologies. Multiple general frameworks JCO Oncology Practice ascopubs.org/journal/op | Volume nnn, Issue nnn |3 AI Clinical Notetaking Risks for Oncology Downloaded from ascopubs.org by 37.228.250.230 on November 24, 2025 from 037.228.250.230 Copyright © 2025 American Society of Clinical Oncology. All rights reserved. exist for responsible AI governance and quality assurance. Recently, researchers have developed one for oncology, with all of its unique clinical and operational nuances. 13 Similar to pharmacy and therapeutics committees for drugs, hospitals can establish committees or task forces that are responsible for AI lifecycle governance, regulatory compliance, and risk management. 14 These bodies would be responsible for registering, evaluating, and monitoring AI models used across a variety of settings including in clinical practice. They would also be responsible for capturing and evaluating care quality issues that arise because of AI technologies. Regarding ambient clinical documentation in particular, hospitals and manufacturers should establish a plan to validate and continually monitor for errors and safety issues in the AI models’output and finalized clinical notes across different clinical settings. One resource for assessment is the Physician Documentation Quality instrument (PDQI), which is an assessment tool that can be used to evaluate the quality of AI-generated notes. 15,16 These kinds of ongoing assessments may also increase awareness of errors on the part of clinicians who will be engaged in the evaluation process. However, not all hospitals have sufficient resources to create and maintain AI governance bodies. To mitigate this problem, hospitals with fewer resources could partner with wellresourced hospitals on specific AI applications and could attempt to negotiate favorable indemnification provisions in contracts with manufacturers. For both legal and ethical reasons, hospitals and clinicians should be cautious about the potential future state in which AI clinical note-taking occurs without clinician review. As a legal matter, tort law liability for such AI tools is still unsettled. As an ethical matter, hospitals and clinicians should consider the desirability of having a nonhuman AI tool decide what information is relevant and goes into the medical records, especially in highly specialized fields such as oncology, where there can be profound effects on patient care. Therefore, maintaining a human in the loop may be the most prudent course of action until tort law becomes more settled and the usefulness of such tools is rigorously proven. 17 Manufacturers The primary responsibility for evaluating AI models for clinical documentation should lie with the manufacturers. They can address some liability risks in ways similar to clinicians and hospitals. For instance, properly validating the technology in specific clinical settings is crucial to reducing liability risks. The manufacturers should continue to evaluate and validate the tool to ensure it is working at least as well as when it was sold to the end user. When human review is being used toevaluate model updates, manufacturers should be careful in hiring and training these reviewers. Manufacturers should carefully consider trying to outsource any human review functions to countries where the language of the reviewer matches those of the clinicians. 5 Because indemnification and hold harmless provisions are only as good as those signing them, manufacturers that do outsource such functions must pay close attention to the choice of law provisions and to the financial wherewithal of firms they contract with. Finally, clinical documentation tools should only be marketed in ways that are consistent with their actual functions and abilities. For example, if there is an error rate of 20% when used in oncology but 2% when used in primary care, the manufacturer should not market the product as having a low error rate. Manufacturers should market the tool for only those specific uses in practice areas or settings in which the tool has been validated. Similarly, manufacturers should disclose the risks and limitations of the tool orally and in writing to hospitals and clinicians. In conclusion, ambient clinical documentation with AI tools has the potential to transform the lives of clinicians, reducing burnout and improving the ability of clinicians to focus on being engaged in conversation with the patient before them. We have described potential issues with the tools’safety and the related liability concerns. We have also suggested various ways that clinicians, hospitals, and manufacturers can mitigate these issues. The key for ensuring the appropriate use and safety of these tools is the need for clinicians, hospitals, and manufacturers to work together to understand how this technology is shaping and improving medical care. These risk-reduction strategies focus on building safer products and providing complete and accurate information to clinicians and hospitals. If manufacturers, clinicians, and hospitals can safely implement this tool with robust AI governance, it may serve as a roadmap for implementation of other AI tools such as Clinical Decision Support (CDS) in clinical workflow. And it should improve patient care. AFFILIATIONS 1 College of Law and European Union Center, University of Illinois Urbana-Champaign, Champaign, IL 2 Northeastern University School of Law, Boston, MA 3 Department of Strategy & Innovation and Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY CORRESPONDING AUTHOR David A. Simon, PhD, JD, LLM; e-mail: [email protected]. DISCLAIMER The views and opinions expressed are those of the authors only and do not necessarily reflect those of their organizations or the grantors—the European Union, the Health and Digital Executive Agency, or the National Institutes of Health. Neither the European Union nor the 4| © 2025 by American Society of Clinical Oncology Gerke, Simon, and Roman Downloaded from ascopubs.org by 37.228.250.230 on November 24, 2025 from 037.228.250.230 Copyright © 2025 American Society of Clinical Oncology. All rights reserved. granting authorities can be held responsible for them. The funder had no role in preparing, reviewing, or approving the manuscript or in deciding to submit it for publication. SUPPORT D.S. and S.G.’s work was supported by grant 101057321 from the European Union. B.R.’s work was supported by the Memorial Sloan Kettering Cancer Center Support Grant P30 CA008748 from the National Institutes of Health/National Cancer Institute. AUTHORS’DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST Disclosures provided by the authors are available with this article at DOI https://doi.org/10.1200/OP-24-01060. AUTHOR CONTRIBUTIONS Conception and design: Sara Gerke, David A. Simon Financial support: Sara Gerke Administrative support: All authors Provision of study materials or patients: Benjamin R. Roman Collection and assembly of data: All authors Data analysis and interpretation: All authors Manuscript writing: All authors Final approval of manuscript: All authors Accountable for all aspects of the work: All authors REFERENCES 1. Kaiser Permanente Improves Member Experience With AI-Enabled Clinical Technology, 2024. https://about.kaiserpermanente.org/news/press-release-archive/kaiser-permanente-improvesmember-experience-with-ai-enabled-clinical-technology 2. NCI: Enhancing Patient Navigation with Technology to Improve Equity in Cancer Care 3. Chen C, Thornton JE: AI-generated clinical summaries. JAMA 331:1967-1968, 2024 4. Steinkamp J, Kantrowitz JJ, Airan-Javia S: Prevalence and sources of duplicate information in the electronic medical record. JAMA Netw Open 5:e2233348, 2022 5. Trang B: As AI promises to revolutionize medical note-taking, concerns mount about accuracy and harm. STAT, 2023. https://www.statnews.com/2023/03/28/hospitals-ai-artificial-intelligencehealth-microsoft-nuance-notes/ 6. Lyell D, Coiera E: Automation bias and verification complexity: A systematic review. J Am Med Inform Assoc 24:423-431, 2017 7. Lisa M. v. Henry Mayo Newhall Meml. Hosp., 907 P.2d 358, 360 (Cal. 1995) 8. Duffourc M, Gerke S: Decoding U.S. Tort Liability in Healthcare’s Black-Box AI Era: Lessons from the European Union, 27 Stan. Tech. L. Rev. 1 (2024) 9. Simon DA: Off-label preemption. Wis L Rev 2024:1079, 2024 10. Simon DA, Kesselheim AS: Physician and device manufacturer tort liability for remote patient monitoring devices, in Shachar C, Kramer DB, Cohen IG, et al (eds): Digital Health Care Outside of Traditional Clinical Settings: Ethical, Legal, and Regulatory Challenges and Opportunities. Cambridge University Press, Cambridge, UK, 2024. pp 109-122 11. Mikolajczyk v. Ford Motor Co., 901 N.E.2d 329, 352 (Ill. 2008), Opinion Modified on Denial of Reh’g (Dec. 18, 2008) 12. Stevens AF, Stetson P: Theory of trust and acceptance of artificial intelligence technology (TrAAIT): An instrument to assess clinician trust and acceptance of artificial intelligence. J Biomed Inform 148:104550, 2023 13. Stetson PD, Choy J, Summerville N, et al: Responsible artificial intelligence governance in oncology. NPJ Digit Med 8:407, 2025 14. Simon DA: Off-label innovation. Ga L Rev 56:701-786, 2022 15. Stetson PD, Bakken S, Wrenn JO, et al: Assessing electronic note quality using the physician documentation quality instrument (PDQI-9). Appl Clin Inform 3:164-174, 2012 16. Croxford E, Gao Y, Pellegrino N, et al: Development and validation of the provider documentation summarization quality instrument for large language models. J Am Med Inform Assoc 32: 1050-1060, 2025 17. Cahill RA, Duffourc M, Gerke S: Surgical video data—“In,”“out,”or “shake it all about”the medical record. Ann Surg 281:382-384, 2025 18. Simon DA: Patients versus profits. Vand J Ent Tech L 2025. https://papers.ssrn.com/sol3/papers.cfm?abstract_id55296927 19. Lord v. Lovett, 770 A.2d 1103 (N.H. 2001). 20. Simon DA, Cohen IG, Balatbat C, et al: The hospital-at-home presents novel liabilities for physicians, hospitals, caregivers, and patients. Nat Med 28:438-441, 2022 JCO Oncology Practice ascopubs.org/journal/op | Volume nnn, Issue nnn |5 AI Clinical Notetaking Risks for Oncology Downloaded from ascopubs.org by 37.228.250.230 on November 24, 2025 from 037.228.250.230 Copyright © 2025 American Society of Clinical Oncology. All rights reserved. AUTHORS’DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST Liability Risks of Ambient Clinical Workflows With Artificial Intelligence for Clinicians, Hospitals, and Manufacturers The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I 5Immediate Family Member, Inst 5My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO’s conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/op/authors/author-center. Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments). David A. Simon Stock and Other Ownership Interests: Pfizer, Procter & Gamble Honoraria: Chan Zuckerberg Initiative, George Mason University Other Relationship: Arnold Ventures, Simon Law Group No other potential conflicts of interest were reported. © 2025 by American Society of Clinical Oncology Gerke, Simon, and Roman Downloaded from ascopubs.org by 37.228.250.230 on November 24, 2025 from 037.228.250.230 Copyright © 2025 American Society of Clinical Oncology. All rights reserved.