TL;DR
An Ontario audit found that more than half of the approved AI scribe systems for healthcare routinely produce inaccurate or incomplete patient notes. The findings raise concerns about AI reliability in medical settings, especially given the minimal weight assigned to accuracy in evaluations.
The Office of the Auditor General of Ontario has found that 60% of the AI note-taking systems approved for use in the province’s healthcare sector frequently produce inaccurate or fabricated patient information, raising concerns over their reliability and safety.
The audit evaluated 20 AI scribe systems used by Ontario healthcare providers, using simulated doctor-patient recordings. It found that nine of these systems fabricated information, such as suggesting treatment plans or symptoms not discussed in recordings. Additionally, 12 systems inserted incorrect medication details into patient notes, and 17 missed key mental health details discussed during consultations.
The evaluation process revealed that the scoring criteria heavily weighted factors unrelated to accuracy; only 4% of the total score was based on the correctness of medical notes, while 30% depended on whether the vendor had a presence in Ontario. This imbalance may have contributed to the selection of systems with poor performance in critical areas.
Why It Matters
This report highlights significant risks in deploying AI tools in healthcare, where inaccuracies can lead to misdiagnosis, improper treatment, or patient harm. The findings suggest that current evaluation methods may inadequately prioritize accuracy and safety, potentially exposing patients to avoidable errors.
As an affiliate, we earn on qualifying purchases.
Background
The use of AI in healthcare documentation has been expanding, with Ontario implementing an AI Scribe program to assist physicians and nurses. The program aims to streamline note-taking but has faced scrutiny after previous reports of AI inaccuracies in medical settings. The recent audit underscores ongoing concerns about AI reliability and evaluation standards in public health systems.
“Inaccurate weightings could result in the selection of vendors whose AI tools may produce inaccurate or biased medical records or lack adequate protection to safeguard sensitive personal health information.”
— Office of the Auditor General of Ontario
“More than 5,000 physicians are participating in the AI Scribe program, and there have been no reports of patient harm linked to the technology so far.”
— Ontario Ministry of Health spokesperson
healthcare AI transcription tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What Remains Unclear
It remains unclear how widespread the use of these AI systems is beyond the evaluated sample, whether corrective measures are being implemented, and how regulators will respond to the audit’s findings. The long-term impact on patient safety and trust in AI tools also remains uncertain.
medical note accuracy verification tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What’s Next
The Ontario Ministry of Health has not yet issued detailed plans following the audit. Future steps may include revising evaluation criteria to prioritize accuracy and safety, mandating manual review of AI-generated notes, or halting the use of certain systems until issues are addressed.
AI scribe systems for healthcare
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
Are AI note-taking systems currently causing patient harm in Ontario?
According to the Ontario Ministry of Health, there have been no reported cases of patient harm linked directly to AI note-taking systems so far, but the audit raises concerns about potential risks due to inaccuracies.
What specific errors did the audit find in the AI systems?
The audit found that 9 out of 20 systems fabricated information, 12 inserted incorrect medication details, and 17 missed key mental health information discussed during consultations.
Will the Ontario government change its evaluation process for AI systems?
The audit criticizes the current scoring criteria for undervaluing accuracy and safety, suggesting that future evaluations may need to be revised to better reflect these priorities, though specific changes have not yet been announced.
Are there any regulations governing AI use in Ontario healthcare?
Regulations are still evolving, and the current evaluation process involves assessments of security, bias control, and compliance, but the recent findings may prompt stricter oversight.
