AI medical scribes are increasingly making serious errors in patient records, from inventing false drug use to recording the wrong body part in cancer diagnoses, and many doctors aren't catching these mistakes before they reach patients. A new privacy watchdog report has exposed how these tools, used by roughly 40% of general practitioners in Australia, can "hallucinate" false symptoms or conditions with little regulatory oversight, putting patients at risk of misdiagnosis, insurance complications, and compromised care.

What Happened to Rebecca Green's Medical Records?

Rebecca Green arrived for her first urologist appointment and consented to AI transcription of her visit. But when she read her post-operative letter after kidney stone surgery in March, she discovered something shocking: the AI scribe had fabricated a claim that she was micro-dosing psychedelic mushrooms, a drug she had never used in her life.

The false entry was particularly alarming because Green was receiving workers' compensation at the time. Any mention of illegal drug use on her medical record could have derailed her case. "For this letter to say something that wasn't based in any evidence but could potentially have huge consequences for me, it was quite scary," she said.

Any

Green's case is far from isolated. A recent report by Digital Rights Watch, an Australian advocacy group, found multiple instances of AI scribes making dangerous errors, including recording the wrong breast in a breast cancer diagnosis and falsely documenting that a patient had epilepsy when they did not.

Why Are Doctors Trusting AI Without Checking?

Australia's peak medical regulator, AHPRA (Australian Health Practitioner Regulation Agency), has made clear that clinicians must verify all AI scribe output for accuracy to meet their professional obligations. Yet many doctors are not doing this consistently.

Research suggests a troubling pattern: doctors check AI transcriptions carefully at first, but after a few weeks of use, they stop verifying the output because they assume it will be correct. Elizabeth Deveny, chief executive of the Consumers Health Forum of Australia, noted this complacency: "There's evidence that for the first few weeks of using AI, people check things pretty carefully and then at a point they stop checking because they assume it will be right".

Some doctors are more diligent. Sean Stevens, a Perth GP who has used AI scribes for three years and serves as the Royal Australian College of GPs' Digital Health and Innovation chair, acknowledged that accuracy has improved but emphasized the need for vigilance. "Three years ago, I would have to correct multiple things on each transcription. These days it's like one correction every third or fourth transcription," he said. However, he warned that AI can still produce plausible-sounding errors: "The classic is, it will get the side of the body incorrect and say right when you've said left, so you need to check things pretty closely, especially things like drug doses".

How to Protect Yourself From AI Scribe Errors?

• Request Explicit Consent: Ensure your doctor asks for your permission to use AI transcription at each appointment, not just once or through a waiting room sign. Consent must be enthusiastic and fully informed, not coerced by appointment refusal.
• Review Your Medical Records: Request copies of letters and notes generated after your appointment and read them carefully for factual errors, especially regarding medications, drug history, or diagnoses that could affect insurance or employment.
• Disclose Sensitive Information Carefully: Be aware that AI is listening during your appointment. If you have concerns about family violence, mental health, or other sensitive issues, you may want to discuss these separately with your doctor without AI transcription.
• Ask Your Doctor About Verification: Directly ask whether your doctor personally reviews AI transcriptions before sending them to other providers or to you. If they cannot confirm this, request manual note-taking instead.

What's the Regulatory Status of AI Scribes?

Despite their widespread use, AI medical scribes in Australia remain largely unregulated. The Therapeutic Goods Administration (TGA), Australia's medical device regulator, has exempted scribes from oversight if they claim to be solely for transcription and summarization. Only tools that suggest diagnoses or treatment plans are classified as medical devices requiring approval.

This regulatory gap is significant: some AI scribes claim to offer clinical suggestions but have never gone through the approval process. The TGA has acknowledged that some scribes may be in breach of the law and has spent the past year reviewing them. The agency said its findings will be released in the coming weeks, with enforcement action to follow.

Lisa Robins, chief executive of Patients Australia, believes all scribes should be regulated because summarizing patient information still involves AI decision-making. However, Sean Stevens expressed concern that extending regulation could stifle innovation in the field.

What Consent Issues Are Patients Facing?

The Digital Rights Watch report found huge discrepancies in how consent is being obtained across Australian medical practices. Some doctors only post a sign in the waiting room, which does not meet privacy law requirements. Others require consent when appointments are booked online, and in some alarming cases, patients are being refused appointments if they do not consent to AI transcription.

Elizabeth Deveny emphasized that consent must be genuine: "Consent must be enthusiastic and fully informed, and patients should feel comfortable asking questions, even if it feels intimidating". Sean Stevens said he asks for consent in every appointment, even with regular patients, because it is critical that patients feel empowered to disclose all health information and know they can refuse AI transcription.

Elizabeth Deveny

The broader concern is that patients may not be forthcoming about sensitive health issues when they know AI is listening. This could lead to incomplete medical histories and compromised clinical decision-making.

What's Next for AI Scribes in Healthcare?

The Digital Rights Watch is calling on Australia's newly established AI Safety Institute to test AI scribes and publish findings to help inform future regulation. Meanwhile, the TGA's review is expected to result in enforcement action against scribes that may be operating outside their regulatory exemption.

For patients like Rebecca Green, the experience has been a wake-up call. She feels fortunate she caught the error, but worries that others may not discover false entries on their medical records, which could have serious repercussions for their health, career, or insurance coverage. Her case underscores a critical truth: as AI becomes more embedded in healthcare, the human responsibility to verify its output has never been more important.