AI healthcare tools are expanding so quickly that regulators struggle to establish safeguards, even as the technology proves valuable for imaging, documentation, and patient care. Just 18 months ago, finding legitimate, high-quality AI tools for healthcare was difficult; today they're everywhere, from Epic Systems' integrated applications to Microsoft's Dragon Copilot and Amazon's Health AI program.

What AI Tools Are Actually Doing in Hospitals and Clinics Right Now?

AI is reshaping healthcare workflows across multiple domains. On the clinical side, ambient scribe technology listens during patient visits, documents encounters, and summarizes what happens. These tools have evolved significantly beyond simple note-taking. Modern versions now enter information directly into medical records, pre-load prescription renewals for clinician approval, check for dangerous drug interactions, and surface relevant medical research.

In medical imaging, AI is delivering measurable improvements. Dr. Richard Bruce from UW-Madison's Department of Radiology noted that AI is producing better image quality from available data, reducing radiation doses, and catching more errors. The technology also performs real-time quality checks during ultrasounds, automates image labeling, and coaches technicians on technique improvement.

For patients, consumer-facing AI tools now offer new capabilities. Amazon's Health AI program can read patient records, explain lab results, answer symptom questions, and route requests to providers. ChatGPT also offers a consumer health tool for analyzing medical records, though these tools are explicitly not designed to provide medical diagnoses.

Healthcare organizations are also adopting unified imaging platforms that combine storage, viewing, AI-assisted reporting, and electronic health record (EHR) integration. These cloud-based systems reduce manual handoffs between departments and keep imaging data connected throughout the diagnostic workflow.

Why Are Regulators Worried About These Tools?

Despite their benefits, AI healthcare tools have made high-profile mistakes that underscore the need for oversight. Last month, an AI scribe incorrectly recorded that a patient had taken psychedelic mushrooms, even though this was never mentioned during the visit. The error was entered into the patient's health record and later surfaced in a letter to her primary care doctor as a possible explanation for her symptoms.

This wasn't an isolated incident. Watchdog reviewers found other cases where AI scribes misidentified breast cancer diagnoses, incorrectly documented epilepsy in patients who didn't have it, and one Google bot that fabricated an entirely fictional body part by combining two different anatomical structures.

"In the past, that's really all it did, but those have significantly advanced, to the point where now they are entering information into the medical record itself, they're being built into automatic prescription renewals so they can pre-load renewals for the clinician's sign-off," said Frank Meyers, director of regulatory innovation and member services at the Federation of State Medical Boards.

Frank Meyers, Director of Regulatory Innovation and Member Services, Federation of State Medical Boards

How Are States Trying to Regulate AI Healthcare?

With federal oversight still developing, individual states are taking action to protect patients and test new regulatory approaches. Six states currently have active AI healthcare sandbox pilots, which allow companies to deploy AI tools within a controlled framework while regulators observe outcomes and gather data.

• Active Sandbox States: Arizona, Connecticut, Delaware, Kansas, Texas, and Utah are currently running AI healthcare pilots that waive certain regulatory requirements to enable experimentation and learning.
• Proposed Sandbox Programs: Eight additional states have proposed sandbox programs, though they differ substantially in scope and requirements from state to state.
• Privacy Protections: States are enacting sectoral privacy laws that give consumers access to lists of third parties their health data is sold to and provide new protections against data-based profiling by AI systems.

Utah's sandbox, for example, is waiving certain medical board regulatory requirements to allow unlicensed AI-related activity within the pilot's scope. Initial findings appear promising, though the effort remains in its early stages.

"The main takeaway for sandboxes is they're primarily designed to allow experimentation of these tools, both by the developers but by the regulators to understand what would be the potential best mechanism for regulating these things," noted Frank Meyers.

Frank Meyers, Director of Regulatory Innovation and Member Services, Federation of State Medical Boards

How to Navigate AI Healthcare Tools Safely?

As AI tools proliferate in healthcare settings, both providers and patients should understand how to use them responsibly and verify their outputs.

• Verify AI-Generated Documentation: Clinicians should carefully review any documentation generated by AI scribes or ambient recording tools before it enters the patient's official medical record, checking for factual errors or misinterpretations.
• Understand Tool Limitations: Consumer-facing AI health tools like ChatGPT are not diagnostic tools and should not be used as substitutes for medical advice; they're designed to help explain existing medical information, not diagnose new conditions.
• Check Your Data Privacy: Patients should ask their healthcare providers which third parties have access to their health data and review state-level privacy protections that give them the right to know where their information is being shared.

What Happens Next in AI Healthcare Regulation?

The regulatory landscape remains uncertain. When asked what comes after the initial one-year sandbox periods, regulators acknowledged they don't yet know the answer. Nobody has a clear roadmap for how to scale these pilots into permanent regulatory frameworks.

What's clear is that the pace of AI adoption in healthcare is outstripping the pace of regulation. Frank Meyers noted that just 18 months ago, finding high-level AI tools from legitimate healthcare providers was difficult. Today, they're ubiquitous. As Wisconsin lawmakers and regulators across the country grapple with oversight, the technology continues to advance, creating both opportunities for better patient care and risks that require careful monitoring and human oversight.