Medical schools and health systems are rethinking how artificial intelligence should work in patient care, moving away from AI as a simple administrative assistant toward AI as a partner in clinical judgment. Rather than using AI primarily to reduce paperwork, leading institutions are now building systems where AI and clinicians work together to make diagnoses and treatment decisions more consistent, transparent, and safe (Source 1, 2, 3).

What's Changing in How Hospitals Use AI?

For the past two years, most AI tools in healthcare focused on operational efficiency. They drafted clinical notes, handled intake paperwork, and managed scheduling. These tools made a real difference for clinicians drowning in documentation, but they operated at the margins of actual patient care.

Now, a new phase is emerging. Instead of asking "How can AI help us complete tasks faster?", institutions are asking "How can AI help us make better clinical decisions?" This shift reflects a fundamental recognition: the most important decisions in healthcare involve judgment calls that can't be automated away.

At Duke University School of Medicine, Dean Dr. Mary Klotman emphasized this balance in the school's strategic direction. She stated that the institution has "an opportunity to be a leader in health AI, but the leader in responsible, ethical and safe AI." The school is integrating AI throughout its research, education, and clinical operations, but with careful attention to preserving the human connections that define medicine.

Dr. Mary Klotman

How Should AI and Clinicians Work Together in Decision-Making?

• Layered Review Process: Instead of a single AI recommendation, systems now use multiple layers. A clinician makes an initial judgment, AI structures that reasoning using clinical criteria, and when disagreement occurs, a supervisor reviews both perspectives before making the final call.
• Structured Escalation Paths: When an AI tool and a clinician disagree, the case doesn't just get flagged. It moves through a defined escalation where a second AI perspective and human supervisor both weigh in, creating a richer decision that incorporates multiple viewpoints.
• Continuous System Learning: Each disagreement between clinician and AI reveals patterns that improve the entire system. Over time, these disagreements help calibrate clinicians, refine the AI, and clarify the underlying clinical rules that guide decisions.
• Clear Human Accountability: Humans remain the ultimate decision-makers, especially in ambiguous or higher-risk situations. AI challenges assumptions and surfaces blind spots, but clinicians retain responsibility for the final call.

This orchestration matters as much as the technology itself. Which decisions belong entirely to AI, which belong to humans, and which require both working in sequence must be defined upfront by clinical and technical leadership working together.

What Real-World Evidence Supports This Approach?

UCLA Health recently launched the Innovations and Outcomes Validation of AI (INOVAi) Center, a research hub dedicated to evaluating whether AI tools actually work safely and effectively in real clinical settings. This represents one of the first programs in the nation focused specifically on testing AI implementation across the full lifecycle, from early usability testing through prospective clinical trials.

Early results from UCLA's research on AI scribes, which automatically draft clinical notes during patient visits, showed measurable benefits. The study found that AI scribes significantly reduced the time physicians spent writing clinical notes. Beyond efficiency, physicians reported improvements in cognitive load, work exhaustion, and overall well-being. Importantly, physicians also reported enhanced patient engagement because they could focus more on connecting with patients rather than typing.

"This new center will help address one of the most important gaps in health care AI: knowing whether these tools are safe, effective and useful in real-world clinical practice," said Johnese Spisso, president of UCLA Health and CEO of the UCLA Hospital System.

Johnese Spisso, President of UCLA Health and CEO of the UCLA Hospital System

The INOVAi Center's approach reflects a broader institutional commitment to what researchers call "responsible health AI." Rather than deploying tools and hoping they work, UCLA Health is systematically testing them in the actual environments where patients receive care, then refining both the AI and the clinical workflows based on what the data reveals.

Why Does This Matter for Patient Safety?

In behavioral health and mental healthcare, the stakes of clinical judgment are particularly high. Two experienced clinicians can look at the same patient case and reasonably reach different conclusions about whether someone is a good fit for a particular treatment program. Sometimes that variation reflects appropriate clinical nuance, but often it reveals unclear criteria, inconsistent application of standards, or differences in training and experience.

When AI enters this space, it can either amplify inconsistency or help standardize it. The emerging model uses AI to make clinical reasoning more explicit and consistent, so that every patient benefits from their clinician's best thinking rather than having outcomes depend on which clinician happens to be on duty.

"The goal is not to replace clinicians, but for clinicians and agents to work in tandem to make clinical reasoning more explicit and consistent so that every patient receives the benefit of their clinician's best thinking," explained Parker Phillips, CTO of a digital behavioral health platform.

Parker Phillips, CTO, Digital Behavioral Health Platform

At Duke, this commitment extends to curriculum reform. The school is redesigning medical education to teach students how to "lead in a world where AI will be fully integrated" into their professions. Vice Dean Dr. Aditee Narayan is developing plans to implement ethical AI usage into the medical curriculum, including large language models that process information from students' patient notes to inform their learning.

What Guardrails Are Essential for Safe Implementation?

None of this works without strong safeguards. Experts emphasize that responsible AI implementation requires keeping patient data in secure, compliant environments, maintaining clear human accountability, monitoring performance over time, and creating feedback loops that allow systems to improve safely.

Dr. Mary Klotman also acknowledged a critical concern: the risk that AI might minimize the personal human connections that are fundamental to medicine. "Our profession is human-to-human," she noted. "We must make sure we don't risk that fundamental connection that is so important".

To address this concern, Klotman identified community partnerships as essential. There is significant mistrust and fear about what AI tools might do to patient care, so working with community partners is more important than ever. Duke has already demonstrated collaborative success through initiatives like the Duke Clinical and Translational Science Institute's Seed Health Atlas, an AI-powered tool that helps community members and researchers understand health factors across the nation.

The institutions leading this shift recognize that the next phase of AI in healthcare is not about replacing clinicians or automating away judgment. It's about building systems where human clinicians, supervisors, and AI can all contribute, challenge each other, and improve how decisions are made over time. The organizations that get this right will be the ones most deliberate about where AI fits, where humans lead, and how the two sharpen each other.