Artificial intelligence is advancing rapidly in healthcare, shaping critical decisions about diagnosis and treatment, but the systems meant to govern it remain scattered across multiple organizations with no single entity accountable for the entire AI lifecycle. A new study from Northeastern University reveals that governance isn't missing so much as hidden within a patchwork of contracts, partnerships, and informal agreements that can be difficult to trace when problems arise.

What Does Fragmented AI Governance Actually Look Like?

Jennifer Stockton, a researcher at Northeastern University College of Professional Studies, spent her career at the intersection of technology, healthcare, and business before diving into a critical question: how do we ensure innovation moves forward while accountability systems keep pace? What she discovered challenged her initial assumptions.

Rather than finding AI in healthcare completely ungoverned, Stockton found something more complex. Governance is happening, but it's distributed across multiple players operating in the spaces between formal regulatory structures. Responsibility is scattered among regulators, healthcare organizations, technology companies, clinicians, and standards bodies, each playing a role but none fully accountable across the entire AI system's lifecycle.

"Governance is increasingly distributed across regulators, healthcare organizations, technology companies, and standards bodies, with cross-sector partnerships functioning as real governance infrastructure that helps organizations coordinate expertise and adapt faster than formal regulation alone allows," explained Jennifer Stockton, researcher at Northeastern University College of Professional Studies.

Jennifer Stockton, Researcher at Northeastern University College of Professional Studies

This distributed approach creates what Stockton calls "governance in practice." Organizations coordinate through formal regulation, voluntary frameworks, contracts, and cross-sector partnerships. The problem emerges after deployment, when an AI system is actively making clinical decisions in real healthcare environments. If something goes wrong, tracing accountability becomes nearly impossible because responsibility is spread so thin.

How Can Healthcare Organizations Strengthen AI Accountability?

• Design Deliberate Accountability Mechanisms: Rather than waiting for perfect regulation, organizations should intentionally design the relationships and accountability structures that already govern AI in practice, making responsibility clear before deployment.
• Map Cross-Sector Partnerships: Document how regulators, healthcare organizations, technology companies, and standards bodies interact around AI systems, ensuring each party understands its role and obligations.
• Establish Clear Escalation Paths: Create transparent processes for identifying and addressing AI failures in clinical settings, with defined responsibility for investigation and remediation.
• Strengthen Patient Safety Oversight: Build governance structures that prioritize patient safety and public trust alongside innovation, recognizing that these goals are complementary rather than competing.

Why Financial Institutions Are Tightening AI Controls Now

The governance challenge extends beyond healthcare. In the financial sector, major institutions are recognizing that distributed AI governance requires more robust controls. HSBC, one of the world's largest banking organizations, recently announced it is strengthening model-risk, data, and technology controls as it prepares to deploy artificial intelligence more widely.

The bank explicitly acknowledged that generative and agentic AI (systems capable of planning and taking actions with greater autonomy) introduce new operational, cybersecurity, and governance risks. HSBC stated it is strengthening "the data, technology and controls needed to deploy these capabilities safely and at scale".

What makes HSBC's approach noteworthy is how it connects AI governance to existing control frameworks rather than treating AI as a separate program. According to Hemant Julka, group head of digital innovation and partnerships at Emirates NBD Bank, this integration is crucial for institutions scaling AI.

"HSBC is not describing AI governance as a separate Responsible AI programme sitting alongside the bank's existing control environment. It is explicitly connecting AI to the wider risk framework, lifecycle management, third-party oversight and monitoring," noted Hemant Julka, group head of digital innovation and partnerships at Emirates NBD Bank.

Hemant Julka, Group Head of Digital Innovation and Partnerships at Emirates NBD Bank

For banks moving beyond isolated AI experiments and embedding the technology into operational processes and customer decision-making systems, the central question becomes whether established risk structures can accommodate AI while maintaining an enterprise-wide view of control. HSBC is enhancing its group-wide AI oversight, governance, lifecycle management, and risk framework to address both internally developed technology and AI supplied by third parties.

What Role Does Human Accountability Play in AI Governance?

Both healthcare and financial institutions emphasize that human accountability must remain central as AI adoption grows. HSBC reinforced this commitment, stating that "as the Group expands its use of AI, we will maintain human judgement, human decision-making and human accountability at the core".

This principle has practical implications for testing and oversight. Maintaining human accountability doesn't eliminate the need to validate automated decisions. Instead, it places additional importance on testing escalation paths, human-review mechanisms, and whether employees receive sufficient information to identify and challenge erroneous AI outputs.

The bank also identified model risk as an important area of control, noting that evolving regulatory expectations are changing how that risk must be managed. HSBC made enhancements to its risk management framework and monitoring processes during the first half of 2026 to support the responsible adoption and oversight of models using AI and generative AI techniques.

Why Does Distributed Governance Matter for Patient and Customer Safety?

The shift toward distributed governance reflects a fundamental reality: AI systems in healthcare and finance are too complex and interconnected for any single regulator or organization to oversee effectively. However, this distribution creates accountability gaps that can harm patients and customers if not carefully managed.

Stockton's research suggests that the solution isn't necessarily more regulation, but rather deliberately designing the relationships and accountability mechanisms that already govern AI in practice. By making governance visible and intentional, organizations can support innovation while strengthening patient safety and public trust.

As AI systems become more autonomous and capable of making decisions with less human oversight, the importance of clear accountability structures only increases. Both healthcare and financial institutions are recognizing that governance must be built into AI systems from development through deployment and ongoing use, with clear lines of responsibility and mechanisms for identifying and addressing failures when they occur.