AI is advancing so rapidly that the policymaking process built for a slower world can no longer keep pace. This fundamental mismatch between technological acceleration and regulatory speed is forcing governments worldwide to rethink how they oversee artificial intelligence, with some proposing dramatic new powers including the ability to block dangerous AI model deployments before they reach the public.

Why Is the Speed Gap Between AI and Regulation Such a Problem?

Just a few years ago, AI models struggled to write functional code. Today, frontier AI systems like Claude Mythos Preview can discover thousands of high-severity software vulnerabilities, including flaws in every major operating system and browser. This trajectory suggests the capabilities gap will only widen, bringing with it escalating risks of catastrophic harm. The problem is that government institutions, legislatures, and regulatory bodies operate on timescales measured in years, while AI capabilities are improving on timescales measured in months.

The UK's AI Security Institute (AISI), established to evaluate advanced AI models and advise policymakers, illustrates both the promise and limitations of current governance approaches. AISI has assembled over 100 technical specialists and receives £66 million in annual funding, making it one of the largest concentrations of AI-testing expertise in any government. Yet despite this significant investment, AISI cannot compel companies to submit models for testing, cannot block dangerous models from being released, and cannot impose fines or suspend distribution of models that cause real-world harm. It can only evaluate and advise.

What Specific Catastrophic Risks Are Policymakers Most Concerned About?

Anthropic's newly published Advanced AI Framework identifies four categories of catastrophic risk that require urgent government attention. These risks escalate as AI systems become more capable and are deployed more widely across critical infrastructure and sensitive domains.

• Biological Risk: AI systems with capabilities in drug discovery and molecular design could be repurposed to make it cheaper and easier for attackers to develop dangerous biological weapons without proper safeguards in place.
• Cyber Risk: Frontier AI models can now identify critical software vulnerabilities at scale, which could be used defensively to secure systems but also raises the stakes for protecting essential infrastructure like hospitals and energy grids.
• Loss of Control Risk: As AI systems improve, it becomes increasingly difficult to ensure they operate within their developers' intended parameters and do not act autonomously outside human oversight.
• Automated R&D Risk: AI systems are beginning to automate the research and development of AI itself, which could amplify all three of the above risks in a feedback loop.

These risks are not theoretical. AISI's evaluation of Anthropic's Claude Mythos in spring 2026 found a sharp rise in the model's ability to carry out cyberattacks compared with previous frontier systems, giving the UK government and national security communities an independent, early read on a qualitative jump in offensive capability.

How Should Governments Regulate Frontier AI Models?

Anthropic proposes a framework that would apply only to the most powerful AI systems, those trained using more than 10²⁵ floating-point operations (FLOPs), developed by companies earning more than $500 million in AI-related revenue or spending more than $1 billion on AI research and development. The framework includes several key mechanisms designed to balance safety with innovation.

• Transparency Requirements: Frontier developers should test their models, publish summaries of results, describe their safety frameworks publicly, and regularly engage independent evaluators to review their risk assessments and overall risk posture.
• Independent Evaluation: Governments and industry should develop an ecosystem of qualified independent evaluators by setting standards, providing funding, and ensuring they have sufficient access to frontier models for rigorous testing.
• Security Standards: Model weights and training infrastructure must be protected against cyberattacks from both external actors and insider threats, with developers describing their security programs publicly and sharing details with designated government agencies on request.
• Enforcement Authority: Governments should have legal authority to block or deter deployment of models posing significant catastrophic risk, with civil penalties tied to global annual revenue that escalate with repeated violations.

The framework also proposes resilience measures beyond regulation. For biological risks, these include gene synthesis screening, early-warning biosurveillance systems, and stockpiling protective equipment. For cyber risks, the recommendations focus on hardening software infrastructure, providing technical support to critical infrastructure operators, and replacing legacy systems.

Importantly, Anthropic argues that transparency alone is no longer sufficient. Several recent state laws, including those in California and New York, require companies to describe their safety practices publicly. However, the rapid pace of AI acceleration means that government must play a more substantial role through enforceable standards and the power to prevent dangerous deployments.

What Are the Gaps in Current Governance Approaches?

While AISI represents a significant investment in independent AI evaluation capacity, its current research agenda focuses narrowly on catastrophic and national security risks. This leaves a wide range of emerging harms outside its scope, including child safety, non-consensual intimate imagery generation, bias and discrimination in high-stakes decisions, and environmental impacts of AI systems.

This gap became apparent following the Grok scandal in January 2026, when an AI chatbot was used to generate millions of sexualized images of real people on the social media platform X. Despite this incident, the UK government had no independent way of assessing how robustly the next generation of chatbots would be safeguarded against similar misuse. AISI had previously run programs on non-consensual imagery abuse and child safety, but these have been cut from its current workplan.

Meanwhile, other nations are taking different approaches to AI governance. Qatar's National Artificial Intelligence Strategy, adopted in October 2019, functions as a guiding policy rather than prescriptive law with immediate penalties. The strategy emphasizes building a skilled workforce, enabling secure and ethical data access, fostering business opportunities, and developing ethical guidelines reflecting both international best practices and Qatar's unique cultural values. However, the strategy itself does not impose penalties or establish liability regimes; instead, it provides a framework that will guide future specific legal and regulatory instruments.

Steps Organizations Should Take to Prepare for Evolving AI Regulation

As regulatory frameworks continue to develop globally, organizations working with AI systems should take proactive steps to align with emerging governance expectations.

• Monitor Regulatory Developments: Designate a team to actively track announcements from relevant authorities regarding new data protection laws, AI procurement guidelines, and industry-specific safety standards, particularly in jurisdictions where your organization operates or plans to expand.
• Audit Data Governance Practices: Review internal data governance frameworks, especially concerning data used for AI training and deployment, to ensure alignment with existing data protection laws and general principles of privacy, security, and responsible data handling.
• Develop Internal Ethical Guidelines: Begin developing internal guidelines or codes of conduct for AI development and deployment that reflect ethical principles like fairness, transparency, and accountability, even before these become legally mandated in your jurisdiction.

The fundamental challenge facing policymakers is that AI capabilities are improving rapidly over the coming months, and governance needs to keep pace. Yet the institutions and processes designed to regulate technology were built for a different era. Whether through Anthropic's proposed framework, AISI's evaluation capacity, or Qatar's strategic guidance, governments are attempting to bridge this gap. The question is whether they can move fast enough.