Anthropic, the company behind the Claude chatbot, is proposing that major artificial intelligence labs worldwide coordinate a verifiable pause in developing advanced AI systems, warning that rapid technological progress could lead to humans losing control over the technology. The proposal marks a significant moment in the ongoing debate over how to govern increasingly powerful AI, even as rival OpenAI argues that governments, not companies, should make such decisions.

Why Is Anthropic Pushing for a Coordinated Pause?

Anthropic's founders Jack Clark and Marina Favaro, head of the company's research institute, outlined their concerns in a blog post, noting that cutting-edge AI systems are becoming faster at carrying out complex tasks like writing and debugging software code on their own. The company highlighted a particularly concerning scenario: as AI models grow more capable, they could eventually design and develop their own successors, a process known as "recursive self-improvement."

While Anthropic acknowledges that self-building AI could bring major benefits in science and healthcare, the company warned that "it also might increase the risks of humans losing control over AI systems". This concern isn't theoretical. Earlier this year, Anthropic's own Mythos model sent shockwaves through banking and software industries by discovering vulnerabilities in existing code with unprecedented ease.

While Anthropic

The proposed pause would give time for what Anthropic calls "societal structures and alignment research" to catch up with AI advances. In industry terms, alignment means ensuring that AI technology matches human values and intentions, rather than pursuing goals that could harm society.

How Would a Global AI Pause Actually Work?

Anthropic's proposal includes several key mechanisms to make a pause credible and enforceable:

• Verification Systems: Advanced AI labs would need to verify that global rivals have actually stopped or slowed their work, preventing any single company from secretly accelerating development while others pause
• Preventing Bad Actors: The coordination would ensure that "a bad actor could not use the auspices of a coordinated slowdown to jump ahead in secret," according to Anthropic's statement
• Global Mechanism: Without international coordination, the company warned that a slowdown could allow the "least cautious" players to catch up, creating pressure on other companies and governments to abandon safety measures
• Collaborative Research: Anthropic said its internal research institute plans to explore the issue in collaboration with others and "take actions" to help build systems for a credible slowdown, though the company did not provide specific details on implementation

The proposal comes at a critical moment. Researchers at the University of Toronto recently demonstrated how AI tools could be weaponized to create a new kind of AI "worm" that adapts its hacking strategy as it spreads from device to device and takes over computing networks. This research underscores that AI security risks extend beyond the largest language models.

"I think it's really important that people understand that it's not just the biggest, most powerful language models that pose the security concerns," said Nicolas Papernot, the lead researcher on the worm study.

Nicolas Papernot, Researcher at University of Toronto

What's the Government's Response?

Anthropic's call for a pause highlights a broader tension in AI governance. The company's proposal directly contrasts with OpenAI's position, outlined in a report published just before Anthropic's announcement. OpenAI argued that "democratic governments, not private companies acting alone, must ultimately determine the rules, safeguards, and accountability mechanisms". OpenAI further stated that "decisions about the pace of AI innovation should not be left to any one lab, company, or special interest group."

In the United States, where most leading AI labs operate, regulation has moved slowly. A Trump administration executive order issued earlier in the week took a different approach, asking AI labs to voluntarily submit their most capable models for government cybersecurity testing before public release, rather than mandating a pause.

Anthropic itself has faced government pressure. Earlier this year, the company refused to let the US military use its models for domestic surveillance and fully autonomous weapons, prompting backlash that resulted in Anthropic being placed on a national security blacklist, set to take effect later in 2026.

Has the AI Industry Tried This Before?

Calls for AI development pauses are not new. In 2023, a coalition of researchers and tech figures, including Elon Musk (who owns AI lab xAI), backed a push by the non-profit Future of Life Institute to halt AI development for six months to allow time for safety guardrails. That effort gained little traction, and development has only accelerated since.

Anthropic's current proposal differs in that it comes from within the industry itself, backed by a major AI company with significant resources and credibility on safety issues. However, the company's timing is notable: Anthropic and OpenAI are both racing to sell shares on the stock market in initial public offerings (IPOs) that could value Anthropic at nearly a trillion dollars. This context raises questions about whether the pause proposal is primarily motivated by safety concerns or by strategic business positioning.

What Do Cybersecurity Experts Say About AI Risks?

The Toronto researchers' work on AI-powered hacking tools reveals how the cost of mounting cyberattacks has plummeted. Papernot explained that attackers no longer need to focus exclusively on high-value targets like banking systems, hospitals, electricity grids, or water treatment facilities. Instead, even old, unmonitored devices can become launch pads for attacks on critical infrastructure.

"That old laptop you have in your basement that you don't check on regularly doesn't seem like a very high-value target, but it can be used as a launch pad to attack these higher-value targets. Anything connected to the internet is now at risk because of how low the cost has become to mount these cyberattacks," Papernot explained.

Nicolas Papernot, Researcher at University of Toronto

Papernot notified Canadian cybersecurity authorities before releasing his research, which was conducted using open-source AI tools that are cheap and easy for software developers to access and modify. He argued that there should be more collaboration between companies, government agencies, and academic researchers to develop countermeasures as AI-powered hacking tools supercharge the search for computer vulnerabilities.

The debate over Anthropic's pause proposal reflects a fundamental challenge in AI governance: how to balance innovation with safety when the technology is advancing so rapidly that traditional regulatory processes struggle to keep pace. Whether companies, governments, or some combination of both should control the pace of AI development remains one of the most contentious questions in technology policy today.