Why the U.S. Government's Demand for 'Unhackable' AI May Be Impossible to Meet
The U.S. government is asking Anthropic to do something that may be technically impossible: create large language models (LLMs), or AI systems trained on vast amounts of text data, that are completely unhackable before releasing them internationally. This demand has triggered a major conflict between the Trump administration and the AI company, with over 100 security experts now pushing back against the government's approach.
What Sparked the Government's Demand?
The conflict began when Amazon, Anthropic's largest investor with roughly $13 billion committed to the company, reported that its researchers had used specific prompts to extract restricted cybersecurity information from Anthropic's newest model, Claude Fable 5. Amazon's chief executive Andy Jassy personally alerted Treasury Secretary Scott Bessent about the finding. The Trump administration interpreted this as evidence that Anthropic had released a model with a serious security vulnerability, or "jailbreak," without government approval.
An administration official told Axios that Anthropic had "screwed us" by ignoring Trump's recent cyber executive order, which called for voluntary government oversight of AI models. Anthropic had welcomed the proposal but released Fable 5 without waiting for the designated government clearinghouse to be established and provide explicit approval.
Why Are Security Experts Calling This Demand Unrealistic?
The core problem is that no AI model can be made completely unhackable before release. OpenAI has already warned that prompt injection, a related hacking method, may never be fully solved. There is no known fix for LLM security that would guarantee immunity from all attacks. This is a fundamental limitation of how these systems work, not a failure of Anthropic's engineering.
Anthropic disputes the severity of Amazon's findings. The company argues that the bypass produced only "previously known, minor vulnerabilities" that other public models, including OpenAI's GPT-5.5, can also find without any special bypass technique. Katie Moussouris, chief executive of cybersecurity firm Luta Security, reviewed Amazon's research and told the Wall Street Journal that the finding was not actually a jailbreak at all, but rather "standard defence-oriented prompting".
Over 100 security experts and tech industry executives have published an open letter calling for export controls on Fable and Mythos, Anthropic's newest models, to be lifted. The signatories include prominent figures such as Alex Stamos of Corridor, Rachel Tobac of SocialProof Security, Katie Moussouris of Luta Security, Dan Lorenc of Chainguard, and Joe Levy of Sophos. Their core argument is that Anthropic's models are not uniquely vulnerable compared to competitors.
What Are the Real Risks of This Government Approach?
The shutdown of Fable 5 and Mythos 5 has created an immediate problem. On June 9, Anthropic launched Fable 5 as its flagship consumer model. Three days later, the U.S. Commerce Department issued an export control directive banning foreign nationals from accessing both Fable 5 and Mythos 5, an unrestricted version available to vetted organizations under a program called "Project Glasswing". Because Anthropic could not filter foreign nationals from U.S. users in real time, the company disabled both models worldwide for every customer at 5:21 p.m. Eastern Time.
This means hundreds of millions of paying customers, from coding teams to research labs, lost access to two flagship models overnight based on a single national security letter, with no public evidence released to justify the action. The security experts warn that this approach is counterproductive. They argue that export controls are stripping defenders of the best tools while Chinese open-weight models, which are freely available to anyone, are only months behind the top U.S. models.
How Could This Precedent Affect Other AI Companies?
The implications extend far beyond Anthropic. If the government standard holds that any frontier AI model can be pulled from the market based on an unspecified security concern, this creates a dangerous precedent for the entire industry. Defense Secretary Pete Hegseth used the moment to remind the public that the Pentagon "kicked Anthropic out of our building" three months ago over a separate dispute about military access.
The timing is particularly sensitive because Anthropic has filed confidentially for an initial public offering (IPO) valuing the company near $965 billion. On Hyperliquid, a platform for pre-IPO futures trading, Anthropic's pre-IPO futures fell 3 percent on the news of the model shutdown. The company says it is working to restore access and calls the situation "a misunderstanding," but as of the time these reports were published, neither model had been restored.
Steps to Understand the Real Security Challenge
- Recognize the difference between a jailbreak and a vulnerability: A true jailbreak bypasses safety protocols entirely, while a vulnerability allows extraction of information that the model should not reveal. Anthropic's critics acknowledge the finding was the latter, not the former.
- Understand that all frontier AI models face similar risks: GPT-5.5, Claude Opus, Claude Sonnet, Claude Haiku, and the Chinese Kimi 2.7 can all find the same cybersecurity information that Fable 5 can find, according to security experts who reviewed the evidence.
- Consider the geopolitical consequences of export controls: If U.S. models are restricted while Chinese open-weight models remain freely available, the net effect may be to weaken U.S. cybersecurity defenses rather than strengthen them.
The fundamental tension is this: the government wants AI models that are completely secure before they ship, but the security research community says that standard is impossible to meet and that pursuing it may actually harm national security. Anthropic CEO Dario Amodei acknowledged in 2023 that "a jailbreak could be life or death" if someone managed to bypass safety protocols in science, technology, and biology. But the question now is whether demanding perfection before release is a realistic path forward, or whether it will simply push development and deployment of frontier AI models to countries with fewer restrictions.
Dario Amodei