Anthropic has filed for a roughly $965 billion initial public offering, but the timing raises a sharp question: can the company credibly warn about AI risks while racing to scale the very technology it's cautioning about? Four days after submitting its confidential S-1 registration statement to the SEC on June 1, the company published a research post titled "When AI Builds Itself" arguing that AI systems are approaching a threshold where they could autonomously design and improve their own successors without human direction.

The contradiction is hard to miss. Anthropic is simultaneously telling investors it's worth nearly a trillion dollars and telling the world that the AI systems driving that valuation may soon escape meaningful human control. The company's own data makes the tension even sharper: internal metrics show that 80% or more of the code merged into Anthropic's codebase in May 2026 was written by Claude, the company's flagship AI model.

What Does "Recursive Self-Improvement" Actually Mean?

Anthropic researchers Marina Favaro and Jack Clark define recursive self-improvement as the point at which an AI system can fully and autonomously design and develop its own successor without human involvement in setting the direction. They emphasize this hasn't happened yet and "is not inevitable," but argue it "could come sooner than most institutions are prepared for".

The mechanism is straightforward in theory: an AI system tasked with making AI research more productive keeps improving the tools and methods used to train future models. Eventually, humans become the bottleneck. The system generates better ideas faster than researchers can evaluate them, and supervision becomes nominal.

The internal data Anthropic disclosed paints a picture of rapid acceleration. Claude Opus 3, released in March 2024, could handle tasks taking roughly four minutes. By March 2025, Sonnet 3.7 handled tasks up to 1.5 hours. Claude Opus 4.6, released in March 2026, reaches 12-hour tasks. The company's projection in the post suggests tasks taking weeks by 2027.

Performance on research benchmarks has followed similarly steep curves. On SWE-bench, a software engineering benchmark, Claude scores went from single digits to saturation in under two years. On CORE-Bench, which measures the ability to reproduce published research, performance jumped from 20% to saturation in just 15 months.

How Is Anthropic's Business Growing So Quickly?

The IPO filing reveals why investors are valuing Anthropic so aggressively. The company reported approximately $44 billion in annualized revenue as of the first quarter of 2026, up from roughly $10 billion in mid-2025, representing roughly 4x annual growth. Anthropic crossed $1 billion in monthly recurring revenue in early 2025 and has maintained that explosive trajectory.

Claude models now power production systems at major enterprises including Goldman Sachs and Accenture, and hundreds of enterprise software products rely on Claude's capabilities. Claude Opus 4.8, released in May 2026, handles critical research workflows at pharmaceutical companies and national laboratories.

This genuine business momentum is precisely what makes the timing of the "When AI Builds Itself" post so fraught. Anthropic is asking for capital to fund more research, more engineers, and larger training runs, while simultaneously warning that AI development may be moving faster than human oversight can manage.

What Is Anthropic Actually Proposing?

The company is not calling for an immediate halt to AI development. Instead, Favaro and Clark propose a coordinated international pause mechanism with three simultaneous conditions: multiple well-resourced frontier labs in multiple countries agree to stop, each can verify the others have actually stopped, and clear conditions specify when the pause ends and what triggers it.

Anthropic's own commitment is explicitly conditional: "If such systems existed, we expect that we would slow down or temporarily pause, if other developers at or near the frontier also did so in a verifiable manner". Jack Clark, Anthropic's cofounder, was more direct in a BBC Newsnight interview, stating: "Right now, it's like the AI industry has a gas pedal, but it doesn't have a brake pedal".

The post compares the challenge to nuclear arms control, specifically the INF Treaty model, then notes a critical difference: nuclear warhead counts involve physical objects in known locations. AI training runs involve code, compute, and datasets that are indistinguishable from ordinary software development. A training run can be concealed without specialized equipment or observable signatures.

Steps to Understanding Anthropic's Verification Challenge

• The Detection Problem: Unlike nuclear weapons, AI training runs leave no physical signatures and use general-purpose computing infrastructure that looks identical to normal software development, making verification nearly impossible without unprecedented transparency agreements.
• The Coordination Problem: Any pause mechanism requires simultaneous agreement from multiple competing frontier labs across different countries, each with financial incentives to continue development unilaterally.
• The Enforcement Problem: Even if labs agree to pause, there is no mechanism to verify compliance without intrusive monitoring of proprietary research, which companies are unlikely to permit.

The post doesn't offer a solution to these problems. Instead, it calls for "deliberative conversations" to investigate the verification challenge. That's a reasonable request, but functionally it amounts to asking the industry to spend time organizing conversations while labs continue training larger models.

Why Are Critics Calling This "Regulatory Capture"?

The reactions from outside Anthropic split along predictable lines. Rob Enderle of the Enderle Group called the effort "more about strategic marketing than any concrete initiative," suggesting that publicizing Anthropic's progress toward recursive self-improvement is "a more calculated move" than it appears.

Holger Mueller of Constellation Research put the conflict of interest directly: "Is it trying to freeze the status quo so it can catch up, or simply retain its lead?". The sharpest critique came from venture capitalist David Sacks, who called it a "regulatory capture agenda," arguing that companies calling loudest for heavy regulation are those whose closed, expensive models benefit most from rules that make smaller open-source alternatives harder to develop.

The timing amplifies these concerns. Anthropic is filing for an IPO that would raise billions in capital for scaling operations, while simultaneously warning that scaling may be dangerous. The company is asking for a coordinated international pause, but only if competitors agree to pause in a verifiable manner. Until then, Anthropic's incentive is to move faster than rivals.

The S-1 filing is a regulatory document, and Anthropic is required to describe material risks to its business, including risks from its own technology. "When AI Builds Itself" reads, in part, like a first draft of what that risk section might say, with the disclosure that 80% of the company's own code is now written by the system it's warning about.

Whether Anthropic's warning reflects genuine concern about AI safety or strategic positioning ahead of an IPO may ultimately depend on what the company does next. If Anthropic voluntarily slows development while waiting for competitors to coordinate, the warning carries weight. If the company accelerates scaling while calling for international pauses, the critique about regulatory capture will likely intensify.