Google DeepMind chief Demis Hassabis is sounding an urgent alarm: the world is so focused on building faster, more powerful AI systems that it's neglecting the safety guardrails needed before artificial general intelligence (AGI) arrives. In a new personal blog post, the Nobel Prize-winning researcher argues that AGI, defined as a system exhibiting all the cognitive capabilities of the human brain, could emerge within three to four years. But instead of celebrating this milestone, Hassabis warns that governments and tech companies are paying too much attention to the competitive race and too little to the risks that come with increasingly capable systems.

Why Should We Care About AGI Safety Right Now?

The stakes are extraordinarily high. Hassabis points out that frontier AI systems, the most advanced models available today, are already creating cybersecurity concerns. As these systems become more powerful and autonomous, the risks could expand dramatically. He specifically warns about potential threats linked to biological and nuclear security, dangers that may emerge as AI capabilities continue to advance. The concern isn't theoretical; it's grounded in the reality that researchers still don't fully understand how today's most advanced AI models arrive at many of their decisions, according to Anthropic, the developer of Claude.

Hassabis argues that researchers will eventually need robust mechanisms to retain control over increasingly autonomous AI systems, particularly if future models become capable of improving themselves with limited human involvement. This is where the competitive pressure becomes dangerous. When companies are racing to deploy the next breakthrough, safety testing and oversight can get sidelined.

What Regulatory Framework Is Hassabis Proposing?

Rather than simply warning about risks, Hassabis has put forward a concrete proposal: a dedicated AI oversight body modeled on the Financial Industry Regulatory Authority (FINRA), the private organization that regulates US brokerage firms under federal supervision. Here's how the system would work:

• Voluntary Submission: Developers would initially submit their most advanced models for evaluation up to 30 days before release, allowing time for safety testing before the public gains access.
• Industry Funding: The regulatory body would receive funding primarily from AI companies themselves, creating a financial incentive for the industry to support robust oversight.
• Mandatory Participation: Once testing procedures mature and prove reliable, participation would eventually become mandatory rather than voluntary, ensuring all frontier models meet safety standards before deployment.
• International Standards: The longer-term goal is to establish internationally accepted standards for frontier AI development, with the United States positioned to lead because of its technological leadership.

"Urgent action is needed to address risks that might arise as we get closer to AGI. We've already seen the challenges frontier models pose for cybersecurity, and other threats including nuclear and bio risks may soon emerge as capabilities continue to advance," Hassabis wrote.

Demis Hassabis, Chief Executive at Google DeepMind

This proposal reflects a broader concern shared by other leading AI companies. Anthropic CEO Dario Amodei has similarly advocated for stronger safeguards and regulatory oversight, acknowledging that the industry still doesn't fully understand how its most powerful systems work.

How Can Governments and Companies Act Before AGI Arrives?

Hassabis emphasizes that the window for action is closing. He argues that decisions made over the coming years will play a critical role in determining how AI shapes the next phase of human civilization. The key steps involve:

• Establish Clear Standards: Create safety benchmarks that frontier AI models must satisfy before deployment, similar to how pharmaceutical companies must prove drug safety before FDA approval.
• Build Technical Expertise: Bring together leading technical experts to evaluate advanced models and understand their capabilities and risks before they reach the public.
• Move Quickly on Regulation: Governments need to act before increasingly capable systems outpace existing safeguards, creating a regulatory framework that can evolve as AI technology advances.
• Foster International Cooperation: Work toward globally accepted standards rather than allowing different countries to develop conflicting regulatory approaches that could create a race to the bottom.

Hassabis acknowledges the uncertainty surrounding AI development. "Nobody in the world knows for sure what is going to happen from here, and even the experts disagree," he wrote. However, he argues that when stakes are this high and uncertainty is this great, proceeding with cautious optimism is the sensible approach.

Hassabis

What Could AGI Actually Achieve if Developed Responsibly?

While much of Hassabis's message centers on reducing risk, he also presents an optimistic vision of what AGI could eventually accomplish. He argues that the technology has the potential to accelerate scientific discovery, transform healthcare through faster drug development, enable breakthroughs in clean energy and advanced materials, and help address some of humanity's most pressing challenges. In his view, AGI's long-term impact could surpass that of previous technological revolutions like electricity or fire.

Reflecting on decades spent pursuing the technology, Hassabis writes that AGI should not be viewed as simply another computing milestone. "It is much more akin to the discovery of electricity or fire. If you stop to think about it, we've essentially found a way to make sand think. It's miraculous," he stated. Yet this miraculous potential makes the safety question even more urgent. The decisions made today about how to develop and deploy AGI responsibly will determine whether this technology becomes humanity's greatest achievement or its greatest risk.

Hassabis

The core tension Hassabis identifies is real: the competitive pressure to build AGI first is pushing companies to move faster, but the complexity and potential dangers of AGI demand that we move more carefully. His proposal for a FINRA-style regulatory body represents an attempt to square that circle, creating a framework that allows innovation while ensuring safety standards are met before systems reach the public.