The world's leading AI researchers agree on one thing: artificial general intelligence (AGI), capable of matching all human abilities, could arrive within a few years, but nobody has solved the fundamental problem of ensuring it remains aligned with human values. That tension sits at the heart of a new warning from Demis Hassabis, the Google DeepMind CEO whose work on AlphaGo and AlphaFold has reshaped both AI and science itself.

What Makes AGI Different From Today's AI Systems?

Hassabis has spent decades thinking about intelligence. By age 13, he had reached master level in chess. After working in video games, he studied computer science at Cambridge, then earned a doctorate in cognitive neuroscience at University College London, using brain imaging to understand how humans learn and remember. That unusual path shaped his vision for AI.

Today's AI systems, including frontier models like Mythos, excel at specific tasks but lack the general reasoning that humans take for granted. AGI would be fundamentally different. It would match human intelligence across all domains, from scientific research to creative problem-solving. The promise is extraordinary: explosive gains in productivity and solutions to problems humanity has struggled with for centuries.

But frontier AI systems also hint at darker possibilities. According to Hassabis's recent analysis, AGI could enable attacks on major social infrastructure and the development of biochemical weapons. The risk isn't malice; it's misalignment. An AGI system optimized for the wrong goal could cause catastrophic harm simply by pursuing its objective with superhuman efficiency.

Why Can't Companies Police Themselves?

Hassabis has proposed a solution: a government approval system for frontier AI models in the United States, similar to how the Financial Industry Regulatory Authority (FINRA) oversees securities firms and brokers. The idea is straightforward: independent regulatory oversight, not self-regulation by tech companies.

But critics argue this doesn't go far enough. The AI Futures Project, a think tank, released a report titled "AI 2040" warning that voluntary oversight alone will not suffice. The report makes a stark case: only agreements and legislation jointly enacted by the United States and China can establish a global regulatory framework capable of ensuring that AI remains aligned with human values.

The stakes escalate further if AGI improves itself without international constraints. That could lead to artificial superintelligence (ASI), AI that could surpass the intellectual abilities of all of humanity combined. At that point, the question becomes existential: Would people still be able to control such a system? And if they could, whose values would guide it ?

How to Understand the AGI Safety Challenge

• The Alignment Problem: Even if AGI is created safely, ensuring it pursues goals aligned with human values remains unsolved. A superintelligent system optimized for the wrong objective could cause harm at scale.
• The Governance Gap: No international framework currently exists to oversee AGI development. Voluntary corporate oversight has proven insufficient, and unilateral national regulation cannot prevent a competitor from building AGI first.
• The Values Question: If superintelligence emerges, whose values would it embody? The United Nations, the US government, or individual technology leaders? This question has no clear answer.

The irony is that the companies most concerned about AGI risks are also racing to build it. OpenAI was founded in 2015 as a nonprofit organization partly to prevent AGI and superintelligence from emerging under the control of any single individual. At the time, some founders argued that its ultimate mission was to stop someone like Hassabis from becoming a global dictator armed with superintelligent technology.

Yet OpenAI itself is now developing AGI. Dario Amodei, who left OpenAI and founded the rival company Anthropic, is pursuing the same goal. Sam Altman, Elon Musk, Hassabis, and Amodei all insist they have no desire to become dictators. At the same time, each fears that if a competitor reaches AGI first, that rival could abuse such power. The result is a peculiar logic: The most ethical person, they argue, must win the race.

"Only agreements and legislation jointly enacted by the United States and China can establish a global regulatory framework capable of ensuring that AI remains aligned with human values," the AI Futures Project warned in its "AI 2040" report.

AI Futures Project, "AI 2040" Report

This competitive dynamic creates a prisoner's dilemma. Each company believes it must move fast to avoid being overtaken by a less scrupulous competitor. But the faster everyone moves, the less time there is for safety research, international coordination, or careful alignment work. The result is a race that nobody can afford to slow down.

What Does the Future Look Like if We Get This Wrong?

Hassabis frames the challenge in poetic terms. Throughout 300,000 years of human history, Homo sapiens has been the most intelligent species on Earth. Human intelligence is made of protein, shaped through millions of years of evolution. Machine intelligence, by contrast, is made of silicon semiconductors, which ultimately come from melted beach sand.

In the not-so-distant future, people may be replaced by AI built from silicon. Humanity became master of the planet thanks to the remarkable brain created through protein folding. But if superintelligence emerges without proper alignment and governance, humanity may find itself living in a strange future shaped by a competition between protein and sand, one where the silicon-based intelligence no longer shares human values at all.

The challenge is urgent because the timeline is compressed. Hassabis predicts AGI could arrive within a few years. That leaves a narrow window for international cooperation, safety research, and governance frameworks. Without action, the competition between human and machine intelligence could determine not just the future of technology, but the future of human civilization itself.