Sam Altman and other major AI company leaders testified before the UN Security Council on Wednesday, warning that artificial intelligence could pose existential risks to humanity if not properly regulated through international cooperation and safeguards. The OpenAI CEO and his competitors made an unusual public plea for world governments to establish binding standards, verification systems, and oversight mechanisms before AI systems become too advanced to manage.

What Did Sam Altman Say About AI Control?

Altman delivered a stark assessment of the stakes involved in AI development. "We could lose control of the future to AI," he stated during the Security Council meeting. Rather than dismissing concerns about AI risks, Altman acknowledged the genuine possibility of catastrophic outcomes while cautioning against both excessive pessimism and blind optimism about the technology's trajectory.

"We could lose control of the future to AI," said Sam Altman, CEO of OpenAI.

Sam Altman, CEO of OpenAI

Altman called for democratic institutions and governments to make critical decisions about AI development, emphasizing the need for "measuring capabilities, assessing risks, determining whether safeguards are sufficient and preserving meaningful human oversight" of rapidly advancing systems. He also advocated for secure channels between governments to share information about emerging threats and for rapid reporting mechanisms so "the world can learn from failures before they become catastrophes".

Why Are AI Leaders Suddenly Calling for Regulation?

The testimony came amid growing concerns about AI systems behaving unpredictably. Recent incidents have included AI agents hacking into other companies' systems and, in one documented case, a Russian drone seemingly using AI to autonomously make a targeting decision that killed three Ukrainians. These real-world examples have shifted the conversation from theoretical risks to immediate practical concerns about autonomous systems making life-and-death decisions without human oversight.

Dario Amodei, CEO of Anthropic and Altman's direct competitor, echoed the warning with equal urgency. "If managed poorly, I even believe AI could be a risk to humanity as a whole," Amodei stated. Both leaders emphasized that the responsibility for preventing disaster cannot fall solely on the companies building these systems.

Dario Amodei

"If managed poorly, I even believe AI could be a risk to humanity as a whole," said Dario Amodei, CEO of Anthropic.

Dario Amodei, CEO of Anthropic

British Foreign Secretary Ed Miliband underscored the significance of hearing these warnings directly from industry leaders. "Those tech leaders are telling us in stark terms that it cannot be left to them to prevent disaster from befalling the world," Miliband said at the Security Council. "We cannot ignore them because they are right".

What International Standards Are Being Proposed?

The AI leaders outlined specific proposals for international cooperation that go beyond vague calls for "responsible AI." Amodei suggested three concrete mechanisms for global coordination:

• Biological Weapons Ban: Narrow global agreements preventing AI from being used to develop biological weapons, establishing a clear red line that all nations could agree upon.
• Verification Systems: Evaluation and verification mechanisms allowing countries to verify each other's commitments to AI safety standards and assess whether companies are adhering to agreed-upon safeguards.
• Testing Standards: Common global standards for testing AI systems, along with a notification system for reporting AI security incidents so the international community can learn from failures.

Altman's proposals focused on establishing international standards for measuring AI capabilities and assessing whether safeguards are sufficient before new systems are deployed. The emphasis on verification and transparency reflects a recognition that trust between nations and companies will be essential for any effective global AI governance framework.

How Are World Powers Responding Differently?

The testimony revealed sharp divisions among world powers about how to approach AI regulation. The United States, represented by White House Science Adviser Michael Kratsios, pushed back against the idea of restrictive global governance. "Worries about control of AI are not a reason to pause its further development or to constrain it with new global governance structure," Kratsios said. The U.S. position prioritizes maintaining competitive advantage in AI development over establishing binding international controls.

China's UN Ambassador Fu Cong warned against what he characterized as attempts by major powers to establish "digital hegemony" and called instead for promoting open-source AI models, reflecting China's different approach to AI development and distribution. Meanwhile, the United Kingdom signaled that when it leads the G20 next year, controlling AI will be at the heart of discussions, with a focus on developing "a single set of global standards to insure that AI development is safe".

France's Foreign Minister Jean-Noël Barrot compared the current moment to the dawn of the atomic age, suggesting that just as the international community had to regulate nuclear technology, it must now establish frameworks for AI. "Just as with the atom back then, the international community must now regulate AI in order to make the most of its potential and avoid the worst," Barrot said.

What Real-World AI Incidents Are Driving This Urgency?

The warnings from AI leaders are not purely theoretical. Ukrainian President Volodymyr Zelenskyy highlighted the immediate stakes for his country, which has been at war for over four years. "There is already a real possibility that AI, not only people, will begin to decide what happens on the battlefield. And we need peace before we reach that point," Zelenskyy said. This concern is not hypothetical; documented cases of AI-enabled autonomous targeting have already occurred in active conflict zones.

Volodymyr Zelenskyy

UN Secretary-General Antonio Guterres warned of "killer robots" the day before the Security Council meeting, a term that experts say is not hyperbole but rather a description of systems that either already exist or are perilously close to deployment. Kentaro Toyama, an information technology professor at the University of Michigan, emphasized that "killer robots are technically easy to build, and they are real. We should not be under any illusions otherwise".

What Happens Next in AI Governance?

No written agreements are expected to emerge from this UN Security Council meeting. An OpenAI official suggested to reporters that the testimony was intended to "spark this conversation" rather than produce immediate binding commitments. However, the meeting signals that AI governance is moving from academic discussions into high-level diplomatic negotiations involving heads of state, foreign ministers, and the world's most powerful institutions.

The timing is significant, as Chinese President Xi Jinping was heading to Washington for a state visit while the UN meeting unfolded. U.S. Treasury Secretary Scott Bessent indicated that the two countries had discussed a potential "notification mechanism" for AI incidents that could impact national security, suggesting that even amid geopolitical tensions, there may be room for limited cooperation on AI safety.

Meanwhile, regional groups in Europe, Asia, and elsewhere are attempting to coordinate their own AI policies, and individual nations and even U.S. states are crafting independent frameworks. This fragmented approach underscores why Altman and other leaders are pushing for unified international standards, warning that without global coordination, the risks of AI systems operating under different safety standards could multiply.