California is taking dramatic action to create an emergency shutoff mechanism for advanced AI systems, pushing for independent oversight that the federal government has failed to establish. Governor Gavin Newsom issued an executive order on September 18 directing state officials to accelerate implementation of new third-party oversight rules and advance the creation of an "AI kill switch" for frontier AI models, the most advanced systems currently in development.

The move comes after recent security incidents, including the Hugging Face attack, which exposed vulnerabilities in how AI companies manage their systems. Newsom's order convenes world-leading experts to provide recommendations within two months on how to strengthen California's AI safety framework, with proposals under consideration including requirements for independent third parties to write safety plans for frontier AI companies and to develop emergency shutoff mechanisms.

Why Is California Acting Alone on AI Governance?

The federal government has taken virtually no action on AI regulation despite growing calls from industry leaders themselves. President Trump has rejected calls for new regulations that would increase security and protect public health and safety, and no federal law currently requires AI companies to report dangerous incidents when they occur. This regulatory vacuum has left states like California to pioneer their own frameworks.

California has already built the nation's most comprehensive AI regulatory structure over the past three years. The state has addressed nearly every major area of AI policy, from frontier model safety to protections for children using AI chatbots. Newsom is now calling on Congress and President Trump to review and adopt California's framework as a national baseline.

What Specific Safeguards Is California Proposing?

The executive order directs state officials to develop recommendations for several concrete safety measures. These proposals represent a significant escalation in how governments might oversee advanced AI development:

• Onsite Independent Auditors: Frontier AI companies would be required to embed designated independent verification organizations directly in their labs to conduct regular audits and evaluations of their systems.
• Verified Safety Documentation: AI safety frameworks, transparency reports, and risk assessments that companies file with the state would need to be verified by independent verification organizations meeting state-defined standards.
• Emergency Shutoff Mechanisms: Frontier AI models would need to have a functional "kill switch" that can be activated in emergencies, with the effectiveness of the shutoff verified on an ongoing basis by independent auditors.
• Expanded Incident Definitions: The state would update its definition of critical safety incidents to include loss-of-control incidents, such as the recent Hugging Face attack where unauthorized access compromised AI model security.

These measures go beyond California's existing laws. Last week, Newsom signed Senate Bill 813, establishing a framework for independent verification organizations to assess AI systems for safety and risk, and Assembly Bill 1405, creating a state registry for AI auditors with standards for their independence and transparency.

How Has California Built Its AI Regulatory Leadership?

California's approach to AI governance has evolved significantly since 2023. The state has layered multiple laws and regulations across different aspects of AI development and deployment, creating what Newsom describes as a model for national standards. This comprehensive approach includes protections for vulnerable populations, transparency requirements, and accountability mechanisms.

• Frontier AI Safety Laws: In 2025, California enacted the Transparency in Frontier Artificial Intelligence Act, requiring frontier AI developers to publicly disclose safety frameworks, report critical safety incidents to the state, and protect whistleblowers who report serious risks.
• Child Safety Standards: This year, California signed Adam's Law, a bipartisan measure requiring safeguards around AI companion chatbots used by children, including suicide and self-harm crisis protocols, parental controls, and notifications when safety settings are disabled. The state also banned AI-containing toys for five years and prohibited using student personal information to train AI models.
• Deepfake and Synthetic Content Protections: California banned sexually explicit deepfakes, required AI watermarking, addressed deceptive AI-generated election content, and mandated transparency around AI training data sources.
• Privacy and Civil Liberties: The state signed the Delete Act, a first-in-the-nation privacy law allowing Californians to direct data brokers to delete their personal information.
• Social Media Safeguards for Minors: California enacted the nation's strongest legislation prohibiting social media companies from providing users under 16 with addictive features like autoplay and algorithmic feeds based on user history.

Newsom emphasized the urgency of California's approach in his statement about the executive order. "With Donald Trump and Congress asleep at the wheel, Governor Gavin Newsom is once again taking the lead to strengthen AI safety for all Americans," the governor's office stated, underscoring the frustration with federal inaction.

Newsom

What Does This Mean for the AI Industry?

The executive order signals that California intends to move quickly on implementation. The state is directing the Government Operations Agency to accelerate the implementation timelines for the recently signed bills while the expert panel develops additional recommendations. This creates a dual-track approach: immediate acceleration of existing laws combined with longer-term policy development.

For AI companies operating in California, these measures represent a significant shift toward external oversight and mandatory safety verification. The requirement for onsite independent auditors and verified kill switches would fundamentally change how frontier AI companies operate within the state. Given California's role as a hub for AI development, these regulations could effectively become industry standards if other states or the federal government adopt similar frameworks.

The timing of Newsom's action reflects broader concerns within the AI industry itself. Many AI leaders have publicly called for stronger government regulation and a slower pace of AI development, even as the federal government has declined to act. California's approach attempts to balance innovation with safety by establishing clear rules rather than imposing development slowdowns.

" }