The Three-Way Fight Over AI Risk Is Tearing Apart How We Regulate It
The artificial intelligence safety community is fracturing into three camps with fundamentally incompatible views on what threatens humanity most, and their inability to agree is reshaping how governments approach AI regulation. One faction warns of existential risk from advanced AI systems; another celebrates AI's potential and wants minimal regulation; a third focuses on immediate harms like misinformation and exploitation. Each group sees the others as either alarmist or dangerously naive, creating what experts call an "attentional turf war" that leaves policymakers paralyzed.
The stakes of this disagreement are enormous. When people in the AI safety community talk about existential risk, they mean humanity could cease to exist, with some expecting this outcome before 2030. These are not fringe voices; they include influential figures shaping Silicon Valley's approach to AI development. Yet the AI ethics community argues that focusing on speculative future catastrophes distracts from concrete harms happening right now: election-flipping misinformation, child exploitation, and non-consensual intimate imagery created with AI tools.
Why Can't the AI Safety Camps Agree on What Matters Most?
The three-sided debate breaks down roughly as follows. The classic AI safety community believes artificial general intelligence (AGI), a system that can match or exceed human intelligence across all domains, is possible and potentially imminent. They view it as catastrophically dangerous and argue for strict regulation and safety research. Accelerationists, by contrast, accept that AGI is real but see it as beneficial; they want minimal regulation to speed development. The AI ethics community remains skeptical that AGI is even achievable in the near term and argues that resources should focus on preventing today's harms instead.
The problem is not that any single camp is obviously wrong. Research suggests that when people encounter arguments about both existential risk and near-term harms, they become more concerned about both, not less. Yet the political reality is different: Congress has finite attention and political capital. If lawmakers focus on existential risk, they may neglect bias, discrimination, and exploitation. If they focus on near-term harms, they may underfund the infrastructure needed to prevent catastrophic AI-enabled attacks on critical systems.
This zero-sum mentality has historical roots. The earliest AI safety researchers, including influential figures like Eliezer Yudkowsky, were often hostile to the AI ethics community, which many viewed as ideologically left-leaning. That tension has softened somewhat, but it still shapes how different groups talk past each other in policy debates.
What Would "No-Regret" AI Preparedness Actually Look Like?
One emerging approach tries to break the deadlock by identifying investments that reduce risk from both advanced AI catastrophes and conventional threats. The logic is straightforward: if a government investment protects against both an AI-enabled bioweapon attack and a hostile state's biological weapons program, then funding it makes sense regardless of which threat materializes first.
These "no-regret" investments would include infrastructure and capabilities that genuinely serve dual purposes. Consider the following categories:
- Biosurveillance and Medical Countermeasures: Systems that detect engineered pathogens protect against both AI-designed bioweapons and naturally occurring pandemics or state-sponsored biological attacks.
- Hardened and Distributed Communications: Communications networks built to survive cyberattacks from advanced AI systems also withstand conventional electronic warfare and state-level cyber operations.
- AI-Generated Influence Detection: Analysts trained to identify AI-generated disinformation campaigns are equally effective at spotting ordinary state propaganda and election interference.
- Critical Infrastructure Resilience: Redundant, decentralized infrastructure protects against both AGI-level cyberattacks and conventional infrastructure sabotage.
- Red-Team and Incident-Response Capacity: Teams trained to anticipate and respond to novel AI-enabled attacks also improve response to conventional security threats.
- Secure Compute Infrastructure: Computing systems designed to resist compromise improve security against both AI-enabled and conventional attacks.
The challenge is that this argument is often asserted rather than rigorously tested. Some claimed co-benefits are genuine; others are motivated reasoning that lets advocates relabel spending they already wanted as AI-safety-relevant. A rigorous framework needs to separate the two.
Researchers are now working to develop an honest scorecard that evaluates which investments genuinely mitigate both advanced-AI and conventional catastrophic risk, and how strong the co-benefit actually is in each case. The goal is to create a decision framework that governments and funders could apply to any proposed investment to test whether it truly qualifies as "no-regret" preparedness.
How Can Governments Avoid Conflicts of Interest in AI Safety Oversight?
A separate but related problem is emerging in how AI systems are audited and certified. Currently, AI developers often hire their own safety auditors, creating an obvious conflict of interest. A developer paying an auditor has financial incentive to receive a favorable safety assessment, much like a pharmaceutical company paying for its own drug trials.
One proposed solution is to require mandatory liability insurance for frontier AI systems. Under this model, insurance companies would have their own capital at stake in risk assessments, creating a powerful incentive for rigorous, independent evaluation. Insurers would not approve coverage for systems they believed posed unacceptable risks, because they would bear the financial consequences if something went wrong.
This approach mirrors how other high-risk industries work. Underwriters Laboratories certifies electrical equipment not because manufacturers pay them to, but because insurers require independent certification before they will insure a product. The insurer's capital is on the line, so the certification is credible. Applying similar logic to frontier AI would shift the incentive structure: instead of developers hiring their own referees, independent insurers would effectively referee the game.
The challenge is that traditional insurance cannot handle the extreme tail risks of frontier AI. Catastrophic AI events could cause losses far exceeding what any insurance company could absorb. This is why some researchers propose catastrophe bonds, financial instruments that transfer extreme tail risk to capital markets rather than traditional insurers. These instruments would compel AI labs to adopt tougher safety standards because the cost of coverage would rise sharply if risk assessments showed inadequate safeguards.
Steps Toward Breaking the Policy Deadlock
Moving forward, several practical approaches could help governments navigate the competing claims about AI risk:
- Identify Convergent Investments: Prioritize infrastructure and capabilities that reduce risk from both advanced AI and conventional threats, allowing policymakers to fund preparedness without choosing between competing risk narratives.
- Require Independent Verification: Mandate that AI safety audits be conducted by parties with financial skin in the game, such as liability insurers, rather than by auditors hired directly by AI developers.
- Build State Capacity for AI Governance: Invest in training government officials and analysts to understand AI capabilities and risks deeply enough to evaluate competing claims credibly, rather than deferring to industry or advocacy groups.
- Create Honest Scoring Frameworks: Develop transparent rubrics for evaluating whether proposed investments genuinely address both AI and conventional risks, including cases where the claimed co-benefits do not hold up under scrutiny.
The underlying problem is that the three camps in the AI safety debate are not simply disagreeing about facts; they are operating from different assumptions about what constitutes a credible threat and what level of precaution is justified. Until policymakers develop the capacity to evaluate these competing claims independently, the debate will continue to paralyze regulation. The good news is that investments in preparedness, independent verification, and government expertise can move forward regardless of which camp's risk assessment ultimately proves correct.