The Great AI Safety Paradox: Why the Industry Flunks Even Its Own Tests
The world's leading artificial intelligence companies are failing to adequately prepare for the most serious risks posed by advanced AI systems, according to a comprehensive safety assessment released this week. Even Anthropic, which topped the rankings, received only a C+ grade, and no company achieved an A in any single safety category.
The Future of Life Institute, a US-based AI safety think tank, evaluated nine of the world's leading AI companies across six critical areas: risk assessment, current harms, safety frameworks, existential safety, governance and accountability, and information sharing. Seven researchers and governance experts compiled the rankings using publicly available information and data provided by the companies themselves.
Which AI Companies Ranked Highest and Lowest?
Anthropic secured the top position with its C+ overall score, followed by OpenAI in second place and Google DeepMind in third. Meta climbed two spots to fourth, while xAI dropped three positions to seventh. China's DeepSeek placed fifth, Alibaba Cloud sixth, and France's Mistral finished last in the rankings.
The poor showing from Mistral sparked particular concern among safety advocates. Max Tegmark, an MIT professor and president of the Future of Life Institute, expressed disappointment with the French company's last-place finish, noting that Europe has positioned itself as a leader in AI safety. Mistral disputed the assessment, arguing that the report's framework was not suited to its approach of developing open-source AI models that users can download and modify, unlike the closed systems favored by competitors like Anthropic and OpenAI.
Several companies, including Alibaba, xAI, and DeepSeek, declined to participate in the survey, according to the institute.
What Specific Safety Gaps Did the Report Identify?
The report highlighted several alarming trends across the industry. Most notably, several companies that had previously pledged not to allow their technology for military use have since softened those positions. The report specifically criticized Anthropic over what it described as "questionable military engagements," noting that the US government used Anthropic's technology during military operations involving Venezuela and Iran over the past year.
The researchers concluded that all nine companies remain inadequately prepared to address "existential" threats associated with artificial general intelligence, or AGI, which refers to AI systems capable of matching human-level intelligence across a wide range of tasks. While the report acknowledged that "constructive attempts exist," it emphasized that industry-wide efforts remain "entirely inadequate".
How Are Companies Addressing Cybersecurity and Misuse Risks?
Beyond existential concerns, the report flagged serious near-term risks. These include the potential for advanced AI to be misused for cyberattacks and to perform tasks harmful to people. Anthropic's experience with its latest model, Mythos, illustrates the tension between capability and safety. The company initially released Mythos only to a small group of trusted organizations in April because of concerns that its cyber capabilities could be exploited by malicious actors.
The US government later blocked Anthropic from releasing Mythos to foreign users on national security grounds on June 12, though the Trump administration lifted that restriction on June 30.
Steps Companies Should Take to Improve Safety Ratings
- Establish transparent risk assessment frameworks: Companies need to develop and publicly disclose methodologies for identifying and quantifying AI risks, moving beyond vague probability estimates to concrete, falsifiable criteria that can be independently verified.
- Implement comprehensive governance structures: Organizations should create clear accountability mechanisms, including independent oversight boards, regular safety audits, and documented decision-making processes for deploying powerful AI systems.
- Increase information sharing and collaboration: The industry should move toward greater transparency about safety testing results, failure modes, and lessons learned, rather than treating safety data as proprietary competitive advantage.
- Address current harms from deployed systems: Companies must prioritize auditing and mitigating documented harms from narrow AI systems already in use, such as algorithmic bias in hiring and lending decisions, rather than focusing exclusively on speculative future risks.
The report's findings arrive amid broader debate about whether the industry's focus on existential AI risks has distracted from more immediate harms. A separate white paper from NextGen Economics argues that the existential narrative surrounding AGI serves the strategic interests of tech corporations seeking regulatory capture and investors seeking valuation premiums. The paper contends that current AI architectures show no empirical evidence of consciousness or self-awareness, and that the real dangers are already present in algorithmic discrimination, ecological devastation, and erosion of institutional trust.
"I was very disappointed to find that they came last, especially since Europe has really been a leader in AI safety," said Max Tegmark, an MIT professor and president of the Future of Life Institute.
Max Tegmark, MIT Professor and President of the Future of Life Institute
The contrast between the two perspectives highlights a fundamental tension in AI safety discourse. While some researchers emphasize the importance of preparing for potential future superintelligent systems, others argue that the industry should redirect resources toward addressing documented harms from systems already deployed in healthcare, criminal justice, employment, and environmental monitoring.
The Future of Life Institute's report suggests that the industry cannot afford to choose between these concerns. The fact that even the highest-scoring company received only a C+ indicates that current safety practices fall short of what experts consider adequate, regardless of whether the primary focus is on near-term harms or long-term existential risks.
As AI systems become increasingly integrated into critical infrastructure and decision-making processes, the pressure on companies to demonstrate meaningful safety improvements will likely intensify. The report's findings suggest that voluntary industry efforts may be insufficient, potentially creating momentum for more stringent regulatory requirements in the coming months.