The most capable AI models are becoming targets for misuse, forcing companies like Anthropic and OpenAI to balance innovation with safety as they race to dominate enterprise markets. Anthropic revealed it disrupted multiple attempts by scientists to use its AI models for biological weapons research this year, while OpenAI simultaneously unveiled a new version of ChatGPT powered by its latest GPT-6 Astra model, designed specifically for Wall Street.

What Threats Are AI Companies Actually Seeing?

Anthropic's security team discovered that researchers attempted to use its Claude AI model to research the development of biological weapons, though the company couldn't determine whether these were legitimate scientific inquiries or malicious efforts. The San Francisco company blocked the activity anyway as a precaution. This isn't an isolated incident. Anthropic documented a pattern of misuse attempts across multiple threat categories, including state-sponsored actors attempting to weaponize its models.

The threats Anthropic identified include:

• Bioweapon Development: Scientists used AI models to research processes for creating more harmful pathogens, with some deliberately circumventing safety blocks.
• Propaganda and Disinformation: Russian state media attempted to use frontier AI to generate fake journalism that appears authentic and credible.
• Conventional Weapons Design: Chinese actors tried to leverage AI models to design and develop traditional military weaponry.

Anthropic noted that some researchers actively tried to hide their intentions while working around safety guardrails, suggesting deliberate evasion rather than accidental misuse. The company emphasized a critical insight: "As models become increasingly capable, their risks will increase, unless AI developers and society's defenders act to make them safer".

Anthropic

Why Is OpenAI Betting Big on Finance Right Now?

While Anthropic grapples with security threats, OpenAI is aggressively pursuing enterprise revenue by tailoring its AI to specific industries. The company unveiled ChatGPT for financial services, powered by its new GPT-6 Astra model, positioning it as a comprehensive solution for major banks and investment firms.

OpenAI's strategy reflects a broader industry shift toward vertical-specific AI products. Rather than offering one-size-fits-all models, leading AI companies are packaging their technology for particular sectors where they can command premium pricing and deeper customer relationships. Finance is one of three industry verticals OpenAI has prioritized, alongside cybersecurity and software engineering.

"This is the canonical product we are hoping the industry adopts," said Nick Turley, OpenAI's vice president and head of ChatGPT, referring to the financial services version.

Nick Turley, Vice President and Head of ChatGPT at OpenAI

Turley has spent considerable time in New York City working directly with major financial institutions to refine the product. His team collaborated with Morgan Stanley and boutique investment advisory firm Evercore to identify the features that would actually work in real-world banking environments, not just in polished product demonstrations. This hands-on approach reflects OpenAI's recognition that enterprise adoption requires deep domain expertise, not just raw AI capability.

How Are AI Companies Balancing Growth With Safety?

The tension between capability and safety is becoming impossible to ignore. Anthropic's security disclosures reveal that as AI models become more powerful, they attract more sophisticated misuse attempts. The company's inability to always determine intent means it must err on the side of caution, blocking legitimate research alongside potential threats.

Meanwhile, OpenAI's aggressive enterprise push suggests the company believes it can manage risks while scaling deployment. The financial services product represents a calculated bet that industry-specific safeguards, combined with institutional oversight from major banks, can mitigate misuse risks better than general-purpose deployment.

This creates a paradox: the more capable AI becomes, the more valuable it is for legitimate enterprise use, but also the more dangerous if misused. Anthropic's recent threat report suggests that state-sponsored actors and sophisticated researchers are already adapting their tactics to circumvent AI safety measures. The company's decision to block research it couldn't fully evaluate indicates that the industry may need to accept false positives as the cost of preventing catastrophic misuse.

OpenAI's financial services strategy, by contrast, assumes that deploying AI within regulated industries with institutional oversight provides sufficient guardrails. Banks have compliance teams, audit trails, and regulatory scrutiny that consumer-facing applications lack. Whether this institutional framework is sufficient to prevent misuse remains an open question as these models become more capable and more widely deployed.