Most organizations are deploying artificial intelligence far faster than they're securing it, creating a dangerous gap between ambition and readiness. As AI systems now run real operations at companies across industries, the race for strategic advantage is outpacing the work needed to protect these systems from both traditional cyber threats and an entirely new class of AI-enabled attacks.

Why Is AI Security Falling Behind Deployment?

The problem is straightforward but urgent: enterprises are investing heavily in AI capabilities while underinvesting in the threat maturity, data security practices, and governance structures needed to protect them. Security and AI ambitions are on a collision course. Organizations pursuing faster insights, lower operating costs, and new revenue streams through AI are building on foundations that lack the resilience to manage the risks these systems introduce.

The threat landscape is growing more complex by the day. Deepfakes are becoming indistinguishable from reality, AI-powered fraud can mimic trusted voices and identities with alarming precision, and AI-generated content is flooding every channel, making it harder to tell what is real and what is not. Attackers are using the same AI capabilities that organizations rely on for growth, turning innovation into vulnerability.

What Are the Two Dimensions of AI Security That Organizations Must Address?

Security experts identify two distinct but interconnected challenges that organizations must tackle simultaneously. The first involves protecting AI systems themselves; the second involves using AI to defend against attacks. Both require fundamentally different approaches than traditional cybersecurity.

When it comes to securing AI, the challenge extends far beyond the model itself. AI systems introduce entirely new attack surfaces that traditional cybersecurity controls were never designed to address. Threats like model poisoning and prompt injection require fundamentally new approaches to detection and monitoring. But even when a model is locked down, the ecosystem around it may not be. A recent example illustrates this risk: Anthropic's Claude Mythos Preview, an advanced AI model capable of surpassing skilled humans at finding software vulnerabilities, was restricted to roughly 40 elite organizations through Project Glasswing. Yet within 24 hours of its announcement, unauthorized users gained access, not by breaching Anthropic's core infrastructure, but through a third-party vendor environment.

AI agents add risk in a different way than traditional applications. They can be granted credentials, access data, call tools, and take actions across systems, often with far less visibility than a human user and with permissions that persist long after the original project ends. A recent Cloud Security Alliance survey found that 82% of enterprises report they have unknown AI agents operating in their environments, and nearly two in three (65%) report AI agent-related incidents in the past year, with impacts ranging from data exposure (61%) and operational disruption (43%) to financial losses (35%).

How Can Organizations Build True AI Resilience?

Building resilience requires moving beyond governance frameworks and preventive controls alone. Organizations must take a multi-layered approach that addresses threats at every stage of AI deployment and operation.

• Anticipate Threats and Failure Modes: AI models and the agents that use them add a layer of complexity that exceeds much human imagination. Focusing on what can go wrong and how it will impact operations or critical data loss will change everything. Organizations need to be prepared for the unexpected.
• Establish AI-Specific Governance: Traditional IT governance wasn't designed for the speed, opacity, or risk profile of AI systems. Organizations need dedicated AI governance frameworks that define accountability for model risk, enforce ethical use policies, mandate transparency in AI decision-making, and align AI initiatives with enterprise risk appetite.
• Invest in AI Runtime Defenses: Don't stop at access controls and policies. Deploy monitoring, anomaly detection, and guardrails that operate during model inference, catching prompt injections, data exfiltration, and adversarial manipulation in real time. Static pre-deployment testing alone cannot account for how models behave under live, adversarial conditions.
• Integrate Supply Chain Risk Management: The most capable AI model in the world is only as secure as its weakest vendor link. Every model provider, training data source, third-party evaluator, and API dependency is an attack surface that must be monitored and managed.

A critical visibility gap undermines many organizations' confidence in their AI security posture. In the same Cloud Security Alliance survey, 68% of organizations said they have strong visibility into AI agents, yet 82% reported discovering previously unknown agents in the past year. This confidence gap reveals that governance frameworks must close the gap between perceived and actual visibility.

What Does the Deepfake Voice Fraud Threat Look Like in Practice?

While securing AI systems is one challenge, defending against AI-powered attacks is another. Deepfake voice fraud has emerged as a particularly effective and growing threat. The Federal Bureau of Investigation (FBI) has warned that criminals are increasingly using artificial intelligence-generated audio deepfakes in "vishing" (voice phishing) campaigns to pressure employees into transferring funds, resetting credentials, or disclosing sensitive information.

According to Deloitte's Center for Financial Services, deepfake fraud losses reached $12.3 billion in 2023 and are projected to climb to $40 billion in the United States by 2027. The threat is effective because it exploits human trust, urgency, and authority. A recent survey by cybersecurity company McAfee found that one in 10 Americans have now experienced a voice clone scam, and approximately 53% say they share their voice online at least once a week.

Most deepfake voice incidents follow a predictable pattern. Attackers gather public recordings such as earnings calls, podcasts, and social media videos, then research organizational charts, reporting lines, vendor relationships, and payment processes. Using widely available AI tools, they generate a synthetic voice that mimics a target's accent, cadence, speech patterns, and emotional tone. Creating a convincing voice clone now requires only three to five seconds of audio.

"If I wanted to make a deepfake of you, I would simply go on Google, look up your name, I'd find a photo of you, perhaps on social media or LinkedIn, and then I would find audio of you," explained Ben Colman, CEO of deepfake detection company Reality Defender.

Ben Colman, CEO, Reality Defender

Attackers then place a call or leave a voicemail with urgent requests such as "This is [CEO name], I need you to process a wire immediately" or "I'm with a client, can't talk long, handle this discreetly." They target vulnerable points, including treasury or accounts payable teams, executive assistants, and IT help desks. Funds are transferred or credentials are reset, often within minutes.

What Practical Steps Can Organizations Take to Defend Against Voice Deepfake Fraud?

The good news is that this risk is highly controllable with the right discipline and organizational commitment. Security experts recommend a comprehensive approach that combines technology, process, and culture.

• Mandatory Out-of-Band Verification: For any high-risk request involving wires, credential resets, or vendor changes, require verification through a second channel using pre-established contact methods only, not what the caller provides. There should be no exceptions for "urgent" or "confidential" requests. The key principle is to trust the process, not the voice.
• No Voice-Only Authorization Rule: Prohibit wire approvals via phone alone, password resets based solely on a call, and vendor payment changes without written and independently verified confirmation. This creates a hard stop that prevents attackers from exploiting urgency and authority.
• Treasury and Accounts Payable Hard Controls: Implement dual authorization for all wires, pre-approved vendor banking details, and cooling-off periods for new payment instructions. These controls ensure that no single person can authorize a transfer, even if they are impersonated.
• Help Desk Identity Verification Protocols: Upgrade beyond knowledge-based questions. Require ticket-based workflows, manager approval for executive requests, and step-up authentication for sensitive changes. Security experts recommend asking questions that AI cannot easily guess, such as what someone had for dinner the night before.
• Executive Communication Discipline: Leadership teams must avoid behaviors that create risk. This means no last-minute "urgent" payment requests outside process, no pressure to bypass controls, and a clear message that controls apply to executives too.
• Targeted Training: Generic phishing training is insufficient. Train employees specifically on voice-based social engineering scenarios, "authority plus urgency" manipulation tactics, and exact steps to verify and escalate. Run simulations that include fake executive calls and help desk impersonation attempts.

The stakes are high, and the regulatory environment is evolving rapidly. Congress passed the Take It Down Act in 2025, which makes it a crime to share intimate images without a person's consent, including deepfake videos. However, the law does not currently cover audio deepfakes, leaving a significant gap in legal protection.

Organizations that treat security requirements, governance, and recovery planning as design constraints rather than features will protect operations, accelerate responsible innovation, and earn stakeholder confidence. Those that don't find that the same AI powering their ambitions is also their greatest vulnerability. In the race to deploy AI, the winners will be those who build resilience in from the start, not those who bolt it on at the end.