South African businesses face a critical vulnerability: 63% of employees say they would likely miss a deepfake attack in the workplace, while 86% no longer trust what they see and hear online. As artificial intelligence becomes embedded in daily work, organizations are struggling to protect workers from increasingly sophisticated impersonation scams, voice cloning, and fraudulent video content.

Why Are Deepfakes and AI Scams So Hard to Spot?

The problem runs deeper than simple awareness. According to research from digital workforce security company KnowBe4, the challenge isn't just that AI-generated content looks and sounds real. It's that employees face mounting pressure and distractions that make them vulnerable to manipulation. Nearly 60% of workers admitted that workplace stress, time pressures, and routine distractions cause them to make security mistakes, even when they know the correct procedures.

Attackers are exploiting this human factor with precision. Authorized push payment (APP) fraud, where scammers impersonate bank employees or government officials to convince victims to transfer money, has become one of the fastest-growing payment fraud threats globally. AI-generated audio, video, and other content make these impersonation scams more convincing and easier to scale across thousands of targets simultaneously.

What Is "Shadow AI" and Why Should Organizations Care?

One of the most alarming findings from the KnowBe4 research is the widespread adoption of unsupervised artificial intelligence tools. Nearly 64% of South African organizations reported that AI is being used without proper governance or approval, a phenomenon the report calls "Shadow AI." More than one-third of employees, or 35%, admitted they would source their own AI applications if approved options were unavailable or too restrictive.

This creates a dual vulnerability. Not only are employees exposed to external attacks, but internal AI systems are also becoming targets. The report found that 38% of South African cyber security leaders said autonomous AI agents are already performing tasks independently within business workflows, yet many organizations lack the governance structures to manage these systems safely.

"Cybersecurity has entered a volatile phase where organizations are trying to secure a hybrid human and AI workforce that's changing more quickly than security leaders can keep up. Attackers are moving at machine speed, using attacks such as deepfakes to target employees and prompt injections to hijack AI agents. Leaving more than half, 64%, of your corporate AI usage ungoverned is a massive open invitation to threat actors," said Anna Collard, Senior Vice President of Content Strategy and CISO Advisor at KnowBe4 Africa.

Anna Collard, Senior Vice President of Content Strategy and CISO Advisor at KnowBe4 Africa

How Payment Fraud Is Becoming Automated and Harder to Stop

The threat extends beyond workplace impersonation. Payment fraud is becoming increasingly organized and automated, with criminal groups using AI to streamline every stage of the attack. Advances in agentic AI, which refers to AI systems that can autonomously plan and execute tasks, could automate credential collection, document forgery, and even password-cracking tools.

Account takeover fraud remains one of the most common payment fraud threats, with 35% of survey respondents ranking it as a top concern. Attackers obtain credentials through phishing campaigns, data breaches, and dark web marketplaces, then use stolen login information to access bank accounts, transfer funds, or open fraudulent credit accounts. Traditional defenses like passwords, security questions, and some forms of multi-factor authentication (MFA) are becoming less effective as criminals use AI to improve phishing campaigns and automate credential theft.

Card fraud is also shifting to digital channels. More than 90% of credit card fraud now involves cards that remain in the owner's possession, meaning criminals are stealing card data rather than the physical cards themselves. Website skimming and automated card credential testing are becoming more common, with North America accounting for 51% of global website skimming detections.

Steps Organizations Can Take to Build Stronger Defenses

• Establish AI Governance Frameworks: Organizations must create clear policies for how AI tools can be used, who can access them, and how they will be monitored. This includes both approved enterprise tools and shadow AI systems that employees may already be using without authorization.
• Invest in Security Culture Over Technology Alone: The research found that organizations achieving the strongest cyber security outcomes were those that embedded security into workplace culture rather than treating it purely as a technology function. Among these organizations, 95% of employees felt comfortable reporting mistakes without fear of blame, enabling faster threat detection and response.
• Provide Targeted Employee Training: Workers need practical training on how to recognize deepfakes, impersonation attempts, and AI-generated scams. This should include real-world examples and regular refresher sessions, not just one-time awareness programs.
• Implement Multi-Layered Identity Verification: As deepfake technology improves, organizations should move beyond voice and facial biometric authentication alone. Combining multiple verification methods, including knowledge-based questions and behavioral analysis, can reduce the risk of account takeover fraud.
• Monitor AI Agent Activity: Since 38% of cyber security leaders reported that autonomous AI agents are already performing tasks independently, organizations need visibility into what these systems are doing and who can access them.

How Far Behind Are Organizations in Preparing for These Threats?

Despite growing awareness of AI-driven risks, many organizations remain unprepared. Only 14% of South African organizations have achieved what KnowBe4 describes as the "gold standard," where organizations have fully integrated strategies to manage both human and AI-related cyber risks. Less than half of cyber security leaders, just 46%, said they felt very well prepared to deal with emerging AI-driven threats over the coming year.

The gap between threat velocity and defensive capability is widening. More than six in ten cyber security leaders, or 62%, reported that mistakes made during routine daily work had the greatest impact on their organizations' cyber security over the past year. This suggests that even well-intentioned employees are struggling to keep pace with the speed and sophistication of modern attacks.

As artificial intelligence continues to transform workplaces globally, the findings from South Africa offer a cautionary tale. Organizations that fail to govern their AI systems, invest in security culture, and equip employees to recognize AI-generated scams will remain vulnerable to a new generation of cyber threats that move at machine speed and exploit human psychology with unprecedented precision.