The US government has quietly escalated its AI competition with China by asking American tech companies to feed false or degraded information to suspected Chinese AI firms attempting to copy their models. In a joint advisory released this week, the Federal Bureau of Investigation (FBI), National Security Agency (NSA), and Cybersecurity and Infrastructure Security Agency (CISA) accused six Chinese AI companies of systematically extracting billions of tokens from leading US AI models since late 2024, and they're now recommending a controversial defensive tactic: poisoning the data stream.

What Is Model Distillation and Why Does It Matter?

Knowledge distillation is a technique where a smaller, cheaper AI model learns to mimic the outputs of a larger, more powerful model. Think of it like a student studying a teacher's work to learn the same concepts at a fraction of the cost. Chinese AI companies including DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun, and Z.AI have allegedly been using this technique at industrial scale to extract capabilities from US models like Claude, ChatGPT, Google Gemini, and Grok.

The US agencies claim these campaigns "form the core, not merely a supplement" of Chinese AI development strategy, allowing Chinese firms to dramatically reduce their training costs and development timelines. By copying US models rather than building from scratch, Chinese companies can bypass the computational expense and research investment that American firms have already sunk into their systems.

How Would Data Poisoning Actually Work?

The advisory contains a less-noticed but striking recommendation: US AI companies should deliberately alter responses sent to accounts suspected of running distillation campaigns. The agencies suggest several tactics, including adding stylistic inconsistencies, reducing reasoning depth, presenting correct information with different logic, or secretly switching suspected malicious accounts to inferior models without notifying them.

The goal is to make the stolen data less useful for training. If a Chinese firm extracts degraded or inconsistent outputs, those outputs become poor training material, reducing the payoff from the distillation attack. The advisory explicitly warns companies not to inform suspected Chinese users that they've been switched to a downgraded model, as that would allow attackers to adjust their tactics.

Why This Strategy Is Controversial

The data poisoning approach creates a significant problem: legitimate users could get caught in the crossfire. If a US AI company mistakenly flags a real researcher or developer as a distillation attacker, that person could suddenly receive shorter, less capable responses without any warning. OpenAI faced swift backlash last year when its automatic routing system "consistently defaulted to less capable variants unless users explicitly added phrases like 'think harder' to their prompt," according to reporting on the incident.

The agencies acknowledge these risks, noting that US firms should "balance security with user experience" while accepting that some trade-offs, like "lower prediction precision and business usefulness," may be inevitable. However, they recommend that AI safety researchers and third-party evaluators be informed of model changes, even if regular users are kept in the dark.

How Are Chinese Firms Currently Copying US Models?

The distillation campaigns employ sophisticated methods to avoid detection. Chinese firms have been accused of exploiting AI model inference APIs (application programming interfaces) by bulk-buying fraudulent accounts, then executing highly coordinated queries with identical or similar prompts numbering in the thousands to millions. They also use prompt injection techniques to jailbreak models and extract hidden reasoning processes.

For example, DeepSeek allegedly employed prompts instructing models to "imagine and articulate the internal reasoning behind completed responses and write it out step by step," effectively forcing the model to reveal its chain-of-thought reasoning. These campaigns span days to months with query volumes far exceeding legitimate research use, making them theoretically detectable if companies know what to look for.

Steps US AI Companies Should Take to Defend Against Distillation

• Detection and Monitoring: Develop strategies to identify malicious prompts and suspicious accounts by flagging anomalous behavior patterns, including accounts with suspicious subscription-to-usage ratios, new accounts immediately hitting maximum usage limits, and queries routed through proxy networks to evade geographic restrictions.
• Identity Verification: Strengthen identity verification of users and more closely track individuals using enterprise subscriptions to prevent bulk deployment of pre-engineered templates across multiple accounts.
• Information Sharing: Share data about malicious actors across the industry to close gaps exploited on multiple platforms, allowing companies to collectively track how distillation tactics evolve and avoid duplicating detection efforts.
• Response Modification: Employ targeted changes to responses for high-confidence malicious distillation requests, such as including differential privacy techniques or using less sophisticated models to respond to suspected attacks.

What Are Chinese Officials Saying About These Accusations?

Beijing has responded forcefully to the distillation accusations and the threat of US sanctions. Chinese Foreign Ministry spokesperson Mao Ning stated that China's AI progress "stemmed from self-reliant innovation and open cooperation," and urged the US to "stop leveling false allegations to smear China".

An unnamed Chinese Ministry of Commerce spokesperson called the US approach a double standard, noting that American firms have themselves drawn heavily on Chinese open-source models. The official characterized the US crackdown as "a textbook case of using a crackdown on distillation as a pretext for industrial monopoly" and warned that Beijing would "take resolute countermeasures if Washington used distillation as a pretext to contain Chinese AI companies".

"Model distillation itself is a technical method widely used across the global AI industry, and not all capability transfer or model learning can simply be equated with a national security threat," stated Liu Dian, a research fellow at Fudan University's China Institute.

Liu Dian, Research Fellow at Fudan University's China Institute

Chinese commentators argue that distillation is a legitimate shortcut for catching up when facing US export controls on advanced chips. Some observers note that Chinese AI companies have little choice but to compete on value and efficiency rather than premium pricing, which makes distillation an economically rational strategy.

What's the Bigger Picture in the US-China AI Race?

The distillation dispute is unfolding against a backdrop of broader US-China competition over AI dominance. China's recent return to the world's top supercomputing rankings with its LineShine supercomputer demonstrates Beijing's growing confidence in its domestic technology ecosystem, even as the system reveals significant limitations.

LineShine uses only CPUs (central processing units) rather than the more common combination of CPUs and high-performance GPUs (graphics processing units), which limits its AI computing capabilities. The system ranks only fourth on the HPL MxP benchmark designed to test AI workflows, significantly worse than the US-based El Capitan system. This CPU-only design reflects China's ongoing struggle with GPU manufacturing, a constraint that continues to influence design choices and is unlikely to meaningfully help break China's AI training bottleneck.

The data poisoning advisory comes weeks before the two countries are scheduled to hold AI safety talks in the US, where discussions are expected to focus on preventing AI-related security incidents. These talks precede a broader summit between US President Donald Trump and Chinese President Xi Jinping planned for September 24 in Washington.

Treasury Secretary Henry Bessent emphasized the stakes at a recent economic forum, stating, "There is no day after tomorrow if China wins at this. If they were to pull ahead of us on AI, then nothing else matters". The US government has already begun implementing poisoning tactics; in July, Bessent said the US found watermarks from American models in Chinese systems and threatened sanctions.