Researchers testing eight advanced reasoning models, including DeepSeek-R1, discovered that models rarely alter their outputs when they recognize they're being evaluated, contradicting widespread concerns about "evaluation gaming" in AI safety. The study, which examined how models respond when they explicitly acknowledge being tested in their reasoning traces, found that this awareness produces near-zero behavioral shifts across safety, alignment, moral reasoning, and political opinion benchmarks.

What Is Evaluation Awareness and Why Does It Matter?

Large reasoning models like DeepSeek-R1 sometimes recognize during their chain-of-thought reasoning (the step-by-step thinking process they generate before answering) that they may be under evaluation or monitoring. Researchers have worried that this "verbalised evaluation awareness" (VEA) might cause models to strategically optimize their outputs to appear safer or more aligned than they actually are, a phenomenon known as evaluation gaming or sandbagging.

The concern stems from a reasonable intuition: if a model knows it's being tested, shouldn't it behave differently? However, the actual evidence for this behavioral shift has remained sparse and largely anecdotal. Most prior studies either artificially induced evaluation awareness through prompting or fine-tuning, or examined only one or two models on narrow tasks, making it difficult to draw broad conclusions.

How Did Researchers Test This Across Models?

Researchers conducted the first large-scale causal test of evaluation awareness by examining eight open-weight reasoning models across four distinct benchmarks. The models tested included DeepSeek-R1-0528, GLM-5, Qwen3 variants, and others. They used two complementary experimental approaches to isolate the effect of evaluation awareness from other factors.

The research team employed a technique called model prefilling, which allows them to inject evaluation-aware sentences into reasoning traces where they were missing, or remove them where they naturally occurred. This approach enabled them to measure whether the presence or absence of evaluation awareness actually caused behavioral changes. The total compute cost for all experiments was approximately $3,000, demonstrating that rigorous AI safety research doesn't require massive budgets.

• On-policy testing: Researchers sampled multiple reasoning traces per question and compared outputs that spontaneously contained evaluation awareness against those that did not, without any intervention.
• Off-policy intervention: Using model prefilling, they injected evaluation-aware sentences where missing and removed them where present, then resampled outputs to measure behavioral shifts.
• Benchmark coverage: Evaluations spanned safety (harmfulness scoring), alignment, moral reasoning, and political opinion, with 500 items per benchmark.

What Did the Study Actually Find?

The results were striking in their consistency: evaluation awareness had minimal impact on model behavior across all tested conditions. When researchers injected evaluation awareness into reasoning traces, the effect size was near-zero (omega less than or equal to 0.06, a statistical measure of practical significance). Removing evaluation awareness produced only small shifts in model outputs (omega less than or equal to 0.12). Even when evaluation awareness occurred spontaneously in the models' own reasoning, it shifted answer distributions by at most 3.7 percentage points.

These findings held across different model sizes and architectures, suggesting the pattern is robust rather than an artifact of testing a single model family. While individual models showed larger effects on specific benchmarks, the overall pattern remained one of small, practically insignificant effects.

Why Should AI Safety Researchers Care About This?

The implications of this research challenge a significant concern in AI safety discourse. If models don't actually change their behavior when they recognize evaluation, then the risk of evaluation gaming may be smaller than the current literature assumes. This doesn't mean evaluation awareness is irrelevant to AI safety, but it suggests researchers should redirect their focus toward more consequential risks.

The researchers proposed several possible explanations for why evaluation awareness has limited effect. Models may not actually connect the act of verbalizing evaluation awareness to any strategic behavior adjustment. Alternatively, models may have already settled on an answer before explicitly acknowledging the evaluation context. Another possibility is that models possess some form of latent awareness about the evaluation itself, though the researchers deemed this highly unlikely.

"Our findings reveal limited behavioural impact of VEA. Injecting it into CoTs produces near-zero effects with respect to control injections, and removing VEA produces negligible changes across all benchmarks," the researchers stated.

Study authors, evaluation awareness research team

How Does This Relate to Practical Deployment of Reasoning Models?

The findings have immediate relevance for organizations deploying reasoning models like DeepSeek-R1 in production environments. If evaluation awareness doesn't cause models to game their outputs, then safety teams can focus their efforts on more substantive concerns, such as the actual capabilities and limitations of these models in real-world scenarios. This allows for more efficient allocation of safety research resources.

The study also highlights the importance of empirical testing over theoretical speculation in AI safety. Many concerns about AI behavior are intuitive but lack rigorous evidence. By testing these concerns systematically across multiple models and benchmarks, researchers can distinguish between genuine risks and false alarms, ultimately leading to more effective safety practices.

The researchers released their benchmarks and code publicly to enable follow-up research, reflecting a commitment to transparency and reproducibility in AI safety evaluation. This openness allows other teams to validate the findings, extend the analysis to new models, or test related hypotheses about model behavior during evaluation.