When an AI safety test shows a score change during training, it might mean the model became riskier, or it might just mean the model became pickier about how questions are phrased. A new research initiative is trying to tell the difference, and the stakes are high: if safety evaluators can't distinguish between real behavioral shifts and measurement artifacts, they risk sounding false alarms or missing genuine problems.

The issue centers on a training technique called RLVR, or reinforcement learning with verifiable rewards. Labs use RLVR to teach reasoning models on tasks with automatically checkable answers, like math or code problems. The method works well for building capability, but its safety effects are messier. A score might drop because the model actually became safer, or because the model became hypersensitive to answer order, wording, or formatting quirks in the test itself.

An independent AI safety researcher based in Karachi recently completed a checkpoint study on the Tülu 3.1 training trajectory, examining 11 training snapshots across 24 authored safety scenarios. The study tested four different phrasings of each scenario and all six possible answer orders, producing 6,912 structured responses. The finding was striking: average risky behavior stayed stable throughout training, within a predeclared safety band. But the test itself became less reliable. Permutation consistency fell from 0.302 to 0.156, and answer-order sensitivity spiked sharply late in training.

Why Does This Matter for AI Safety?

The distinction between behavioral drift and measurement drift is not academic. If a safety evaluator misreads a measurement artifact as a real safety problem, they waste attention and credibility on a false alarm. If they miss a genuine behavioral shift because it hides behind measurement noise, they could overlook a real risk. The goal is to make safety evidence from training trajectories trustworthy enough that labs and regulators can act on it with confidence.

The SPAR Project, a mentorship initiative launching this fall, is designed to replicate and stress-test these findings on a second open training lineage. The team will freeze all evaluation rules before looking at results, test across multiple prompt wordings and answer orders, and check whether structured and open-ended response formats agree. If the result replicates, it would support a general claim about RLVR. If it fails to replicate, that would be equally informative, suggesting the first finding was tied to a particular model or evaluation design.

How to Spot Measurement Drift in AI Safety Evaluations

• Test Multiple Phrasings: Run the same safety scenario with four or more meaning-preserving wordings to see if the model's behavior changes based on how the question is asked, not what it asks.
• Check All Answer Orders: Present the same multiple-choice answers in different sequences; if the model's choice flips based on position, that signals answer-order sensitivity rather than stable reasoning.
• Compare Structured and Open-Ended Formats: Use both constrained responses and free-form answers to see if the safety signal holds across different response types.
• Freeze Analysis Rules Before Training Ends: Decide on thresholds for behavioral and measurement change before examining results, preventing post-hoc adjustments that could mask drift.
• Track Intermediate Checkpoints: Monitor safety metrics at multiple training snapshots, not just at the start and end, to catch when sensitivity shifts occur.

The research team is looking for mentees to own substantial pieces of the replication work. Candidates should be strong in Python, comfortable with transformer language models using PyTorch and Hugging Face, and familiar with statistical methods like bootstrap intervals and repeated-measure designs. The commitment is at least 8 hours per week, with one weekly meeting between 14:00 and 18:00 UTC.

"Labs are using RLVR and related methods to build stronger reasoning models. Safety evaluations often run alongside capability training, but a raw score change can be misread if training also changes how the model reacts to the test," the researcher explained.

Senior AI Software Engineer and Independent AI Safety Researcher, OCN

The mentees will split ownership by workstream. One person might lead dataset and semantic review, another might own inference and analysis, and a third could handle open-ended evaluation and robustness checks. Each mentee will have a concrete deliverable they can take from implementation through written analysis, and everyone will contribute to the final report and public release.

The goal is to produce three concrete outputs: a public evaluation harness that other researchers can use, a checkpoint-level dataset with frozen analysis rules, and a technical report documenting what changed and why. If the replication succeeds, it would give evaluators a reproducible way to audit open RLVR trajectories and detect answer-order, wording, and formatting effects before they turn into safety claims. If it fails, that failure would itself be valuable, narrowing the scope of what we know about RLVR's safety profile.

Applications for the SPAR mentorship are open through August 18, 2026. Candidates must read the existing study on GitHub, choose one design choice they would change before replicating, and explain why in 250 words or fewer. They must also propose two plausible explanations for a scenario where average behavior stays flat but answer-order sensitivity rises, and describe an experiment to distinguish them.