AI models don't just make mistakes in probability estimates; they make them in consistent, opposite directions depending on which company built them. A new study published on arXiv reveals that 14 of 16 tested models systematically overestimate positive outcomes, while Anthropic's two largest Claude models do the opposite, consistently underestimating success probabilities. The divergence isn't random noise,it's a fingerprint of the alignment training method each lab uses.

Why Standard AI Safety Tests Miss This Problem?

The research, which introduced a measurement framework called OptimismBench, tested models from eight major AI providers including OpenAI, Google, Meta, Mistral, DeepSeek, and Anthropic. The core finding challenges how the AI field measures whether models make reliable probability estimates. Existing benchmarks, such as Expected Calibration Error, measure overall accuracy but fail to catch directional bias because overestimates and underestimates cancel each other out in aggregate scoring.

Researchers Seonglae Cho and Adriano Koshiyama of Holistic AI and University College London illustrated the problem with a concrete example: an AI that rates a startup's success at 70% but failure at only 15% appears well-calibrated in standard metrics. Yet the two estimates should sum to approximately 100%. The missing 15 percentage points represent a systematic distortion that no existing benchmark would flag. For domains where AI assists with risk assessment, investment screening, medical outcome estimation, or project forecasting, this is not a minor statistical quirk.

How Does OptimismBench Detect Bias Without Real-World Outcomes?

The benchmark's innovation lies in its method for detecting directional bias without requiring ground-truth data. The researchers tested each model on inverted pairs: for the same event, they asked the model to estimate both the probability of success and the probability of failure. Logically, these should sum to 100%. A model that consistently produces estimates where success probability exceeds 100% minus failure probability is systematically optimistic; the opposite pattern indicates pessimism.

The full benchmark spans 3,870 items across 10 languages and covers four axiom batteries: conjunction fallacies, conditional probability, dose-response monotonicity, and explicit calibration items. A secondary finding emerged: the language in which you pose a probability question to an AI makes almost no difference to the direction of its error. Inter-model variance in directional bias was 4.7 times larger than inter-language variance, meaning which lab built the model matters far more than whether you ask in French, Japanese, or English.

What Do the Results Show About Different AI Models?

All 16 models tested showed statistically significant directional bias. None were neutral. The pattern researchers describe as "sign by lab, magnitude by scale" has two components: first, which direction a model errs is largely determined by the lab that built it; second, how large the error is scales with model size within a given lab's family.

• Optimistic Models: Fourteen of the 16 models tested, including OpenAI's GPT family, Google's Gemini, Meta's Llama variants, Mistral, DeepSeek, and Zhipu's GLM series, consistently overestimate the probability of positive outcomes.
• Pessimistic Models: Claude Sonnet 4.6 and Claude Opus 4.6 from Anthropic are the only exceptions, systematically underestimating success probabilities relative to what their failure-probability estimates imply.
• Scale Matters Within Labs: Claude Haiku 4.5, Anthropic's smallest model, shows optimistic bias with a Skew score of plus 6.1, placing it alongside most other models tested, suggesting the pessimistic pattern is tied to scale rather than being a blanket property of all Anthropic models.

How Does Post-Training Alignment Create These Biases?

The most consequential finding distinguishes this research from earlier work on AI overconfidence: the mechanism behind the bias. By comparing 11 matched pairs of base models against their instruction-tuned or chat-aligned counterparts across four model families, the team observed directly how directional bias changes during post-training, the stage where models are fine-tuned using alignment techniques.

Post-training does not merely amplify a bias that was already present in the base model. It actually sets the sign of the directional distortion. Even more striking, the direction of the shift differs between model families. In some families, post-training pushes the model toward optimism; in Anthropic's frontier tier, it pushes in the opposite direction.

The training approach is the most plausible explanation for this divergence. Reinforcement Learning from Human Feedback, or RLHF, is the dominant post-training technique across most major AI labs. RLHF is well-documented to amplify sycophantic behavior because human annotators systematically prefer responses that are encouraging and positively framed. A model trained to produce human-preferred outputs learns to frame things optimistically because optimism is preferred. This is the structural mechanism through which RLHF installs an optimistic directional tilt.

Anthropic's Constitutional AI approach uses principle-based feedback rather than raw human approval ratings, which does not have this structural bias toward positive framing. This difference in alignment methodology explains why Anthropic's largest models show the opposite pattern from the rest of the industry.

Steps to Understand Directional Bias in AI Decision-Making

• Recognize Inverted Pairs: When using AI for probability estimates, ask the model for both the probability of success and failure for the same event, then verify they sum to approximately 100 percent to catch directional distortion.
• Know Your Model's Training Method: Understand whether the AI system you're relying on uses RLHF-based alignment, which tends to produce optimistic bias, or principle-based approaches like Constitutional AI, which may produce different patterns.
• Apply Skepticism to High-Stakes Decisions: When using AI for risk assessment, investment screening, or medical outcome estimation, treat probability estimates as one input among many rather than as ground truth, especially when the estimates seem suspiciously aligned with a positive or negative outcome.
• Request Transparency on Alignment Methods: When evaluating AI systems for critical applications, ask vendors to disclose their post-training alignment approach and any known directional biases in probability estimation.

The implications extend beyond academic interest. For any domain in which AI is used as a decision aid, a systematic thumb on the scale toward optimism or pessimism can compound over time. A venture capital firm using GPT-family models for startup screening might systematically overestimate success probabilities across its entire portfolio. A hospital using a different model for patient outcome prediction might systematically underestimate risks. Neither scenario is acceptable when human welfare is at stake.

This research opens a new frontier in AI alignment evaluation. While constitutional AI and RLHF are both widely discussed in AI safety circles, their concrete effects on something as fundamental as probability estimation have rarely been measured this directly. The finding that alignment choices, not model size, determine directional bias suggests that how labs choose to train their models has consequences far beyond the obvious measures of helpfulness and harmlessness that dominate current safety benchmarks.