Advanced AI reasoning models, including OpenAI's o3, stereotype job candidates far more aggressively than humans do, according to a new study from Princeton University and the University of Chicago. Researchers tested ChatGPT, Claude, Gemini, and DeepSeek's R1 in a simulated hiring game and found that reasoning-heavy models locked in ethnic stereotypes faster and more rigidly than any other AI system tested.

What Did the Princeton Study Actually Test?

Researchers ran a 40-round hiring simulation where each model acted as a consultant filling 20 different job roles, from doctors and lawyers to janitors and childcare aides. The candidate pool consisted of four fictional ethnic groups: Tufa, Aima, Reku, and Weki. In each round, one candidate from each group applied for a position, the model selected one, and then received feedback on whether that hire succeeded. Crucially, all candidates were equally likely to succeed at every job, so any patterns the models developed were pure stereotyping.

The models didn't hesitate to generalize. When a model learned that an Aima candidate had failed as a doctor, it immediately began sorting Aimas away from professional roles and into janitorial work instead. Human participants in the original psychology experiment reached a segregation score of 0.84 by the end of the 40 rounds. The AI models blew past that benchmark, with OpenAI's o3 scoring 1.83 on a segregation scale where 2 represents complete ethnic sorting across all job categories.

Why Are Reasoning Models the Worst Offenders?

The finding contradicts a widespread assumption in AI development: that better reasoning capabilities lead to more careful, less biased decision-making. Instead, the opposite occurred. The models most praised for step-by-step thinking and transparent reasoning traces were the fastest to lock in stereotypes and the hardest to change their minds.

"LLMs are optimized on math, coding, and science problems that reward extracting patterns from small samples, and that same instinct misfires in social settings," explained Ryan Liu, a Princeton PhD student and coauthor of the paper.

Ryan Liu, PhD Student, Princeton University

This pattern-extraction instinct, which makes reasoning models excel at technical problems, becomes a liability in hiring scenarios. The models treat a single failed hire as a reliable signal about an entire demographic group's suitability for a role, then use that signal to make increasingly confident predictions about future candidates.

What Fixes Actually Work for Reducing AI Hiring Bias?

The researchers tested multiple interventions to reduce stereotyping. Simply instructing the models to be fair produced almost no change in behavior. However, changing the underlying incentive structure worked dramatically. When researchers offered the models an explicit bonus for diverse hiring outcomes, bias dropped sharply.

The models also became less biased when given relevant personal information about candidates, such as age and education history. However, they reverted to ethnic stereotyping when given irrelevant information like hair color or tattoo shape, suggesting the models were still searching for patterns to exploit.

• Instruction-Based Fairness: Telling models to be fair or avoid bias produced almost no measurable change in stereotyping behavior during the hiring simulation.
• Incentive Alignment: Offering explicit bonuses for diverse hiring outcomes significantly reduced bias across all tested models.
• Relevant Information: Providing candidates' actual qualifications like education and work history reduced reliance on ethnic stereotyping.
• Irrelevant Information: Adding noise data like hair color or tattoo details caused models to revert to ethnic-based sorting patterns.

Why This Matters for Real-World AI Hiring Systems

The implications extend far beyond academic research. This study identifies a failure mode that standard bias audits won't catch. Companies typically audit AI hiring tools for bias inherited from training data, assuming a model trained on clean data will make fair decisions. This research shows that a model can arrive at deployment completely unbiased and still invent new stereotypes after processing just a handful of hiring outcomes.

"A model can arrive at a job clean and still invent a bias against Aima doctors after three data points," noted Ryan Liu, describing what he calls "novel biases" that emerge during deployment.

Ryan Liu, PhD Student, Princeton University

The problem becomes more acute as companies deploy AI systems with persistent memory and personalization features. A chatbot that remembers every prior interaction has more raw material to over-generalize from, according to Angelina Wang, a Cornell computer scientist who reviewed the research.

The regulatory and legal landscape around AI hiring has focused almost entirely on preventing models from repeating historical biases. This study points to a second failure mode: models actively generating new biases through pattern-matching on limited feedback. Expect the next round of AI hiring lawsuits to cite exactly this kind of research.

What's the Practical Challenge for AI Companies and Enterprises?

For the AI industry, the uncomfortable takeaway is that alignment through instruction keeps failing where alignment through incentive succeeds. This has significant implications for every vendor selling an HR copilot or hiring assistant. The fix is not a system prompt or fairness instruction, but a redesigned objective function, and objective functions are what enterprise buyers rarely have the appetite or expertise to tune.

The study was presented at ICML, a major machine learning conference, in Seoul in July 2026. The research tested four of the most widely deployed AI systems in the world, making the findings directly relevant to hiring decisions happening at scale across Fortune 500 companies and mid-market enterprises today.