Advanced AI models stereotype job candidates 65% more aggressively than humans do, according to new research from Princeton University and the University of Chicago. The study tested OpenAI's o3 reasoning model alongside Claude, Gemini, and DeepSeek's R1 in a simulated hiring scenario, revealing a troubling pattern: the models most praised for step-by-step logical thinking were the fastest to lock in ethnic stereotypes and the hardest to correct.

What Did the Princeton Hiring Study Actually Test?

Researchers created a simulated hiring game where each AI model acted as a consultant filling 20 different job roles, from doctors and lawyers to child-care aides and janitors. The candidate pool belonged to four fictional ethnic groups: Tufa, Aima, Reku, and Weki. In each round, one candidate from each group applied for a position, the model selected one person, and then received feedback on whether that hire succeeded. Critically, all candidates were equally likely to succeed at every job, meaning any patterns the models developed were purely learned biases.

The results were stark. OpenAI's o3 scored 1.83 on a segregation scale where 2 represents complete ethnic sorting into separate job categories. By comparison, human participants in the original psychology experiment reached a segregation score of 0.84. The models didn't hesitate to generalize from minimal data: when told that a candidate from the Aima group had failed as a doctor, the models stopped hiring Aimas for that role and began slotting them into janitorial positions instead.

Why Do Reasoning Models Perform Worse Than Other AI Systems?

The counterintuitive finding challenges a widespread assumption in AI development: that better reasoning capabilities lead to fairer decisions. Instead, the opposite occurred. Reasoning-heavy models like o3 and DeepSeek's R1 posted the worst segregation scores, suggesting that the very training methods that make these systems excel at math, coding, and science problems may be backfiring in social contexts.

"LLMs are optimized on math, coding, and science problems that reward extracting patterns from small samples, and that same instinct misfires in social settings," explained Ryan Liu, a Princeton PhD student and coauthor of the paper.

Ryan Liu, PhD Student, Princeton University

Liu further noted that the models either cannot translate fairness values into action or that process gets submerged under the drive to optimize for hiring accuracy. This distinction matters enormously for companies deploying AI in human resources, because it suggests the problem is not simply a matter of better instructions or system prompts.

How to Address AI Bias in Hiring Systems

• Incentive Redesign: When researchers offered the models an explicit bonus for diverse hiring, bias dropped sharply, demonstrating that changing the objective function works where instruction-based fairness appeals fail.
• Relevant Context Provision: Models became less biased when given meaningful personal information about candidates, such as age and education, but reverted to ethnic sorting when provided irrelevant details like hair color or tattoo shape.
• Objective Function Tuning: The practical fix requires redesigning the underlying goal the model optimizes for, not simply adding fairness language to system prompts, which enterprise buyers rarely have the expertise to implement.

The finding that instruction-based fairness produced almost no change in behavior has serious implications for the AI industry. Most vendors selling HR copilots rely on system prompts and fairness guidelines, but this research suggests those approaches are largely ineffective. The real solution requires changing the objective function itself, a technical intervention that goes far beyond what most enterprise customers are equipped to handle.

What Does This Mean for Real-World Hiring?

The experiment gave models instant feedback on every hiring decision, a luxury that real-world resume screening systems rarely enjoy. A Fortune 500 company using AI to filter applications might wait months for performance reviews, and often never receives clean signal about whether a hire succeeded. However, the underlying mechanism remains the same: a model reading too much into whatever outcomes it does observe and generalizing those patterns into rigid categories.

Angelina Wang, a Cornell computer scientist who did not work on the paper, emphasized that these findings matter more now that chatbots are shipping persistent memory and personalization features. A system that remembers every prior interaction has more raw material to over-generalize from, amplifying the risk of entrenched stereotypes.

"We still are trying to figure out just the right amount that isn't too much or too little," Wang said regarding how much memory systems should retain.

Angelina Wang, Computer Scientist, Cornell University

The regulatory landscape around AI hiring tools has focused almost entirely on bias inherited from training data, the assumption being that models simply repeat patterns learned from decades of biased human decisions. This study points to a second failure mode that audits of training data cannot catch: a model can arrive at a job screening task clean and still invent novel biases against specific groups after just three or four data points. That distinction is critical for policymakers and companies deploying these systems at scale.

For the AI industry, the practical takeaway is clear: alignment via instruction keeps failing where alignment via incentive succeeds. That has uncomfortable implications for every vendor selling an HR copilot, because the fix is not a system prompt but a redesigned objective function. Expect the next round of AI hiring lawsuits to cite exactly this kind of research as evidence that vendors knew or should have known about these failure modes.