Major artificial intelligence models, including GPT-4o and Gemini 2.5 Pro, continue to exhibit significant gender, racial, and socioeconomic biases despite their advanced capabilities. A comprehensive benchmark study published in July 2026 tested 14 leading large language models (LLMs) on 66 bias evaluation questions across multiple demographic categories, revealing that even state-of-the-art systems inherit and amplify human prejudices embedded in their training data.

The findings underscore a critical challenge facing the AI industry: as these models become more integrated into high-stakes decisions like hiring, healthcare, and criminal justice, their biases can have real-world consequences for millions of people. The research tested models across gender, race, age, disability status, socioeconomic status, and sexual orientation, using both open-ended and multiple-choice question formats to isolate whether the phrasing itself introduced bias.

What Specific Biases Did the Study Uncover?

The benchmark revealed troubling patterns across multiple categories. In one racial bias test, GPT-4o was presented with a scenario where race was the only distinguishing factor between suspects. Rather than declining to answer, the model cited statistical crime rates for a specific race as justification and concluded that the perpetrator was "most likely" from that race. This type of reasoning perpetuates harmful stereotypes by treating statistical correlations as causal evidence.

Gender bias appeared equally pronounced. When researchers presented Gemini 2.5 Pro with identical scenarios using stereotypical male and female names, asking which person might be a doctor versus a nurse, the model identified the male as the doctor and the female as the nurse, even though the prompt explicitly stated that models could answer "cannot be determined" for any question. This suggests the bias was not a limitation of the question format but rather a reflection of the model's training data.

Socioeconomic bias also emerged in unexpected ways. When presented with scenarios where one suspect was described as wealthy and another as financially struggling, several LLMs indicated the less affluent person was "most likely" guilty, despite having no other distinguishing information. These assumptions reveal how models can encode and amplify existing social inequalities.

How Are These Biases Entering AI Systems?

The root cause of these biases lies in the training data itself. AI models learn patterns from vast datasets of text, images, and other information scraped from the internet and historical records. If those datasets contain human biases, the models will learn and reproduce them. A 2024 UNESCO study analyzing major LLMs found they associate women with "home" and "family" four times more often than men, while disproportionately linking male-sounding names to "business," "career," and "executive" roles. This is not a minor discrepancy; it directly reproduces societal gender stereotypes found in the training data.

Real-world examples demonstrate the tangible harms. A health care risk-prediction algorithm used on more than 200 million U.S. citizens was found to demonstrate racial bias because it relied on previous patients' healthcare spending as a proxy for medical need. Since income and race are highly correlated, this flawed metric systematically favored white patients over Black patients. Similarly, facial recognition systems developed by major tech firms have misidentified darker-skinned women at error rates as high as 35%, while light-skinned men had error rates below 1%, according to research from MIT Media Lab's Joy Buolamwini in 2018.

Ways to Reduce Bias in AI Systems

• Diversify Training Data: Ensure datasets used to train models include balanced representation across racial, gender, age, and socioeconomic categories. Homogeneous training data is a primary driver of biased outputs.
• Implement Bias Auditing and Testing: Regularly benchmark models against standardized bias evaluation frameworks, similar to the 66-question study, to identify and measure biases before deployment in high-stakes applications.
• Redesign Proxy Metrics: Avoid using correlated variables as proxies for outcomes. For example, do not use healthcare spending as a proxy for medical need, as this conflates income with actual health requirements.
• Establish Diverse Development Teams: Include people from different racial, gender, and socioeconomic backgrounds in AI development and decision-making to catch biases that homogeneous teams might overlook.
• Use Multiple Question Formats: Test models with both open-ended and multiple-choice formats to ensure bias is not introduced by the question structure itself.
• Prioritize Transparency and Accountability: Make bias evaluation results public and establish clear accountability mechanisms when biased systems cause harm, as Facebook did when it ended allowing employers to target job ads by age, gender, or race.

The benchmark study noted one bright spot: Claude 4.5 Sonnet notably avoided most of the gender and socioeconomic bias errors that other models exhibited, suggesting that some architectural choices or training approaches may be more effective at mitigating these problems.

The implications extend beyond individual models. When biased AI systems are deployed in hiring, lending, criminal justice, and healthcare, they can systematically disadvantage entire groups of people. A 2024 University of Washington study investigating gender and racial bias in resume-screening AI tools found that the systems favored names associated with white males, while resumes with Black male names were never ranked first. These findings highlight how even advanced AI can perpetuate harmful stereotypes when trained on unbalanced data.

As AI becomes increasingly central to consequential decisions, the industry faces mounting pressure to address these biases. The 2026 benchmark study provides a roadmap for what needs to change: more diverse training data, rigorous bias testing before deployment, and a commitment to transparency about where and how these systems fail. Without these changes, AI risks becoming a tool that automates and scales human prejudice at unprecedented speed and scale.