Yann LeCun, Meta's longtime chief AI scientist, confirmed in January 2026 that Meta manipulated the benchmark results for its Llama 4 model by using different model versions for different tests. Rather than testing a single model fairly, Meta selected whichever variant scored highest on each benchmark and compiled those cherry-picked results into a single table, presenting it as though one model had achieved all the scores. The revelation exposed a much larger problem: the entire AI industry runs on incentives that reward benchmark gaming, and Meta was simply the company that got caught.

What Exactly Did Meta Do With Llama 4 Benchmarks?

When Meta launched Llama 4 in April 2025, the company published benchmark tables claiming the model performed "equally well or better" than closed-source competitors from OpenAI and Google. The numbers looked impressive on paper. Within days, independent testers running their own evaluations found something was wrong. Community-run tests showed significantly lower scores than Meta had published. On LMSys Chatbot Arena, the most visible public benchmark, Llama 4's Maverick variant dropped from 2nd place to 32nd place, while the Scout variant fell out of the top 100 entirely.

Meta's initial response was denial. Ahmad Al-Dahle, a Meta vice president, attributed the discrepancies to "cloud differences" between Meta's internal testing environment and public deployment. The explanation satisfied almost no one. The AI community had already identified the actual problem: Meta was not testing one model. It was testing multiple models and reporting the best score from each.

LeCun's January 2026 interview with the Financial Times confirmed the manipulation. "The results were fudged a little bit," he told the publication. The team "used different versions of the model for specific benchmarks, completely violating the principle of fair evaluation". The method was straightforward: train multiple checkpoints, run each on every benchmark, select the highest score per test, and compile those cherry-picked results into a single table. No single model actually achieved the published results.

Why Did LeCun Leave Meta, and What Changed Afterward?

LeCun departed Meta after more than a decade shaping the company's AI strategy, and the benchmark scandal was central to the breakdown. Notably, no one at Meta was fired for the manipulation. Instead, the company created a new organization called Meta Superintelligence Labs (MSL) under Alexandr Wang, the former Scale AI CEO, effectively replacing the AI research leadership that had overseen Llama 4. The organization was split into four divisions: TBD Lab for foundation models under Wang himself, FAIR for research, Products, and Infrastructure.

Wang, who joined Meta in June 2025 after Meta acquired a 49 percent stake in Scale AI in a deal valued at $14.3 billion, has defended Meta's hiring approach and the new organizational structure. In a podcast interview published in May 2026, Wang argued that top AI researchers joined Meta not just for money, but for culture and computing resources. "It's an incorrect assumption to think that the researchers are just money-motivated," Wang said. He pointed to high compute per researcher, streamlined teams, and a willingness to back ambitious research bets as key draws for talent.

Wang

However, Wang has also faced criticism from LeCun himself. In a January Financial Times interview, LeCun described Wang as "young" and "inexperienced". The two appear to have since reconciled: they met in India a few weeks after the interview, where LeCun congratulated Wang on MSL's recent Muse Spark model release.

How to Evaluate AI Models Responsibly: What the Industry Should Do

• Independent Third-Party Evaluation: Establish evaluation bodies that labs do not control, removing the conflict of interest where companies test their own models and report the results.
• Dynamic Benchmarks: Create benchmarks that change with each evaluation cycle, making it impossible for labs to overfit their models to static test sets.
• Mandatory Disclosure: Require labs to publish the exact model version, training date, and checkpoint used for each published score, preventing the cherry-picking of results.
• Community-Run Testing: Prioritize independent, community-run evaluations like LMSys Chatbot Arena over official company benchmarks, as these are harder to manipulate.

Is This Problem Unique to Meta, or Systemic Across AI?

The Llama 4 scandal is not about one model cheating on one set of benchmarks. It is about an entire evaluation economy that incentivizes exactly this behavior. The AI industry runs on leaderboards. Companies choose models based on benchmark scores. Investors value AI labs based on leaderboard positions. Researchers build careers on benchmark improvements. The incentive to game those benchmarks is structural, not incidental.

Meta's sin was not that it fudged benchmarks. It was that it got caught, and that its own chief scientist confirmed it. Every major AI lab optimizes for benchmarks. The difference is that most labs have not had their most senior researcher publicly admit that the optimization crossed the line into manipulation. The community's response on r/LocalLLaMA was immediate and lasting: trust in official benchmark numbers has been permanently eroded.

One analysis estimated that a $10 billion industry has been built on leaderboard gaming, where model selection strategies are driven by scores that may not reflect real-world performance. When the scores are unreliable, every downstream decision built on them, which model to use, which lab to invest in, which API to integrate, is also unreliable.

Meta's structural response confirms the systemic nature of the problem. The company did not punish anyone for the manipulation. It created a new organization under new leadership. The message was clear: the Llama 4 team's execution was the problem, not the underlying incentives that produced it. Those incentives remain unchanged. Every AI lab today faces the same pressure Meta faced: publish numbers that beat the competition, or lose funding, talent, and market position to the labs that do.

"Benchmarks are not neutral measurements. They are targets. And when you make something a target, people will aim at it," according to analysis of the scandal's implications.

Analysis from Remio.ai reporting on the Llama 4 benchmark investigation

LeCun's confession matters precisely because it came from inside the system. He was not an external critic with an agenda. He was the person responsible for Meta's AI research for more than a decade. And he still said the results were fudged. When the insiders stop believing the benchmarks, the rest of us should stop too.

The community has already begun building alternatives. LMSys Chatbot Arena and similar platforms have gained credibility because they are harder to game than static benchmarks. Head-to-head blind comparisons judged by human preference are replacing automated accuracy metrics. But even arenas have their own manipulation vectors. The only evaluation that cannot be gamed is the one you run yourself, on your own data, for your own use case.

Meta built its AI reputation on open-source leadership. Llama 1, 2, and 3 established the company as the champion of open-weight models, the counterweight to OpenAI and Google's closed ecosystems. Llama 4 was supposed to extend that legacy. Instead, it became the symbol of everything the open-source community distrusts about corporate AI: the benchmarks are rigged, the transparency is selective, and the claims cannot be verified without independent testing.