Meta's Llama 4 launched in April 2025 with impressive benchmark claims, but independent testers found significantly lower scores. In January 2026, Yann LeCun, Meta's outgoing chief AI scientist, confirmed to the Financial Times that the published results were "fudged a little bit." The team had used different model variants for different tests, selecting the highest score from each variant and compiling them into a single table as though one model had achieved all of them.

What Happened to Meta's Llama 4 Benchmarks?

When Llama 4 launched with two variants, Scout and Maverick, Meta's blog post claimed the models performed "equally well or better" than closed-source competitors from OpenAI and Google. The community became skeptical within days. Independent testers running their own evaluations consistently found lower scores than Meta had published. On LMSys Chatbot Arena, the most visible public benchmark, Maverick dropped from 2nd place to 32nd, while Scout fell out of the top 100 entirely.

Meta's initial response blamed "cloud differences" between internal testing and public deployment. Ahmad Al-Dahle, a Meta VP, attributed the discrepancies to environmental variations. The explanation satisfied almost no one. The community had already identified the real problem: Meta was not testing one model across benchmarks. It was testing multiple models and reporting the best score from each.

"The results were fudged a little bit. The team used different versions of the model for specific benchmarks, completely violating the principle of fair evaluation," said Yann LeCun.

Yann LeCun, Chief AI Scientist at Meta

LeCun's method was straightforward: train multiple checkpoints, run each on every benchmark, select the highest score per test, and compile into one table. No single model actually achieved the published results. No one was fired. Instead, Meta created Superintelligence Labs under Alexandr Wang, the former Scale AI CEO, effectively replacing the AI research leadership that had overseen Llama 4.

Why Does This Problem Extend Beyond One Model?

The Llama 4 scandal is not about one model cheating on one set of benchmarks. It reveals a structural problem in how the entire AI industry operates. The AI industry runs on leaderboards. Companies choose models based on benchmark scores. Investors value AI labs based on leaderboard positions. Researchers build careers on benchmark improvements. The incentive to game those benchmarks is structural, not incidental.

Meta's sin was not that it fudged benchmarks. It was that it got caught, and that its own chief scientist confirmed it publicly. One analysis estimated that a $10 billion industry has been built on leaderboard gaming, where model selection strategies are driven by scores that may not reflect real-world performance. When the scores are unreliable, every downstream decision built on them becomes unreliable: which model to use, which lab to invest in, which API to integrate.

Meta was not unique in optimizing for benchmarks. It was unique in getting caught and having the chief scientist confirm it. Every major AI lab optimizes for benchmarks. The difference is that most labs have not had their most senior researcher publicly admit that the optimization crossed the line into manipulation. The community's response on r/LocalLLaMA was immediate and lasting: trust in official benchmark numbers has been permanently eroded.

How Can the AI Industry Fix Its Evaluation Problem?

• Independent Third-Party Evaluation: Create evaluation bodies that labs do not control, removing the direct incentive to manipulate results for marketing purposes.
• Dynamic Benchmarks: Change benchmarks with each evaluation cycle to make overfitting and optimization impossible, forcing genuine capability improvements.
• Mandatory Disclosure: Require labs to publish the exact model version used for each published score, preventing the practice of cherry-picking variants.
• Cultural Shift: Value real-world reliability over leaderboard position, rewarding models that perform well on diverse tasks rather than optimized benchmarks.

None of these reforms are technically difficult. All of them are politically difficult, because they threaten the marketing machinery that the AI industry has built around benchmark supremacy. The community has already begun building alternatives. LMSys Chatbot Arena and similar platforms have gained credibility because they are harder to game than static benchmarks. Head-to-head blind comparisons judged by human preference are replacing automated accuracy metrics.

Meta built its AI reputation on open-source leadership. Llama 1, 2, and 3 established the company as the champion of open-weight models, the counterweight to OpenAI and Google's closed ecosystems. Llama 4 was supposed to extend that legacy. Instead, it became the symbol of everything the open-source community distrusts about corporate AI: the benchmarks are rigged, the transparency is selective, and the claims cannot be verified without independent testing.

What Does This Mean for the Future of Open-Weight Models?

Despite the Llama 4 scandal, the open-weight model ecosystem is thriving. In March 2026, Alibaba released a 9-billion-parameter language model that scored 81.7 on GPQA Diamond, a graduate-level reasoning benchmark, beating OpenAI's GPT-OSS-120B at one-thirteenth the size. The model, Qwen 3.5-9B, runs on a laptop with 16 gigabytes of RAM and is open-weight under Apache 2.0 license.

Six major labs now ship production-grade open-weight models: Alibaba with Qwen 3.6, Google with Gemma 4, Meta with Llama 4, Zhipu AI with GLM-5.1, DeepSeek with V4, and Mistral with Small 4. The competition is genuine. Each lab differentiates on a different dimension: Qwen on broad capability across sizes, Gemma on usability, GLM on benchmark performance, DeepSeek on reasoning, Mistral on efficiency at small scales.

The r/LocalLLaMA community's April 2026 "Best Local LLMs" megathread drew 143 posts and 440 interactions, producing a consensus ranking. The rankings are not based on vendor claims. They are based on community-run evaluations on real hardware. Gemma 4 leads for general usability. Qwen3-Coder-Next dominates local coding benchmarks. GLM-5 and GLM-4.7 perform strongly across reasoning, coding, and general knowledge. DeepSeek V4 brings chain-of-thought reasoning to local hardware. Qwen 3.5 small series represents the efficiency frontier.

The Llama 4 footnote remains significant: Llama 4 is a capable model with permanently damaged credibility. The benchmark scandal means every claim must be independently verified. The community includes it in rankings with the caveat: trust, but verify yourself.

LeCun's confession matters precisely because it came from inside the system. He was not an external critic with an agenda. He was the person responsible for Meta's AI research for more than a decade. And he still said the results were fudged. When the insiders stop believing the benchmarks, the rest of us should stop too.