Frontier AI models like GPT-5 and Gemini-3 aren't actually forgetting facts; they're struggling to retrieve information they already have encoded in their parameters. A new study by researchers at Google Research and Technion reveals that when large language models (LLMs) hallucinate or give wrong answers, the problem often isn't missing knowledge but rather a failure to access what's already stored. By allowing models to spend more computational effort at inference time, or test time, engineers can recover up to 65% of these inaccessible facts without scaling up model size or adding external databases.

Why Are AI Models Failing to Recall Facts They Actually Know?

When developers encounter factual errors from AI models, the standard assumption is that the model lacks the required information. The typical response involves scaling up the model, expanding training data, or building retrieval systems. But the Google Research and Technion study challenges this assumption by distinguishing between two separate problems: encoding and recall.

The researchers evaluated 13 different language models on over 4 million responses using WikiProfile, a benchmark containing 2,150 facts extracted from Wikipedia. Each fact was tested across multiple formats, from exact context completion to multiple-choice questions. The findings were striking: frontier models successfully encode 95-98% of tested facts. However, these same models fail to directly recall 26-34% of those encoded facts without additional thinking time.

"When facts come out wrong, the go-to move is to scale, meaning train a larger model or add more data. Both are expensive, and if the facts are already encoded, neither helps," said Nitay Calderon, Research Scientist at Google.

Nitay Calderon, Research Scientist at Google

This distinction matters enormously for how companies approach AI reliability. A model that encodes a fact but cannot recall it requires different solutions than a model that never learned the fact in the first place. Encoding failures call for pre-training interventions like scaling model size or expanding training data. Recall failures, by contrast, suggest post-training interventions that improve how models access what they already know.

How Does Test-Time Compute Help Recover Lost Knowledge?

The researchers identified five distinct knowledge profiles that explain how models store and retrieve information. Understanding these profiles reveals why inference-time computation, or test-time compute, acts as a powerful recovery mechanism.

• Direct Recall: The model encodes the fact and readily accesses it to answer direct questions without extra inference compute.
• Encoding Failure: The model neither encodes nor knows the fact, signaling a need for more pre-training data or greater model capacity.
• Recall Failure: The model has the fact encoded but cannot access it during direct generation, even though it can complete the original training text containing that fact.
• Recall with Thinking: The fact is encoded but inaccessible to direct generation; it is only successfully recalled when the model uses inference-time computation such as Chain-of-Thought reasoning.
• Inference Without Encoding: The model never explicitly encoded the fact but successfully answers the question by reasoning across other encoded facts it does know.

When models are given extra computational effort to think through a problem, they successfully retrieve 40-65% of the encoded facts they initially failed to recall directly. The researchers compare this to the human tip-of-the-tongue phenomenon, where deliberate mental effort, such as retracing context, eventually helps recover information.

Consider the example of Oasis and their first gig at the Boardwalk club. A model might fail to answer "Where did Oasis play their first show?" when asked directly. But by generating intermediate thoughts about the band's early history in Manchester, the model structurally primes itself to locate and recall the locked answer. This mechanism is what researchers call recall facilitation.

What Happens When You Scale Model Size Without Addressing Recall?

The study reveals a counterintuitive finding: scaling model size does not automatically solve recall failures. In fact, companies often make an expensive architectural mistake by trying to solve recall failures through fine-tuning larger internal models.

When researchers scaled the Gemma3 model from 1 billion to 27 billion parameters, they observed a shift in the problem rather than a solution. Scaling decreased encoding failures from 85% to 23%, successfully filling what researchers call the "empty shelves" problem. However, at the same time, the share of recall failures increased, peaking at 40% without thinking. This suggests that scaling mainly solves the storage problem rather than the access problem. As the model memorizes vastly more facts, a larger pool of knowledge becomes trapped in an "encoded but inaccessible" state.

The bulk of model errors shifts from missing data to failed recall as models grow larger. This pattern has significant implications for how companies should approach AI reliability and infrastructure investment.

How Should Teams Rethink Their Approach to AI Factuality?

The research suggests that recall is tightly coupled to the conditions under which facts were learned, degrading when queries diverge from training-time patterns. How a user asks a question directly dictates whether the model can unlock the stored answer. The study found a large recall gap between long-tail and highly popular facts that exceeds 25% for frontier models, even though rare facts are encoded at rates similar to popular facts.

Models also struggle with reverse questions, where they must generate the subject instead of the object. A model might easily answer that Oasis played their first gig at the Boardwalk club but fail to answer who played their first gig at that same club. Yet the same models show they know the correct answer when given the question in multiple-choice format. This reframes both phenomena as recall failures rather than missing knowledge.

Steps to Optimize AI Factuality Without Scaling Models

• Profile Knowledge at the Fact Level: Instead of scoring whether an LLM answers an isolated prompt correctly, test a single piece of information across multiple conditions, evaluating whether the fact is stored in the model's parameters and what computational effort is required to retrieve it.
• Prioritize Inference-Time Computation: Allocate resources to enabling models to think longer during inference rather than automatically scaling model size or expanding training data, which may not address the underlying recall problem.
• Avoid Blanket Retrieval Solutions: While Retrieval-Augmented Generation (RAG) is appropriate for fresh or internal data, using it as a universal fix for hallucinations adds latency and cost when the model already encodes the required information.
• Test Query Variations: Evaluate model performance across different phrasings and question formats to identify whether failures stem from encoding gaps or recall limitations specific to certain query patterns.

The high encoding rates of frontier models require a fundamental shift in how developers approach factuality and pipeline architecture. Rather than treating every factual failure as a retrieval problem requiring external databases or larger models, teams should first diagnose whether the model already knows the answer but simply cannot access it. For many cases, enabling test-time compute through techniques like extended reasoning offers a more cost-effective path to reliability than architectural overhauls.