Frontier large language models encode 95–98% of tested facts but still miss 26–34% of them during direct recall, according to a new study from Google Research and Technion. The gap stems not from missing knowledge but from retrieval failures, where models can’t surface stored facts without additional inference-time computation.
The researchers propose fact-level profiling to distinguish between whether a fact is stored and whether it can be reliably accessed. Their experiments show that inference steps like Chain-of-Thought recover 40–65% of encoded facts that models initially fail to recall. Scaling model size doesn’t fix this issue—in fact, larger models often trap more facts in an "encoded but inaccessible" state, shifting errors from missing data to failed recall.
Developers should avoid blanket fixes like RAG for parametric knowledge and instead use targeted inference-time reasoning, generate-then-verify pipelines, and query reformulation to improve factuality. The team also released the WikiProfile benchmark to help teams profile their own models’ access to stored facts.
