内容来源：https://research.google/blog/empty-shelves-or-lost-keys-recall-is-the-bottleneck-for-parametric-factuality/

内容总结：

谷歌研究揭示：前沿大模型“知道但想不起”成事实错误主因

科技日报讯（记者 张梦然）大型语言模型（LLM）给出错误答案，究竟是因为它从未学过这个知识，还是因为知识已存储却无法调取？谷歌研究团队日前发布的一项新研究给出了明确答案：对于前沿大模型而言，绝大多数事实性错误源于“回忆失败”，而非“编码缺失”。这就像家中钥匙其实在抽屉里，但一时找不到，而不是根本没有这把钥匙。

该研究发表于2026年8月12日。研究团队引入了一种名为“知识画像”（Knowledge Profiling）的行为分析框架，将事实性评估从“逐题判断对错”升级为“对知识状态进行分类”，从而精确区分两种本质不同的失败模式：编码失败（模型参数中从未存储该事实）与回忆失败（事实已存在于参数中，但无法在需要时被提取出来）。

为支撑这一分析，团队构建了名为“WikiProfile”的基准测试集，包含2150条从维基百科提取的事实，每条事实配有多达10个不同角度的问题，以分别探测编码、回忆和识别能力。研究人员对13款主流大模型（包括Gemini 3系列、GPT-5等）进行了约450万次响应测试。

核心发现：前沿模型“饱读诗书”，但“提笔忘字”

结果显示，在Gemini-3-Pro和GPT-5等前沿模型中，95%至98%的事实已被成功编码进参数，但直接回忆失败率却高达26%至34%。即便启用“思维链”等深度思考模式，仍有11%至12%的事实无法被正确调取。这意味着，当前大模型的事实性瓶颈已从“知识储备不足”转向“知识利用低效”——模型并非“无知”，而是“暂不可得”。

规模越大，回忆短板越突出

研究还发现，模型规模的扩大显著改善了编码能力（“记得住”），但对回忆能力的提升相对有限。在Gemma 3系列中，更大规模的模型编码失败率大幅下降，但回忆失败率依然显著，并在剩余错误中占据更大比例。这表明，单纯扩大参数规模或数据量，已难以解决“想不起来”的核心问题。

长尾事实与“逆转诅咒”的重新解读

研究对两类常见现象提出了新见解。一是长尾（低频）事实：研究表明，冷门事实的编码率与热门事实相差无几，但回忆率差距显著。这说明“长尾问题”的本质并非模型容量不够，而是存储后的调取通道不够畅通。二是“逆转诅咒”（即模型知道“A是B”，却答不出“什么是B”）：在开放式提问中，反向问题确实更难；但在选择题（识别任务）中，反向问题并不更难。这一分离现象表明，双向知识并非缺失，而是训练时形成的单向检索路径限制了反向回忆能力。

思考模式成“知识解锁”利器

研究进一步发现，启用“深度思考”模式能有效地帮助模型调取已编码但无法直接回忆的知识，尤其在低频事实和反向提问上效果显著，可恢复约40%至65%的“存而难取”知识。但对于真正未编码的事实，思考模式几乎无能为力，这印证了其作用机制是“回忆促进”而非“凭空推理”。

研究团队指出，该发现重新定义了提升大模型事实准确性的路径：在编码能力接近饱和的当下，未来改进重点应转向优化知识调取机制，例如开发更高效的“回忆触发”策略或动态判断何时启用深度思考，而非一味堆叠参数和数据。这一“知识画像”方法为行业提供了一套更精细的诊断工具，有望指导下一代大模型在真实性提升上“有的放矢”。

中文翻译：

2026年8月12日
Nitay Calderon 和 Gal Yona，谷歌研究院研究科学家
当大语言模型（LLM）答错事实时，是因为它们从未学过这些事实，还是因为它们无法回忆起已经编码的信息？我们的知识画像框架揭示了后者：前沿LLM几乎编码了所有事实，但其中许多事实却难以被回忆起来。
事实准确性对于让大语言模型（LLM）变得可靠至关重要。当模型错误回答一个事实性问题时，是因为该事实从未被编码，还是因为该事实已被编码但无法访问？标准准确率指标将这两种情况混为一谈，尽管它们暗示着截然不同的局限性和截然不同的干预措施。编码失败需要扩大模型规模或扩展数据覆盖范围，而回忆失败则可能指向训练后方法和推理时方法，帮助LLM更好地利用它们已经编码的信息。
在《空书架还是丢失的钥匙？回忆是参数化事实性的瓶颈》一文中，我们引入了知识画像，这是一个同时衡量编码和回忆的行为框架，并以此考察前沿LLM（如Gemini 3和GPT-5）事实性的潜在瓶颈。随后我们表明，前沿LLM中的许多事实错误更适合被理解为“丢失的钥匙”（回忆失败），而非“空书架”（编码失败）。
打个比方，我们用“编码”来表示事实的参数化表征，“回忆”表示在没有外部线索的情况下检索已编码的事实，“识别”表示在备选答案中辨认出正确事实。为支持这一分析，我们引入了WikiProfile，这是一个包含2，150个源自维基百科的事实的基准测试集，每个事实配对了十个问题，分别探测编码、回忆和识别能力。
知识画像将分析单位从单个问题转向事实本身。我们不问模型是否答对了某个特定问题，而是问一个更广泛的问题：该事实处于什么状态？我们将每个事实归入五种知识画像之一：（1）编码失败，（2）回忆失败，（3）直接回忆，（4）思考后回忆，（5）无编码推断。这些画像提供了比单纯问题级准确率更具信息量的诊断。
分类基于该事实是否被编码及其可访问程度：无法回忆、可以直接回忆、或仅能通过思考回忆（在最终答案之前引出中间计算，包括思维链提示和针对思考优化的LLM）。
我们用三个行为概念来具体实现这一框架：
为了实现知识画像的操作化，我们构建了WikiProfile，这是一个专门用于衡量自然发生事实的事实性基准测试集。WikiProfile通过一个完全自动化的流水线构建，由经过提示的LLM——Gemini-2.5-Pro（带思考能力）驱动。提示词通过在小型预留子集上手动优化开发而成。我们从维基百科页面中提取候选事实，方式为识别事实：涉及有序实体对（主语和宾语）的命题，其中主语在文档中首先出现。每个事实配对了10个任务：两个用于编码，四个用于知识评估，四个多项选择变体用于识别。
我们通过生成、精炼和过滤的三步流程来生成正向和反向问题，确保每个问题都无歧义、具体、简洁且具有唯一答案。所有问题都经过基于搜索引擎的过滤。我们丢弃返回多个答案或需要澄清的情况。经过自动过滤和最终人工验证步骤后，该基准包含2，150个事实。
我们评估了13个LLM。每个模型在有无思考两种情况下均被评估。对于每个模型、事实和任务，我们采样八个回答。回答由提示驱动的LLM自动评分器自动评分（更多细节见论文），总计产生约450万个回答。
在前沿LLM（Gemini-2.5-Pro、Gemini-3-Pro和Flash、GPT-5）中，事实编码接近饱和，但回忆并未饱和。对于Gemini-3-Pro和GPT-5，95%–98%的事实已被编码，但这些模型仍无法直接回忆26%–34%的事实。即使经过思考，它们仍然在11%–12%的事实上失败。这意味着在前沿模型中，事实错误越来越多地并非来自知识的缺失，而是来自已存储但无法可靠访问的知识。换言之，瓶颈正在从知识获取转向知识利用。
规模扩展强化了这一图景。在Gemma 3系列中，更大的模型展现出远更少的编码失败，但回忆失败仍然显著，并在剩余错误中占据更大份额。规模扩展对模型存储内容的改善程度大于对模型可访问内容的改善程度。
我们的结果表明，回忆与事实被学习时的条件紧密耦合。当查询偏离训练时的上下文、措辞或事实出现的顺序时，回忆变得更加困难。我们重点指出两种系统性出现这种情况的情形。
先前的研究表明LLM在长尾（罕见）事实上存在困难，通常将其框定为模型容量问题。我们的结果提供了一个互补的视角。当我们比较低流行度和高流行度的事实时，发现罕见事实的编码率接近热门事实。编码差距相对较小；然而，回忆差距更大。这重新框定了长尾问题：许多罕见事实并非不存在于模型参数中。它们存在，但难以访问。瓶颈已从知识获取转向知识利用。
我们还重新审视了“反转诅咒”：当LLM知道“A是B”却无法回答“什么是B？”时。乍一看，这可能表明LLM缺乏双向知识。但我们的结果表明需要对这一观点进行细化。在开放式生成（即回忆）中，反向问题始终比正向问题更难。然而，在多项选择验证（即识别）中，反向问题并不比正向问题更难，而且往往更容易。这种分离很重要。如果一个模型能在干扰项中识别出正确答案，却无法在反向查询中生成它，那么问题就不只是双向知识的缺失。相反，该事实似乎已被编码，甚至可被识别，但当查询方向偏离训练中事实出现的方式时则难以回忆。反转诅咒是一个回忆问题。
现在我们转向什么问题能够恢复原本无法访问的知识。为此，我们考察思考在扮演这一角色方面的潜力。思考在直接回忆最弱的地方对回忆的提升最为显著。这些收益在罕见事实和反向问题上尤其明显，同时缩小了流行度差距和方向性差距。
更具体地说，在针对思考优化的模型中，思考能够恢复大约40%–65%的已编码但无法直接知晓的事实。相比之下，它对未编码的事实的帮助要小得多。这种模式表明，思考主要作为一种回忆促进机制：它帮助模型访问已经编码的事实，而非主要通过复杂的多步推理来推导答案。话虽如此，思考并非没有代价。它带来计算成本，且目前仍不清楚如何精确确定模型何时应该调用思考。
知识画像使我们能够精确诊断LLM中的事实行为。将这一方法应用于维基百科事实，我们的结果表明我们应该重新思考前沿LLM中事实错误的方式。如果编码已接近饱和，那么事实性的进一步提升可能更少来自规模扩展（模型规模或数据）。通过表明思考可以恢复已编码但无法直接知晓事实的相当大一部分，事实性的下一轮改进可能不仅来自更好的知识获取，还来自对模型中已编码知识的更好利用。

英文来源：

August 12, 2026
Nitay Calderon and Gal Yona, Research Scientists, Google Research
When LLMs get facts wrong, is it because they never learned them or because they can't recall what they’ve already encoded? Our knowledge profiling framework reveals the latter: frontier LLMs encode nearly all facts, yet struggle to recall many of them.
Factuality is essential for making Large Language Models (LLMs) reliable. When a model answers a factual question incorrectly, is it because the fact was never encoded, or because the fact is encoded but not accessible? Standard accuracy metrics collapse these cases together, even though they suggest very different limitations and very different interventions. Encoding failures call for scaling model size or expanding data coverage, while recall failures might also point to post-training and inference-time methods that help LLMs better utilize what they already encode.
In “Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality”, we introduce knowledge profiling, a behavioral framework that measures both encoding and recall, and use it to examine the underlying bottlenecks of factuality in frontier LLMs (such as Gemini3 and GPT-5). We then show that many factual errors in frontier LLMs are better understood as lost keys (recall failures), not empty shelves (encoding failures).
By analogy, we use encoding to denote parametric representation of facts, recall to denote retrieving encoded facts without external cues, and recognition to denote identifying the correct fact when it is presented among alternatives. To support this analysis, we introduce WikiProfile, a benchmark of 2,150 Wikipedia-derived facts, each paired with ten questions that probe encoding, recall, and recognition.
Knowledge profiling shifts the unit of analysis from individual questions to facts. Instead of asking whether a model answered a specific question correctly, we ask a broader question: what is the state of the fact? We classify each fact into one of five knowledge profiles: (1) encoding failure, (2) recall failure, (3) direct recall, (4) recall with thinking, and (5) inference without encoding. These profiles provide a more informative diagnosis than question-level accuracy alone.
The classification is based on whether the fact is encoded and how accessible it is: Cannot be recalled, can be directly recalled, or can be recalled only with thinking (eliciting intermediate computations before the final answer, including chain-of-thought prompting and thinking-optimized LLMs).
We operationalize this with three behavioral notions:
To operationalize knowledge profiling, we constructed WikiProfile, a benchmark designed to measure factuality on naturally occurring facts. WikiProfile is constructed using a fully automated pipeline powered by a prompted LLM, Gemini-2.5-Pro with thinking. Prompts were developed through manual optimization on a small held-out subset. We extract candidate facts from Wikipedia pages by identifying facts: a proposition involving an ordered pair of entities (subject and object), where the subject appears first in the document. Each fact is paired with 10 tasks: two for encoding, four for knowledge evaluation, and four multiple-choice variants for recognition.
We generate direct and reverse questions through a three-step process of generation, refinement, and filtering, ensuring that each question is unambiguous, specific, minimal, and has a unique answer. All questions undergo filtering grounded in a search engine. We discard cases where multiple answers are returned or clarification is needed. After this automated filtering and a final manual validation step, the benchmark contains 2,150 facts.
We evaluate 13 LLMs. Each model is evaluated both with and without thinking. For each model, fact, and task, we sample eight responses. Responses are graded automatically by prompted LLM autoraters (more details in the paper), producing approximately 4.5 million responses.
Across the frontier LLMs (Gemini-2.5-Pro, Gemini-3-Pro and Flash, GPT-5), factual encoding is close to saturation, but recall is not. For Gemini-3-Pro and GPT-5, 95–98% of facts are encoded, yet these models still fail to directly recall 26–34% of facts. Even with thinking, they still fail on 11–12% of facts. This means that in frontier models, factual errors increasingly come not from absent knowledge, but from knowledge that is stored and not reliably accessible. In other words, the bottleneck is shifting from knowledge acquisition to knowledge utilization.
Scaling reinforces this picture. In the Gemma 3 family, larger models show far fewer encoding failures, but recall failures remain substantial and become a larger share of the remaining errors. Scaling improves what the model stores more than it improves what the model can access.
Our results suggest that recall is tightly coupled to the conditions under which a fact was learned. When the query diverges from the training-time context, phrasing, or ordering in which the fact was encountered, recall becomes harder. We highlight two cases where this happens systematically.
Prior work has shown that LLMs struggle with long-tail (rare) facts, often framing this as a problem of model capacity. Our results suggest a complementary picture. When we compare low-popularity and high-popularity facts, we find that rare facts are encoded at rates close to popular facts. The gap in encoding is relatively modest; however, the gap in recall is larger. This reframes the long-tail problem: Many rare facts are not absent from the model's parameters. They are present, but difficult to access. The bottleneck has shifted from knowledge acquisition to utilization.
We also revisit the reversal curse: when LLMs know "A is B" but can't answer "What is B?". At first glance, this could suggest that LLMs lack bidirectional knowledge. But our results suggest a refinement of this view. In open-ended generation (i.e., recall), reverse questions are consistently harder than direct questions. In multiple-choice verification (i.e., recognition), however, reverse questions are no harder than direct ones, and are often easier. This dissociation matters. If a model can recognize the correct answer when it is presented among distractors, but cannot generate it in a reverse query, then the issue is not simply that the bidirectional knowledge is missing. Rather, the fact appears to be encoded, and even recognizable, but difficult to recall when the query direction departs from how the fact was encountered during training. The reversal curse is a recall problem.
We now turn to the question of what enables the recovery of otherwise inaccessible knowledge. To this end, we examine the potential of thinking to fill this role. Thinking improves recall most strongly exactly where direct recall is weakest. The gains are especially pronounced for rare facts and reverse questions, narrowing both the popularity gap and the directionality gap.
More specifically, in thinking-optimized models, thinking recovers roughly 40–65% of encoded-but-not-directly-known facts. By contrast, it helps much less on facts that are not encoded. This pattern suggests that thinking primarily acts as a recall-facilitation mechanism: it helps the model access facts it already encoded, rather than mainly deriving answers through complex multi-step reasoning. That said, thinking is not free. It carries a computational cost, and it remains unclear how to determine exactly when a model should invoke it.
Knowledge profiling enables us to precisely diagnose factual behavior in LLMs. Applying this methodology to Wikipedia facts, our results suggest a shift in the way we should think about factual errors in frontier LLMs. If encoding is already near saturation, then further gains in factuality may come less from scaling (of model size or data). By showing that thinking can recover a substantial fraction of encoded-but-not-directly-known facts, the next improvements in factuality may come not only from better knowledge acquisition, but from better utilization of knowledge already encoded in the model.