科学一体框架:通过证据链实现可验证的自主研究框架

内容总结:
谷歌云发布“科学一号”框架:AI科研论文零幻觉、可验证
2026年7月30日,谷歌云 研究科学家芮梦与总监托马斯·普菲斯特联合宣布,推出名为“科学一号”的实验性研究原型系统。该系统基于全新的“证据链”框架设计,旨在从根本上消除AI生成科研论文中的“幻觉”问题(即编造不存在的引用或数据),同时发布了配套的“证据链审计”自动化协议,用于评估AI论文的完整性。
背景:AI科研系统遭遇“信任危机”
随着大语言模型从编程助手进化为能自主完成文献综述、提出假设、执行实验并撰写论文的“全流程科研智能体”(如Sakana的AI-Scientist、AutoResearchClaw等),其产出论文的表面质量虽已接近人类水平,但一个关键的结构性缺陷日益凸显:可验证性严重不足。当前自主科研系统迭代生成文本时,错误会逐级放大,导致部分系统出现杜撰参考文献、论文方法描述与实际代码不符、实验结果无法复现等“幻象”问题。
核心突破:“证据链”框架确保每项声明可追溯
研究团队提出的“证据链”概念框架,类比数据库领域的ACID事务可靠性标准,定义了值得信任的科研产出的必备属性:科研成果中的每项声明(包括参考文献、数据结果、方法描述、结论)都必须附带一条完整的、可回溯的证据链,且每条链必须真实支撑其所对应的声明。
基于该框架研发的“科学一号”系统,通过三大核心模块在生成论文时自动构建证据链:
- 问题调查员:以检索方式获取真实文献,而非让模型凭记忆生成。
- 协议实验员:记录实验完整日志,确保代码与结果一一对应。
- 制表研究员:自动生成可验证的分析图表。
审计结果:零幻象、全可验证,性能不妥协
团队在75篇论文的自动化设计研究系统基准测试中,采用“证据链审计”协议对四大指标进行严格核查,结果如下:
- 科学一号系统:参考文献零虚构,全部指向真实可检索的论文(基线系统幻觉率最高达21%);
- 得分完全可复现,方法描述与代码高度一致(基线系统常描述“混合神经符号求解器”算法,提交的代码却是简单的确定性启发式算法);
- 在MLE-Bench、Parameter-Golf等前沿基准测试中取得顶尖性能,并在Cloudcast和EPLB两项任务中达到或超越人类专家水平。
意义与展望
研究团队强调,随着自主科研系统处理的问题日益复杂,单纯比拼“解题能力”已不够,产出的可信度将成为核心区分项。通过在生成声明的同时建构证据链(而非事后补救),科学一号框架证明了AI智能体可以产出严谨、可信且极具竞争力的科学研究。团队希望证据链框架及审计协议能成为社区建设下一代AI科学家的基础工具。
注意: 科学一号框架目前仅为实验性研究原型,并非生产就绪的正式工具。
中文翻译:
2026年7月30日
谷歌云研究科学家芮萌与总监Tomas Pfister联合发表
推出实验性研究原型"科学一号框架",通过原生构建可验证的证据链消除幻觉;同时发布自动化审计协议"CoE审计",用于评估AI生成论文的完整性。
大语言模型正从代码助手演变为能独立完成端到端科研流程的自主智能体。近期系统(如Sakana的AI-Scientist、AutoResearchClaw、DeepScientist、AI-Researcher)已能完成文献综述、假设提出、实验执行及整篇论文撰写,质量可与人类论文媲美。但随着AI生成论文的表面质量提升,一个关键结构性问题浮现:可验证性。当前自主研究流水线通过迭代生成文本,任何阶段的错误都会被放大。现有系统常出现虚假引用、方法描述与实际代码不符、实验数据无法通过代码复现等问题。
我们提出"证据链"这一AI驱动研究的新型可验证框架。通过"科学一号框架"(原生构建维护证据链的自主研究原型)与"CoE审计"(衡量AI生成论文与代码及证据一致性的自动化评估指标集)实现该框架。实验表明,基线系统引用虚构率高达21%,且代码与文本频繁错位;而科学一号框架实现零虚假引用与完全可验证数据,在MLE-Bench、Parameter-Golf等前沿基准测试中达到顶尖水平。
证据链是界定研究制品可信度的概念框架,如同ACID定义数据库事务可靠性。该框架不规定如何构建研究智能体,而是设定输出必须满足的规范,遵循双向原则:研究制品中的每个声明必须附带记录完整的证据链(完整性),且每条证据链必须真实支撑其所属声明(正确性)。声明可以是参考文献、报告数据、方法描述或结论,必须对应到同行评审论文、实验日志、实际运行代码或结果表格等证据。
虚假引用指向不存在的论文,不可复现的数据在重新运行代码时无法复现,错误描述的方法宣称采用某算法而实际代码另用他法。每个声明都对应着断裂的证据链,CoE审计使这些断裂可量化测量。
为证明可验证AI研究能在不牺牲解题能力的前提下实现,我们设计了科学一号框架。区别于事后回溯链接事实的前代系统,该框架通过三大模块原生构建证据链。
为严格评估科学一号框架原型与顶尖基线系统(Sakana AI的AI Scientist v2、AutoResearchClaw、DeepScientist、AI-Researcher)的优劣,我们开发了CoE审计。这个事后评估协议如同自动化法医审查员,对生成制品(论文、方案、代码、参考文献)执行四项完整性校验:
我们将CoE审计应用于来自"研究系统自动化设计"基准的75篇论文,涵盖五项系统优化任务(Prism、Cloudcast、EPLB、LLM-SQL、事务调度)。
科学一号框架在可验证性上显著超越现有基线。CoE审计对每个系统执行统一独立协议,通过实时学术数据库复核每项引用。在该评估体系下,科学一号框架四项完整性指标全部领先:零虚假引用(每项引用均指向真实可查论文,而基线系统虚假率高达21%),问题调查员从数据库检索引用而非依赖记忆生成;实现满分数据验证与最高方法-代码对齐度。相比之下,基线系统常在提交的代码仅为简单确定性启发式算法时,声称使用了"混合神经符号求解器"等复杂算法。
关键的是,强制执行可验证性并未损害智能体的科研能力。科学一号框架在五项ADRS任务中均达到或超越人类专家水平,在Cloudcast与EPLB两项任务中获得系统总评分最高。
为测试泛化能力,我们将科学一号框架部署至六项高度复杂的外部任务:
当自主研究系统规模化解决日益复杂的科学问题时,求解器质量已不足以区分高下。真正决定产出价值的是结果是否可信。我们的发现表明,必须将可验证性作为一等架构约束。通过在声明生成时同步构建证据链,而非事后重建依据,科学一号框架证明AI智能体能产出严谨可信且极具竞争力的科学研究。希望证据链框架及其审计工具能为下一代AI科学家社区提供有益支持。
感谢Bhavana Dalvi Mishra、Jiefeng Chen、Chun-Liang Li、Palash Goyal、Mihir Parmar、Yiwen Song、Yale Song、Raj Sinha、Parthasarathy Ranganathan、Burak Gokturk、Jinsung Yoon对本研究的宝贵贡献。
科学一号框架为实验性研究原型,非生产级工具。
英文来源:
July 30, 2026
Rui Meng, Research Scientist, and Tomas Pfister, Director, Google Cloud
Introducing the Science One Framework, an experimental research prototype designed to eliminate hallucinations by natively building verifiable evidence chains, and CoE Audit, an automated protocol to evaluate the integrity of AI-generated papers.
Large language models (LLMs) are increasingly being deployed not just as coding assistants but as autonomous agents capable of conducting end-to-end scientific research workflows. Recent systems (e.g., Sakana’s AI-Scientist, AutoResearchClaw, DeepScientist, AI-Researcher) can review literature, formulate hypotheses, execute experiments and write complete manuscripts that are comparable to human-authored papers. However, as the surface-level quality of these AI-generated manuscripts improves, a critical structural problem has emerged: verifiability. Because current autonomous research pipelines generate text iteratively, errors introduced at any stage are amplified. Some existing systems can generate non-existent citations, exhibit misalignments between the described methods and the actual code, and report experimental scores that are not fully reproducible from the provided code.
In our paper, we tackle this problem by introducing Chain-of-Evidence (CoE), a new verifiability framework for AI-driven research. We instantiate CoE with the Science One Framework, an autonomous research prototype that natively builds and maintains evidence chains, and the CoE Audit, a set of automated evaluation metrics that measures the integrity of AI-generated papers against their underlying code and evidence. Our results show that baseline systems hallucinate up to 21% of their references and frequently misalign their code and text, whereas the Science One Framework achieves zero phantom references and fully verifiable scores while achieving state-of-the-art performance on frontier benchmarks like MLE-Bench and Parameter-Golf.
The CoE is a conceptual framework that defines what makes a research artifact trustworthy, much as ACID defines what makes a database transaction reliable. Rather than prescribing how to build a research agent, the framework specifies the properties its outputs must have. It follows a single principle with two halves: every claim in a research artifact must carry a recorded evidence chain (completeness), and each chain must genuinely support the claim it is attached to (correctness). A claim may be a reference, a reported number, a method description, or a conclusion, that must link back to corresponding evidence, such as a peer-reviewed paper, an experimental log line, the code that actually ran, or the results table.
A hallucinated reference points to a paper that does not exist. An unreproducible score does not reappear when the code is re-run. A misdescribed method claims one algorithm in the paper while the code implements another. Each is a claim whose chain back to its evidence is broken; the CoE Audit makes these breaks measurable.
To demonstrate that verifiable AI research is possible without sacrificing problem-solving performance, we designed the Science One Framework. Unlike previous agents that generate a paper and try to link facts retroactively, the Science One Framework instantiates the CoE framework by construction through three main modules:
To rigorously evaluate the Science One Framework prototype against state-of-the-art baselines (e.g., Sakana AI's AI Scientist v2, AutoResearchClaw, DeepScientist, AI-Researcher), we developed the CoE Audit. This post-hoc evaluation protocol acts as an automated forensic reviewer, running four strict integrity checks on the generated artifacts (paper, solution, code and references):
We applied the CoE Audit to 75 papers generated across five systems-optimization tasks (Prism, Cloudcast, EPLB, LLM-SQL, and transaction scheduling) from the Automated Design of Research Systems (ADRS) benchmark.
The Science One Framework significantly outperformed existing baselines in verifiability. The CoE Audit applies the same independent protocol to every system, re-checking each reference against live scholarly databases, and under it, Science One Framework led on all four integrity checks. None of its references were phantom: every one pointed to a real, retrievable paper, compared to hallucination rates as high as 21% in baselines, because the Problem Investigator retrieves every reference rather than generating it from memory. It also achieved perfect score verification and the highest method-code alignment. In contrast, baseline systems frequently described sophisticated algorithms (like "hybrid neuro-symbolic solvers") when their submitted code was a simple, deterministic heuristic.
Crucially, implementing strict verifiability did not compromise the agent's scientific capabilities. The Science One Framework matched or exceeded human expert performance on all five ADRS tasks, achieving the best overall score among all systems on two of them (Cloudcast and EPLB).
To test its generalization, we deployed the Science One Framework on six highly complex external tasks:
As autonomous research systems scale to solve increasingly difficult scientific problems, solver quality alone will no longer be enough to differentiate them. What will separate their outputs is whether the resulting research can be trusted. Our findings demonstrate that verifiability must be treated as a first-class architectural constraint. By building evidence chains at the time a claim is produced rather than attempting to reconstruct grounding after the fact, The Science One Framework demonstrates that AI agents can produce rigorous, trustworthy and highly competitive scientific research. We hope the Chain-of-Evidence framework and its audit will serve as valuable tools for the community as we continue to build the next generation of AI scientists.
We would like to thank Bhavana Dalvi Mishra, Jiefeng Chen, Chun-Liang Li, Palash Goyal, Mihir Parmar, Yiwen Song, Yale Song, Raj Sinha, Parthasarathy Ranganathan, Burak Gokturk and Jinsung Yoon for their valuable contributions to this work.
The Science One Framework is an experimental research prototype, not production-ready tools.
文章标题:科学一体框架:通过证据链实现可验证的自主研究框架
文章链接:https://news.qimuai.cn/?post=4680
本站文章均为原创,未经授权请勿用于任何商业用途