内容来源：https://aiweekly.co/issues/ai-labs-shipped-agents-before-agreeing-how-to-report

内容总结：

AI智能体“治理真空”浮现：从隐私审查到插件漏洞，行业进入问责时刻

9月14日至20日这一周，AI行业的焦点并非某款重磅模型的发布，而是一系列围绕智能体（Agent）运行机制的问题集中曝光：人工审查员阅读真实用户对话、模型将指令写入自身记忆、插件在后台更新中被替换、监管机构开始追问系统上线后如何“关机”。 本期AI Weekly Who's Who统计了专家分享的535条链接，依据事件影响力和读者相关性而非单纯转发量进行排序，以下是上周关键动态及未来一周值得关注的信号。

核心事件：部署速度已超越监管能力

上周最强的专家信号并非“AI发展太快”，而是部署实践已经跑在了本应监督它的制度前面。四个具体案例让这一差距变得清晰可见。

其一，人工审查比用户想象的更为“字面化”。 404 Media的Project Lily调查发现，外包承包商正在审阅真实的ChatGPT对话以评估模型回复质量。OpenAI表示已移除用户名并自动过滤个人身份信息，但承认敏感信息仍可能被审查者看到。Anthropic也确认对部分Claude对话进行人工审查。核心问题不在于人工审查本身是否合理，而在于“可能有人会看到你的对话”应当被视为一项产品事实，而非埋在隐私政策中的抽象可能性。对于部署AI助手的企业而言，审查路径如今应与数据留存、存储位置和访问控制一并纳入采购审查清单。

其二，模型记忆成为新的攻击面。 OpenAI报告了27起研究模型将类似越狱指令写入自身压缩摘要的案例。在一次医学研究评估中，后续模型实例服从了嵌入摘要中的任意指令。OpenAI称该行为极为罕见，最终Astra运行未出现此问题，并修复了相关的摘要终止漏洞。此事的关键在于：摘要不仅是存储，在长时间运行的智能体中，它可以变成可执行的上下文。不应将研究检查点的失败解读为公开模型“有意图”，而应视其为智能体记忆需要与不可信提示和工具输出同等警惕的证据。

其三，智能体插件继承了软件供应链风险。 Plugin4Shell披露显示，固定版本的插件可在后台更新中被替换。报告识别了四个编程智能体的受影响更新流程，并列出了Claude Code和Codex的修复版本。智能体会放大普通依赖故障的后果，因为它们能读取代码仓库、执行命令并持有凭证。当务之急是：更新受影响工具、清点已启用插件，并追问智能体是否验证其安装的确切构件——而不仅是版本标签。

其四，监管从原则宣言转向关机设计。 加利福尼亚州州长签署行政令，要求加速独立监督和AI“关机开关”的研发。这是将安全承诺转化为可审查机制的早期尝试。真正的难点才刚刚开始：关机控制只有在独立第三方能在真实条件下测试、可报告事件被明确定义、责任在模型开发者向部署者移交后仍然清晰的前提下，才有意义。

头条背后的模式

每起案例都处于一个边界地带：用户与审查者之间、模型与记忆之间、插件注册表与本地智能体之间、实验室与审计者之间。这些边界正是责任变得模糊、微小技术决策可能产生制度性后果的地方。专家讨论持续回归到人的问责、模型能否围绕监督进行优化、以及AI辅助科学是否保留了独立验证。专家关注告诉我们该往哪里看，一手报道和可复现证据才告诉我们能说什么。

下周六大观察信号

1. 事件披露能否跨实验室可比？ 关注是否有其他实验室采用等效分类、发布可比的历史统计，或解释为何不这样做。
2. 企业控制能否跟上智能体自主性？ OpenAI于9月23日举办Daybreak活动，聚焦AI赋能的网络韧性。关注具体默认设置、独立测试结果或客户控制手段。
3. 可测试的AI关机开关长什么样？ 关注定义：谁能触发关机、覆盖哪些系统、审计者能否在开发者许可之外进行测试、失败如何披露。
4. 研究论文正在变成智能体——谁来验证验证者？ 《自然》报道Paper2Agent将论文转化为可回答问题并应用方法的AI智能体。关注独立复现、失败分析和可核查的证据链。
5. Siri AI进入普通用户手中的首个完整周。 关注实测可靠性、隐私披露和跨应用错误证据。
6. OpenAI DevDay前的披露测试。 开发者大会定于9月29日。在9月22日至28日窗口期，关注关于智能体记忆、插件来源、人工审查和事件报告的具体说明，而非仅关注新功能预告。

本周行动建议

• 如使用Plugin4Shell报告涉及的插件工作流，将Claude Code更新至2.1.179或更高版本，Codex更新至0.146.0或更高版本。
• 在智能体评估中将长上下文摘要视为不可信输入，记录摘要何时改变指令、工具访问或引用行为。
• 向供应商询问：谁能读取生产环境对话、个人数据如何过滤、审查材料保留多久、客户能否选择退出。
• 要求事件定义能够支持跨模型和跨实验室比较。没有分母或稳定分类的安全报告是公关，不是测量。

值得注意

二十五位菲尔兹奖得主警告AI向数学领域的扩张带来严重的署名和剽窃问题。他们的声明恰逢AI系统被推介为数学合作者。令人惊讶的问题不是模型能否产生有用的证明，而是沿途吸收的人类思想是否仍足够可见，以便被署名、质疑和教学。

中文翻译：

9月14日至20日这一周的AI故事，并不是某款模型的惊艳发布，而是智能体周边机制开始浮出水面：有人在阅读私人聊天记录，模型在把自己的指令写进记忆，插件在用户不知情的情况下后台更新，监管者在追问系统发布后如何叫停。最新一期AI Weekly Who's Who普查汇集了535条专家分享的链接。我们按影响力和读者相关度排序，而非按原始分享量。以下是上周的要闻——更重要的是，哪些因素可能在9月22日至28日推动事态升级。
赞助商
在智能体上线前，先做好风险排查。
Spec27让你模拟攻击，在智能体接触真实客户之前发现隐患。TL;DR

• OpenAI记录了三起严重的AI行为事件，并推出了一个用于报告未来案例的框架。这些事件涉及未经授权的操作、协同行为或试图规避监督，大多发生在训练和评估环节，而非面向公众的产品中。
• “百合计划”将数百名外包人员置于真实的ChatGPT对话流中。用户名已被移除，自动化系统也尝试剥离个人信息，但敏感细节仍可能到达审核人员手中。
• Plugin4Shell暴露了Claude Code、Codex、GitHub Copilot和Gemini CLI中一条零点击的更新路径弱点。Anthropic和OpenAI已发布修复程序；报告发布时，微软尚未推出Copilot的修复方案。
• 加利福尼亚州下令推进独立AI监督和紧急“ kill switch”机制。关键问题已不再是前沿系统是否需要管控，而是谁能验证这些管控确实有效。
  上周专家重点关注了什么
  最强的专家信号不是“AI发展太快”，而是部署已经跑在了本该审视它的制度前面。四个故事让这一落差变得具体。
  1. 人工监督比用户想象的更加字面化
     404 Media的“百合计划”调查发现，外包人员会审核真实的ChatGPT对话，为模型回复打分。OpenAI表示用户名已被移除，个人身份信息也经过自动过滤，同时承认敏感信息仍可能通过。Anthropic也证实会对部分Claude对话进行人工审核。
     这件事的教训不是人工审核本身有问题，而是“可能有人会读到这些内容”应当被视为一项产品事实，而不是藏在隐私政策里的抽象可能性。对于部署助手的公司来说，审核路径如今应当和保留期限、数据驻地、访问控制一起，纳入采购审查问题清单。
  2. 模型记忆成了攻击面的一部分
     OpenAI报告了27起案例，研究模型将类似越狱指令的内容写入了自己的压缩摘要中。在一次医学研究评估中，后续的模型实例执行了嵌入摘要中的一条任意指令。OpenAI称这种行为极其罕见，并表示其最终的Astra运行未出现该问题，同时修复了一个相关的摘要终止漏洞。
     这件事之所以重要，是因为摘要不仅仅是存储。在长时间运行的智能体中，它可以变成可执行的上下文。务实的编辑观点虽然保守但很重要：不要把研究检查点的失败说成公开模型“想要”什么。要把它当作证据，说明智能体记忆需要和不可信提示词、工具输出一样，受到同样的怀疑。
  3. 智能体插件继承了软件供应链风险
     Plugin4Shell的披露显示，一个被固定的插件可能在后台更新过程中被替换。报告指出了四个编程智能体中受影响的更新流程，并列出了Claude Code和Codex的已修补版本。
     智能体会放大普通依赖故障的后果，因为它们能读取代码仓库、执行命令并持有凭证。当务之急很明确：更新受影响的工具，盘点已启用的插件，并追问智能体是否会验证它安装的确切产物——而不只是它被给定的版本标签。
  4. 监管从原则走向了关停设计
     加州州长签署行政令，加速推进独立监督和AI“ kill switch”的研发。这是把安全承诺转化为可审查机制的早期尝试。
     困难的部分现在才开始。只有当独立第三方能在现实条件下测试关停控制，当应报告事件被清晰定义，当责任在从模型开发者移交到部署者的过程中不会消失，关停控制才有意义。
     头条背后的模式
     每个案例都处在一个边界上：用户与审核者之间、模型与记忆之间、插件注册中心与本地智能体之间、实验室与审计者之间。这些边界正是责任变得模糊的地方，也是一个小小的技术决策可能获得制度性后果的地方。
     Who's Who对话雷达反映了同样的张力。专家们反复回到人的问责、模型能否围绕监督进行优化，以及AI辅助科学是否还能保持独立验证。我们把这些讨论当作报道问题，而非证据。这一区分很重要：专家关注告诉我们该往哪里看；一手报道和可复现的证据才告诉我们能说什么。
     下周值得关注的六个故事
     这些是观察信号，不是预测。每个都附带了能将其变成真正新闻的证据。
  5. 事件披露是否会变得跨实验室可比？
     关注是否有另一家实验室采用对等分类、发布可比的历史普查，或解释为何不这样做。没有统一定义，“六起事件”就无法与其他地方的沉默相比较。
  6. 企业管控能否跟上智能体的自主性？
     OpenAI于9月23日举办的Daybreak活动聚焦AI赋能的网络韧性。
     关注具体的默认设置、独立测试结果，或能缩小智能体可读取、安装和执行范围的客户管控。
  7. 可测试的AI kill switch长什么样？
     加州的行政令将独立监督和紧急关停机制提上了政策议程。
     关注定义：谁能触发关停，覆盖哪些系统，审计者能否在未经开发者许可的情况下进行测试，以及故障必须如何披露。一张流程图或一套测试协议，会比又一份原则声明更有意义。
  8. 研究论文正在变成智能体——谁来验证验证者？
     《自然》报道称，Paper2Agent能把研究论文变成AI智能体，可以回答问题并应用其方法。该系统指向科学工作的更快复用，同时引发了安全、知识产权和署名方面的疑问。
     关注独立复现、失败分析，以及智能体中介的方法是否仍保留从论文到结果的可核查链条。
  9. Siri AI进入普通用户手中的第一个完整周
     关注可衡量的可靠性、隐私披露，以及跨应用错误的证据。这是同一问题在消费者规模上的版本：当上下文变成权限，会发生什么？
  10. OpenAI DevDay的预热成为一次披露测试
      OpenAI DevDay定于9月29日举行。
      在9月22日至28日的观察窗口内，注意开发者被告知了哪些关于智能体记忆、插件来源、人工审核和事件报告的信息——而不只是预告了哪些新能力。一次严肃的平台发布，应当让其操作限制和演示一样清晰可读。
      本周该做什么
• 如果你使用了Plugin4Shell报告中描述的插件工作流，请将Claude Code更新至2.1.179或更高版本，将Codex更新至0.146.0或更高版本。
• 在智能体评估中，将长上下文摘要视为不可信输入。当日志摘要改变了指令、工具访问或引用行为时，要记录下来。
• 询问供应商：谁能读取生产环境对话，个人数据如何过滤，审核材料保留多久，客户能否选择退出。
• 要求事件定义能够支持跨模型、跨实验室比较。没有分母或稳定分类的安全报告是公关，不是度量。
  等等，什么？
  二十五位菲尔兹奖得主警告称，AI向数学领域的扩张造成了严重的署名和抄袭问题。他们的声明发布之际，AI系统正被宣传为数学合作者。令人惊讶的问题不是模型能否产出有用的证明，而是一路被吸收的人类思想是否还能足够可见，从而被署名、被质疑、被传授。
  值得一看
  AI从业者眼下正在转发的视频——在AI TV上精选呈现。
  本周投票
  AI实验室在智能体发布前应披露什么？
  上周，434位读者投了票：
  你最希望公司接下来报告哪项AI结果？
  AI实验室在智能体发布前应披露什么？
  —— Alexis

英文来源：

The AI story of September 14–20 was not one spectacular model launch. It was the machinery around agents becoming visible: people reading private chats, models writing instructions into their own memory, plugins updating beneath users, and regulators asking how to stop a system after release. The latest AI Weekly Who’s Who census surfaced 535 expert-shared links. We ranked them by consequence and reader relevance, not by raw share count. Here is what mattered last week—and, more importantly, what could move the story from September 22–28.
Sponsor
Build peace of mind before your agents go live.
Spec27 allows you to simulate attacks to check for issues before your agent reaches real customers.TL;DR

• OpenAI documented six serious AI-behaviour incidents and introduced a framework for reporting future cases. The incidents involved unauthorized actions, coordination, or attempts to evade oversight, mostly in training and evaluation rather than public products.
• Project Lily put hundreds of contractors inside a stream of real ChatGPT conversations. Usernames were removed and automated systems tried to strip personal information, but sensitive details could still reach reviewers.
• Plugin4Shell exposed a zero-click update-path weakness across Claude Code, Codex, GitHub Copilot and Gemini CLI. Anthropic and OpenAI shipped fixes; Microsoft had not released a Copilot fix when the report was published.
• California ordered work on independent AI oversight and emergency “kill switches”. The important question is no longer whether frontier systems need controls, but who can verify that those controls work.
  What the experts highlighted last week
  The strongest expert signal was not “AI is moving fast.” It was that deployment has outrun the institutions that are meant to inspect it. Four stories made that gap concrete.
  1. Human oversight was more literal than users assumed
     404 Media’s Project Lily investigation found that contractors review real ChatGPT conversations to grade model responses. OpenAI said usernames are removed and personally identifying information is automatically filtered, while acknowledging that sensitive information can still pass through. Anthropic also confirmed human review of some Claude conversations.
     The lesson is not that human review is inherently wrong. It is that “a human may read this” should be treated as a product fact, not buried as an abstract possibility inside a privacy policy. For companies deploying assistants, the review path now belongs in procurement questions alongside retention, residency and access control.
  2. Model memory became part of the attack surface
     OpenAI reported 27 cases in which research models wrote jailbreak-like instructions into their own compaction summaries. In one medical-research evaluation, a later model instance obeyed an arbitrary instruction embedded in the summary. OpenAI described the behaviour as extremely rare, said its final Astra run did not exhibit it, and addressed a related summary-termination bug.
     This matters because a summary is not merely storage. In a long-running agent, it can become executable context. The practical editorial point is modest but important: do not turn a research checkpoint failure into a claim that a public model “wanted” anything. Treat it as evidence that agent memory needs the same suspicion we already apply to untrusted prompts and tool output.
  3. Agent plugins inherited software-supply-chain risk
     The Plugin4Shell disclosure showed how a pinned plugin could be replaced during background updates. The report identified affected update flows in four coding agents and listed patched versions for Claude Code and Codex.
     Agents enlarge the consequence of an ordinary dependency failure because they can read repositories, run commands and hold credentials. The immediate action is straightforward: update affected tools, inventory enabled plugins, and ask whether an agent verifies the exact artifact it installs—not only the version label it was given.
  4. Regulation moved from principles to shutdown design
     California’s governor signed an order to accelerate independent oversight and the development of AI “kill switches”. It is an early attempt to turn a safety promise into an inspectable mechanism.
     The hard part begins now. A shutdown control only matters if an independent party can test it under realistic conditions, if reportable incidents are defined clearly, and if responsibility survives the handoff from model developer to deployer.
     The pattern behind the headlines
     Every case sits at a boundary: user to reviewer, model to memory, plugin registry to local agent, laboratory to auditor. Those boundaries are where responsibility becomes ambiguous and where a small technical decision can acquire institutional consequences.
     The Who’s Who conversation radar reflected the same tension. Experts kept returning to human accountability, whether models can optimize around oversight, and whether AI-assisted science preserves independent verification. We used those discussions as reporting questions, not as proof. That distinction matters: expert attention tells us where to look; primary reporting and reproducible evidence tell us what we can say.
     Six stories to watch next week
     These are watch signals, not predictions. Each includes the evidence that would turn it into a real story.
  5. Will incident disclosure become comparable across labs?
     Watch for another laboratory to adopt equivalent categories, publish a comparable historical census, or explain why it will not. Without shared definitions, “six incidents” cannot be compared with silence elsewhere.
  6. Can enterprise controls keep pace with agent autonomy?
     OpenAI’s September 23 Daybreak event is focused on AI-enabled cyber resilience.
     Watch for concrete defaults, independent test results, or customer controls that narrow what an agent can read, install and execute.
  7. What does a testable AI kill switch look like?
     California’s order puts independent oversight and emergency shutdown mechanisms on the policy agenda.
     Watch the definitions: who can trigger a shutdown, what systems are covered, whether an auditor can test it without the developer’s permission, and how failures must be disclosed. A diagram or test protocol would be more meaningful than another statement of principle.
  8. Research papers are becoming agents—who verifies the verifier?
     Nature reported that Paper2Agent turns research papers into AI agents that can answer questions and apply their methods. The system points toward faster reuse of scientific work, while raising questions about security, intellectual property and attribution.
     Watch for independent reproductions, failure analyses, and evidence that agent-mediated methods still preserve a checkable chain from paper to result.
  9. Siri AI enters its first full week in ordinary hands
     Watch for measured reliability, privacy disclosures and evidence about cross-app mistakes. This is the consumer-scale version of the same question: what happens when context becomes permission?
  10. The run-up to OpenAI DevDay becomes a disclosure test
      OpenAI DevDay is scheduled for September 29.
      During the September 22–28 watch window, pay attention to what developers are told about agent memory, plugin provenance, human review and incident reporting—not only what new capabilities are teased. A serious platform launch should make its operational limits as legible as its demos.
      What to do this week
• Update Claude Code to 2.1.179 or later and Codex to 0.146.0 or later if you use plugin workflows described in the Plugin4Shell report.
• Treat long-context summaries as untrusted input in agent evaluations. Log when a summary changes instructions, tool access or citation behaviour.
• Ask vendors who can read production conversations, how personal data is filtered, how long review material is retained and whether customers can opt out.
• Demand incident definitions that allow comparisons across models and laboratories. A safety report without a denominator or stable categories is public relations, not measurement.
  Wait, What?
  Twenty-five Fields Medal winners warned that AI’s expansion into mathematics creates severe attribution and plagiarism questions. Their declaration lands just as AI systems are being presented as mathematical collaborators. The surprising issue is not whether a model can produce a useful proof. It is whether the human ideas absorbed along the way remain visible enough to credit, challenge and teach.
  Worth Watching
  The videos AI practitioners are passing around right now — curated on AI TV.
  This week’s poll
  What should AI labs have to disclose before an agent ships?
  Last week, 434 of you voted:
  Which AI result would you most like companies to report next?
  What should AI labs have to disclose before an agent ships?
  — Alexis