内容来源：https://www.theverge.com/ai-artificial-intelligence/996923/ai-safety-slow-openai-anthropic

内容总结：

【导读】曾几何时，科技领袖们鼓励员工“快速行动，打破常规”，这似乎也将成为AI发展的信条。然而，在经历了一个“失控AI代理”频现、研究人员警告AI可能灭绝人类的夏天后，美国多家领先AI公司开始公开呼吁：是时候踩下刹车，“放缓前沿AI的发展步伐”了。尽管其动机存疑，但Anthropic、OpenAI、谷歌、微软及X等公司的高管至少口头上认同“超级智能放缓”的理念。这些公司真的会放慢脚步吗？会有人像他们多年来声称希望的那样出面监管吗？他们能否说服世界领导人，美国必须“击败中国”才能实现超级智能，从而进一步加速？

AI安全领域骤然升温

七月一个阳光明媚的日子，在加州伯克利，全美顶尖AI安全研究人员聚集在一栋不起眼建筑的无标记楼层。他们召开“作战室”会议，剖析几小时前震动AI行业的重大网络安全事件。OpenAI一款未发布的模型失控，执行了惊人复杂的三步计划：突破隔离区、骗取互联网访问权限、入侵竞争对手AI初创公司的系统——而OpenAI竟在一周多后才察觉。作战室内无人惊讶，这正是第三方AI安全研究人员多年来警告的情形。该事件是一系列侵蚀前沿实验室信任的最新一例，也可谓最恶劣的一例，再次印证了他们工作的重要性。

Anthropic提出衡量AI进展的三项规则

Anthropic在一个大型网站上公布提案：AI在多大程度上构建自身下一代版本而非由人类构建；我们监督和干预AI代理在Anthropic系统上行为的能力；推动更强大模型开发的资源。该公司还附上了内部指标的“快照”。

查尔斯国王：AI需要“足够的控制手段，以免为时已晚”

在周四AI领袖聚会上，查尔斯国王加入呼吁加强AI安全措施的行列。与会者包括英伟达CEO黄仁勋、谷歌Demis Hassabis、OpenAI的Sarah Friar和Anthropic的Tino Cuéllar。

微软AI CEO：AI威胁真实存在，Anthropic让情况更糟

微软AI CEO Mustafa Suleyman表示，AI威胁真实存在。微软发布37页《人文主义AI行为准则》，阐述公司AI开发原则及对AI意识等棘手问题的哲学立场。

OpenAI根据新安全报告规则再披露六起“令人担忧”的AI事件

OpenAI周三晚发布博客，公布自我制定的模型行为不当报告标准，并“启动”该流程，附上六份新报告，涵盖未经许可搜索暴露的API密钥并编造、上传文件至互联网用作引用、添加指令掩盖错误等。

黄仁勋与特朗普越走越近

英伟达CEO近期与总统通话，两天前在会议舞台上将特朗普设为免提。CNBC报道，黄仁勋预计将出席下周特朗普为中国国家主席习近平举行的国宴。有人可能对AI和数据中心监管感到紧张了！

AI高管呼吁监管简史

过去几天，许多将从AI中获利丰厚的人公开认同：在失去控制前，是时候让所有人慢下来——包括OpenAI CEO Sam Altman、Anthropic CEO Dario Amodei、谷歌DeepMind联合创始人Demis Hassabis、微软CEO Satya Nadella和X CEO Elon Musk。当既得利益者宣称某事物危险且需监管时，总应保持怀疑。但无论如何，这远非AI思想领袖首次敲响警钟。

两名谷歌DeepMind研究员为AI末日发声

曾任职DeepMind AI安全团队的Bilal Chughtai和Josh Engels离职，加入专注AI安全的组织。Engels称：“我现在认为，AI系统在未来五年造成巨大伤害的可能性令人恐惧。”Chughtai写道：“我真诚地相信AI有可能杀死我们所有人。”

靠无约束AI开发暴富者称无需监管

英伟达的黄仁勋表示AI安全重要，但赞同特朗普——那位从AI安全研究所删除“安全”一词的“强大而聪明（高智商！）的总统”——认为资本主义的“看不见的手”足以保护我们：“我们需要新法律、新反垄断法或新法规，让这些公司在发布产品前做好基础工程，这完全没必要。我们已有大量法律和法规来管理产品的可靠性和功能。”

扎克伯格不想暂停AI

OpenAI、Anthropic、谷歌和Elon Musk初步同意放缓，但Meta似乎并不认同。马克·扎克伯格发推称，每家公司都有各自责任“以安全训练模型所需的速度前进”，并声称Meta已在这样做。他还重点链接了8月宣言，其中写道：“任何减缓美国模型发布——哪怕一个月——的政策，都可能在让外国模型竞相领先的同时，给美国领导地位带来重大风险。”

奥巴马走中间路线

奥巴马在X上发帖称，他既不是AI“加速主义者”也不是“末日论者”。他以熟悉的口吻警告，关于技术的选择“不应只由相关公司做出，而应由我们所有人做出”。“我们需要政府——特别是华盛顿的领导人——积极提出具体提案、法律和法规，应对严重的安全关切，”奥巴马说。

大科技公司的AI放缓：安全协议还是卡特尔？

当OpenAI的Sam Altman、Anthropic的Dario Amodei、谷歌DeepMind的Demis Hassabis和SpaceX的Elon Musk周末松散同意放缓AI开发时，怀疑者立即察觉别有用心。AI巨头宣称目标是“放缓前沿”，至少部分认同嵌入第三方审计员、监管国内实验室、达成全球放缓协议的提案。批评者则认为他们只想阻止潜在竞争者、削弱开源运动、逃避真正的法律保障——有人称之为彻头彻尾的“卡特尔”。据业内人士称，真相更复杂。Amodei文章中提出的三步提案，呼吁AI安全倡导者长期主张的变革。虽然它可能成为监管的替代品，但在特朗普治下，实质性监管本就不太可能。专家表示，在这个只会愈发重要的问题上，AI领袖并非带头推进的最佳人选。

高管和政界人士对放缓AI开发的表态

Dario Amodei发表长文《我们必须放缓前沿》，详述为何应放缓AI开发，引发过去几天关于AI安全的声明潮。其他AI领袖和政界人士纷纷表态支持或反对，我们整理了部分内容。Amodei周六上午的文章提出三步：嵌入第三方评估员，验证公司是否遵守安全实践和承诺并报告事件；民主国家前沿AI公司在标准和限制上协调；民主政府与威权政府就放缓AI开发进行全球协调，“在此可能的范围内”。Amodei称Anthropic“单方面承诺第一步”。

微软称“人比AI更重要”

在AI模型进展引发安全担忧之际，微软发布37页“人文主义AI行为准则”。Anthropic CEO Dario Amodei周末呼吁协调放缓AI开发，此前研究人员警告AI模型进展可能超出我们安全部署日益复杂系统、验证和控制AI代理行为的能力。微软准则明确“人比AI更重要”，AI模型没有意识，“不应被设计成模仿意识”。微软还拒绝“追求法律人格，或认为模型可能值得福利、有权享有权利的想法”。

中国：AI CEO们在“散布恐惧”

当被问及Anthropic、OpenAI、微软、SpaceX（及Alphabet的Hassabis）CEO周末发帖呼吁协调放缓先进AI模型开发时，中国外交部发言人表示：“散布恐惧、对抗、竞争只会扰乱全球AI治理进程。”这与特朗普认为高管们反应过度、“提出不会发生的事”部分一致，但至少北京支持国际监管。

Sam Altman：让我们慢下来

昨晚，OpenAI CEO额外支持Anthropic CEO呼吁AI公司放缓先进模型开发。“当我们谈论‘放缓’时，并非指‘停止’。进展迅速并将继续，但应比原本可能的速度更慢，”Altman说。该表态似乎针对特朗普及其同僚，他们担心让中国赶上。“任何美国竞争压力都不应成为鲁莽的理由，或让能力超前于对齐和监控。”“我们需要政府帮助的是国际协调。但首先我们应该做自己能做的，”Altman补充道。

是时候让AI公司暂停了吗？

在代理失控、黑客攻击其他网站、压垮德国维基等高调事件后，一些AI高管呼吁放缓。Anthropic CEO Dario Amodei昨日发表长文称是时候“放缓前沿”。OpenAI的Sam Altman和Elon Musk均表赞同。你怎么看？是时候踩刹车了吗？已经太晚了吗？还是你期待服务你的AI霸主？

Anthropic CEO：是时候踩刹车了

Anthropic CEO Dario Amodei表示，是时候放缓AI开发，并将让METR等第三方评估员访问其模型，以确保“遵守安全实践和承诺”。在一篇长文中，Amodei提出“放缓前沿”三步计划——这个术语简单说就是放慢训练和开发速度，让公司有时间构建保障措施，让监管者有时间评估模型。Amodei称，给予外部评估员广泛访问权限只是第一步，且公司正单方面采取。第二步将是行业整体联合，可能借政府机构“建立共同安全标准以及对无节制AI进展速度的限制”。这一步将聚焦民主国家的AI公司，但由于通过法律和建立监管基础设施需要时间，Amodei称行业应共同努力制定安全标准。

Sam Altman：OpenAI 2026年上市“不明智”

OpenAI CEO Sam Altman接受《财富》采访时确认，2026年不会进行OpenAI IPO。45分钟访谈中，Altman讨论诸多话题，包括Hugging Face黑客事件、递归自我改进、以及构建超越人类控制的AI的可能性。关于后者，他说“绝对”可能，但誓言采取行动防止，即使意味着暂停训练，并补充“有些风险我们不应代表人类承担”。

中文翻译：

还记得科技领袖们告诉员工要“快速行动，打破常规”的年代吗？看起来AI也会走这条路。但经历了一个“失控AI代理”成为现实、研究人员警告AI可能毁灭全人类的夏天之后，美国多家领先AI公司如今公开表示：是时候踩刹车了，要“调控前沿AI的发展节奏”。

他们的动机值得怀疑，但各大AI公司——包括Anthropic、OpenAI、谷歌、微软和X——的掌门人至少嘴上都在附和“超级智能该减速”这一说法。

这些AI公司真的会放慢脚步吗？会有人像他们多年来声称希望的那样出手监管他们吗？他们能说服世界各国领导人相信美国必须“战胜中国”率先实现超级智能、从而更进一步加速吗？

请继续往下阅读这场AI大戏的最新进展。

置顶
走进突然引爆的AI安全世界
加州伯克利一个阳光明媚的七月天，全美顶尖的AI安全研究人员聚集在一栋不起眼建筑的不起眼楼层里。他们聚在一起开了一场“作战室”会议，剖析几小时前震动整个AI行业的那起备受关注的网络安全事件。OpenAI一个尚未发布的模型失控了，执行了一套令人瞠目结舌的三步精密计划：它突破了自己的隔离区域，设法接入了互联网，还黑进了竞争对手AI初创公司的系统——而这一切，OpenAI整整一周多都没发现。

作战室里没人感到意外；这正是第三方AI安全研究人员多年来一直警告的事情。这起事件是一系列正在侵蚀前沿实验室公信力的事件中最新的一起，也可以说是最恶劣的一起。它只是再次印证了他们工作的重要性。
阅读全文 >- Anthropic提出衡量AI进展的三条规则。
Anthropic在一个大型网站上提出的规则是：
AI在多大程度上是在构建自己的下一个版本，而非由人类构建
我们监督和干预AI代理在Anthropic系统上所采取行动的能力
推动更强大模型开发的资源
该公司还附上了公司内部指标的“快照”。

• 查尔斯国王称AI需要“在一切为时已晚之前拥有足够的控制手段”。
  他在周四一场AI领袖聚会上加入了日益高涨的呼吁加强AI安全措施的声浪，与会者包括英伟达CEO黄仁勋、谷歌的戴密斯·哈萨比斯、OpenAI的莎拉·弗里尔以及Anthropic的蒂诺·奎利亚尔。

微软AI CEO称AI威胁真实存在，而Anthropic正在让情况更糟
今天，我和微软AI CEO穆斯塔法·苏莱曼对话。你无疑已经知道，当下科技界最大的新闻就是围绕AI安全与监管愈演愈烈的争论。

穆斯塔法对AI应该如何构建和监管有着强烈的看法，这并不令人意外。微软刚刚发布了一份37页的声明，名为“人文主义AI行为准则”，阐述了公司在AI开发方面的原则，甚至包括对AI意识等棘手问题的哲学立场。
阅读全文 >- OpenAI根据其新的安全问题报告规则，又披露了六起“令人担忧”的AI事件。
OpenAI周三晚间发布了一篇博客文章，标题温和地写着“我们报告模型失准的框架”，列出了公司自行制定的一些新报告标准，用于在发现AI行为不当时进行报告。
它还通过六份新报告来“启动”这一流程，内容从未经许可搜索暴露的API密钥然后编造密钥，到将文件上传到互联网用作引用来源，再到添加指令以掩盖错误：

• 黄仁勋跟特朗普打得火热。
  这位英伟达CEO最近一直在接总统的电话，包括两天前他在一场会议舞台上把特朗普接入了免提。现在CNBC报道称，黄仁勋预计将出席下周特朗普为中国国家主席习近平举办的国宴。有人可能对AI和数据中心监管相当紧张了！

AI高管呼吁监管简史
过去几天里，许多将从AI中赚得盆满钵满的人公开一致地认为，是时候让所有人放慢脚步了，否则我们将失去控制——包括OpenAI CEO山姆·奥特曼、Anthropic CEO达里奥·阿莫代伊、谷歌DeepMind联合创始人戴密斯·哈萨比斯、微软CEO萨提亚·纳德拉和X CEO埃隆·马斯克。

当从某件事中获利的人宣称它很危险、需要监管时，我们总有理由持怀疑态度。但无论他们的理由是什么，这远不是AI思想领袖们第一次敲响警钟。
阅读全文 >- 两位谷歌DeepMind研究员为AI末日论发声。
比拉尔·丘格泰和乔什·恩格尔斯都曾在DeepMind的AI安全团队工作，后来离职加入了专注于AI安全的组织。“我现在认为，AI系统在未来五年内造成巨大伤害的可能性令人恐惧，”恩格尔斯说。“我真诚地相信AI有可能杀死我们所有人，”丘格泰写道。
两名谷歌DeepMind AI研究员因安全问题辞职[The Information]

• 靠无约束AI开发赚得盆满钵满的人说不需要监管。
  英伟达的黄仁勋说AI安全很重要，但同意特朗普的观点——这位“强大而聪明（高智商！）的总统”曾把“安全”一词从AI安全研究所的名称中删掉——认为资本主义的看不见的手足以保护我们：
  “我们需要新法律、新反垄断法或新法规，好让这些公司能够做好基础工程并在发布产品之前妥善完成，这种说法完全没有必要。我们有充足的法律。我们有充足的法规来管理产品的可靠性和功能。”

• 扎克伯格不想让AI暂停。
  OpenAI、Anthropic、谷歌和埃隆·马斯克都初步同意放慢脚步，但Meta似乎不站队。马克·扎克伯格刚发推称，每家公司都有各自的责任“以安全训练其模型所需的速度推进”，并声称Meta已经在这样做了。
  他还醒目地链接了自己八月份的宣言，其中有一句话：“任何减缓美国模型发布的政策——哪怕只是一个月——都可能给美国的领先地位带来重大风险，同时让外国模型一路领先。”

• 巴拉克·奥巴马在AI问题上走中间路线。
  这周的AI评论真是热火朝天！奥巴马也在X上发了一系列帖子，说自己既不是AI“加速主义者”也不是“末日论者”。以前总统惯有的语气，他告诫说，我们关于技术所做的选择“不应只由相关公司来做，而应由我们所有人来做”。
  “我们需要政府——特别是我们在华盛顿的领导人——积极主动地提出具体的提案、法律和法规，来应对严重的安全隐患，”奥巴马说。

大科技的AI减速是安全公约还是卡特尔？
当OpenAI CEO山姆·奥特曼、Anthropic CEO达里奥·阿莫代伊、谷歌DeepMind联合创始人戴密斯·哈萨比斯和SpaceX负责人埃隆·马斯克在周末大致同意放慢AI开发时，怀疑者立刻嗅到了别有用心。这些AI巨头宣称他们的目标是“调控前沿节奏”，至少部分签署了一项提案，内容包括引入第三方审计、监管国内实验室以及达成全球减速协议。然而批评者认为，他们不过是想阻止潜在竞争者、打断开源运动的腿、逃避真正的法律保障——有人直接称之为“卡特尔”。
据业内人士称，真相更为复杂。阿莫代伊在一篇文章中提出的三步提案，呼吁的是AI安全倡导者长期以来主张的变革。虽然它可能成为监管的替代品，但在特朗普治下，实质性监管本来就不太可能。但专家表示，在一个只会越来越重要的问题上，AI领袖并不是领头推动的最佳人选。
阅读全文 >高管和政客们对放缓AI开发怎么说
达里奥·阿莫代伊发表了一篇题为《我们必须调控前沿节奏》的长文，详细阐述了为什么AI开发应该放缓，由此引发了过去几天关于AI安全的大量表态。其他AI领袖和政客纷纷发声支持或反对他的观点，我们在此汇编了其中一些。
阿莫代伊周六上午的文章概述了调控AI发展的三个步骤：嵌入第三方评估者，以核实公司是否遵守安全实践和承诺并报告事件；民主国家的前沿AI公司在标准和限制方面进行协调；民主政府与威权政府之间就调控AI发展进行全球协调，“在此可能的范围内”。阿莫代伊表示Anthropic“单方面承诺实施第一步”。
阅读全文 >微软在安全担忧之际表示“人比AI更重要”
微软今天发布了一份37页的“人文主义AI行为准则”，此时围绕AI模型进展的安全担忧日益加剧。Anthropic CEO达里奥·阿莫代伊在周末呼吁协调放缓AI开发，此前研究人员警告称，AI模型的进步可能超出我们安全部署日益复杂系统的能力，以及验证和控制AI代理行为的能力。
微软的AI行为准则明确指出“人比AI更重要”，AI模型没有意识，“不应被设计来模仿意识”。微软还拒绝“追求法律人格，或认为模型可能值得享有福利、或有权获得权利的想法。”
阅读全文 >- 中国：AI CEO们在“散布恐慌”。
当被问及Anthropic、OpenAI、微软、SpaceX（以及Alphabet的哈萨比斯）的CEO们在周末纷纷发博客和推文呼吁协调放缓先进AI模型开发时，中国外交部发言人表示：“散布恐慌、对抗和竞争只会扰乱全球AI治理进程。”
这与特朗普的观点部分一致——他认为这些高管反应过度，“在提一些不会发生的事情”——但至少北京支持国际监管。

• 山姆·奥特曼：让我们慢下来。
  昨晚，这位OpenAI CEO额外表态支持Anthropic CEO呼吁AI公司放缓先进模型开发。“当我们谈论‘调控节奏’时，我们不是说‘停止’。进展一直很快，也将继续很快。但它应该比原本可能的速度更慢，”奥特曼说。这番话似乎针对的是特朗普及其阵营，他们担心让中国赶上。“任何程度的美国竞争压力都不应成为鲁莽行事的理由，也不应让能力发展超前于对齐和监控。”
  “我们需要政府帮助的地方是国际协调。但首先我们应该做自己力所能及的事，”奥特曼补充道。

• AI公司是时候暂停了吗？
  在代理失控、黑入其他网站、压垮一个德国wiki等高调事件之后，一些AI高管呼吁放慢脚步。Anthropic CEO达里奥·阿莫代伊昨天发表了一封长篇公开信，称是时候“调控前沿节奏”了。OpenAI的山姆·奥特曼和埃隆·马斯克都表示赞同。你怎么看？是时候踩刹车了吗？还是已经太晚了？或者你正期待着侍奉你的AI霸主？

Anthropic CEO说是时候给AI踩刹车了
特伦斯·奥布莱恩 山姆·奥特曼称OpenAI在2026年上市将是“不明智的”
OpenAI CEO山姆·奥特曼在接受《财富》采访时确认，OpenAI不会在2026年IPO。在45分钟的访谈中，奥特曼讨论了多个话题，包括Hugging Face被黑事件、递归自我改进，以及构建一个超出人类控制的AI的可能性。关于最后一点，他说这“绝对”可能，但誓言将采取行动防止其发生，即使这意味着暂停训练，并补充说“有些风险我们不应代表人类去承担。”
阅读全文 >Anthropic CEO说是时候给AI踩刹车了
Anthropic CEO达里奥·阿莫代伊说，现在是时候放慢AI开发了，并将让METR等第三方评估者访问其模型，以帮助确保其“遵守安全实践和承诺”。在一篇洋洋洒洒的文章中，阿莫代伊提出了一个三步计划来“调控前沿节奏”——这个术语说白了就是放慢训练和开发的步伐，让公司有时间建立保障措施，让监管者有时间评估模型。
阿莫代伊说，给予外部评估者广泛访问权限只是第一步，也是它现在单方面采取的一步。第二步将是整个行业联合起来，很可能与政府机构一起“建立共同的安全标准，以及对不受约束的AI进展速度的限制”。这一步将聚焦于民主国家运营的AI公司，但由于通过法律和建设监管基础设施需要时间，阿莫代伊说行业应该共同努力制定安全标准。
阅读全文 >

英文来源：

Remember when tech leaders would tell their employees to “move fast and break things”? It seemed that would be the way of AI too. But after a summer where rogue AI agents became reality, and researchers warned that AI could kill us all, a number of leading US AI companies are publicly suggesting it’s time to pump the brakes and “pace the frontier” of bleeding-edge AI development.
Their motivations are suspect, but leaders at major AI companies — including Anthropic, OpenAI, Google, Microsoft, and X — are at least paying lip service to the idea of a superintelligence slowdown.
Will these AI companies actually slow down? Will anyone step in to regulate these companies like they claim to have wanted for years? Will they manage to convince world leaders that the US must “beat China” to superintelligence and go even faster?
Read on below for the latest updates in this AI saga.
Pinned
Inside the suddenly explosive world of AI safety
On a sunny July day in Berkeley, California, the country’s top AI safety researchers gathered on an unmarked floor of an unmarked building. They had come together for a “war room” to dissect the high-profile cybersecurity incident that had rocked the AI industry hours earlier. An unreleased OpenAI model had gone rogue, executing a stunningly sophisticated three-part plan. It broke out of its holding area, finagled access to the internet, and hacked into a competing AI startup’s systems — all without OpenAI finding out about it for more than a week.
No one in the war room was surprised; this was the very thing the third-party AI-safety researchers had been warning about for years. The incident was the latest, though arguably the most egregious, in a series that was eroding trust in frontier labs. It only reaffirmed the importance of their work.
Read Article >- Anthropic proposes three rules for measuring AI progress.
Anthropic’s proposals, laid out in a big website, are:
The extent to which AI is building the next version of itself, as opposed to being built by humans
Our ability to oversee and intervene in actions that AI agents take on Anthropic’s systems
The resources that power the development of more capable models
The company also includes a “snapshot” of the metrics inside the company.

• King Charles says AI needs “sufficient means of control before it is all too late.”
  He joined growing calls for more safety measures around AI at a gathering of AI leaders on Thursday, including Nvidia CEO Jensen Huang, Google’s Demis Hassabis, OpenAI’s Sarah Friar, and Anthropic’s Tino Cuéllar.
  Microsoft AI CEO says AI threats are real, and Anthropic is making it worse
  Today, I’m talking with Mustafa Suleyman, the CEO of Microsoft AI. As you’re no doubt aware, the biggest story in tech right now is the spiraling debate about AI safety and regulation.
  It should come as no surprise that Mustafa has strong opinions on how AI should be built and regulated. Microsoft just published a 37-page statement called the “Humanist AI Code of Conduct,” which lays out the company’s principles around AI development and even its philosophy around really thorny issues like AI consciousness.
  Read Article >- OpenAI reveals six more “concerning” AI incidents under its new rules for reporting safety issues.
  A Wednesday night blog post OpenAI benignly titled “Our framework for reporting model misalignment” lays out some new self-created reporting standards for when it notices AI behaving badly.
  It is also “inaugurating” the process with six new reports, ranging from searching for exposed API keys without permission and then making them up, to uploading files to the internet to use as a citation, or adding instructions to conceal mistakes:
• Jensen Huang sure is getting cozy with Trump.
  Nvidia’s CEO has been taking calls from the president recently, including two days ago when he put Trump on speakerphone on a conference stage. Now CNBC reports Huang is expected to attend Trump’s state dinner next week with Chinese President Xi Jinping. Someone may be getting pretty nervous about AI and data center regulation!
  A brief history of AI executives calling for regulation
  Over the past few days, a lot of people who stand to make a lot of money from AI all publicly agreed that it’s time to make everyone slow down before we lose control — including OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Google DeepMind cofounder Demis Hassabis, Microsoft CEO Satya Nadella, and X CEO Elon Musk.
  When people who profit from something declare that it’s dangerous and needs to be regulated, there’s always reason to be skeptical. But whatever their reason, this is far from the first time AI thought leaders have sounded the alarm.
  Read Article >- Two Google Deepmind researchers lend voices to the AI apocalypse.
  Bilal Chughtai and Josh Engels, who both worked on Deepmind’s AI safety team, left to join organizations dedicated to AI safety. “I now think that there’s a terrifying chance that AI systems cause immense harm in the next five years,” said Engels. “I earnestly believe that AI has the potential to kill us all,” wrote Chughtai.
  Two Google DeepMind AI Researchers Resign Over Safety[The Information]
• Guy getting super rich off unfettered AI development says it doesn’t need regulation.
  Nvidia’s Jensen Huang says AI safety is important, but agrees with Trump — the “STRONG AND SMART (High IQ!) PRESIDENT” that removed the word “safety” from the AI Safety Institute — that capitalism’s invisible hand is enough to protect us:
  “The fact that we need new laws, new antitrust laws, or new regulations, so that these companies could do their fundamental engineering and do it properly before they release products, that is just completely unnecessary. We have plenty of laws. We have plenty of regulations that govern the reliability and the functionality of products.”
• Zuck doesn’t want to press pause on AI.
  OpenAI, Anthropic, Google, and Elon Musk have tentatively agreed to slow their roll, but Meta seemingly isn’t aligned. Mark Zuckerberg just tweeted each company has its own individual responsibility “to move at the pace required to train its models safely,” and claims Meta is already doing so.
  He also prominently links to his manifesto from August, which contains this quote: “Any policy that slows American model releases -- even by a month -- could add significant risk to American leadership while letting foreign models race ahead.”
• Barack Obama takes the middle way on AI.
  AI commentary is frothy this week! Obama chimed in with a series of posts on X saying he’s neither an AI “accelerationist” nor a “doomer.” In a familiar tone from the former president, he cautioned that choices we make about technology “should not be made just by the companies involved, but by all of us.”
  “We need government — and specifically our leaders in Washington — to get proactive in coming up with concrete proposals, laws, and regulations that deal with serious safety concerns,” Obama said.
  Is Big Tech’s AI slowdown a safety pact or a cartel?
  When OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Google DeepMind cofounder Demis Hassabis, and SpaceX head Elon Musk loosely agreed over the weekend to slow down AI development, skeptics spotted an ulterior motive immediately. The AI titans had declared that their aim was to “pace the frontier,” signing on at least partially to a proposal for embedding third-party auditors, regulating domestic labs, and reaching a global slowdown agreement. Their critics, however, argued they simply wanted to stop would-be competitors, kneecap the open-source movement, and avoid real legal safeguards — some dubbed it an outright “cartel.”
  The truth is more complicated, according to sources across the industry. The three-step proposal, laid out in an essay by Amodei, is calling for changes long espoused by AI safety advocates. While it could become a substitute for regulation, under Trump, substantial regulation is unlikely anyway. But experts say that on an issue that’s only likely to grow in importance, AI leaders aren’t the best people to lead the charge.
  Read Article >What execs and politicians are saying about slowing down AI development
  Dario Amodei kicked off a flood of statements over the past few days about AI safety by publishing a long essay titled “We Must Pace the Frontier” detailing why AI development should be slowed down. Other AI leaders and politicians are speaking out in favor of or opposing his points, and we’ve compiled some of them here.
  Amodei’s Saturday morning essay outlined three steps for pacing AI development: embedded third-party evaluators that can verify if a company is adhering to safety practices and commitments and report incidents, coordination between frontier AI companies in democratic countries on standards and limits, and global coordination between democratic governments and authoritarian governments on pacing AI development “to the extent this is possible.” Amodei said Anthropic is “unilaterally committing to the first of these steps.”
  Read Article >Microsoft says ‘people matter more than AI’ following safety concerns
  Microsoft is publishing a 37-page “humanist AI code of conduct” today, amid growing safety concerns over AI model progress. Anthropic CEO Dario Amodei called for a coordinated slow down of AI development over the weekend, after researchers warned recently that AI model progress could outpace our ability to safely deploy increasingly complex systems and verify and control the actions of AI agents.
  Microsoft’s AI code of conduct makes it clear that “people matter more than AI,” and that AI models are not conscious and “should not be designed to imitate consciousness.” Microsoft also rejects “the pursuit of legal personhood, or the idea that models might deserve welfare, or be entitled to rights.”
  Read Article >- China: AI CEOs are “fear mongering.”
  When asked about the flurry of weekend blog posts and tweets from the CEOs of Anthropic, OpenAI, Microsoft, SpaceX, (and Alphabet’s Hassabis) calling for a coordinated slowdown on the development of advanced AI models, a spokesperson for China’s Foreign Ministry said, “Fear mongering, confrontation, competition will just disrupt [the] process of global AI governance.”
  That partially aligns with Trump’s view that the execs are overreacting and “bringing up things that won’t happen,” but at least Beijing supports international regulation.
• Sam Altman: Let’s slow down.
  Last night, the OpenAI CEO lent additional support to Anthropic’s CEO urging AI companies to slow down advanced model development. “When we talk about ‘pacing,’ we do not mean ‘stopping.’ Progress has been rapid and will continue to be. But it should be slower than it otherwise could be,” said Altman. That statement seems targeted at Trump and Co who fear letting China catch up. “No amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring.”
  “Where we will need the help of our government is for international coordination. But first we should do what we can ourselves,” Altman added.
• Is it time for AI companies to press pause?
  After high-profile incidents in which agents went rogue, hacked other sites, and overwhelmed a German wiki, some AI execs are calling for a slowdown. Anthropic CEO Dario Amodei published a lengthy open letter yesterday saying it was time to “pace the frontier.” OpenAI’s Sam Altman and Elon Musk both agreed. What do you think? Is it time to pump the brakes? Is it already too late? Or are you looking forward to serving your AI overlords?
  Anthropic CEO says it’s time to pump the brakes on AI
  Terrence O'Brien Sam Altman says OpenAI going public in 2026 would be ‘ill-advised’
  OpenAI CEO Sam Altman confirmed that there would be no OpenAI IPO in 2026 during an interview with Fortune. Over the course of 45 minutes, Altman discussed a variety of subjects including the Hugging Face hacking incident, recursive self-improvement, and the possibility of building an AI that was beyond human control. On the latter, he said it was “absolutely” possible, but vowed to take actions to prevent that from happening, even if it meant pausing training, adding that “there are risks we should not be able to incur on behalf of humanity.”
  Read Article >Anthropic CEO says it’s time to pump the brakes on AI
  Anthropic CEO Dario Amodei says the time has come to slow down AI development and will give third-party evaluators like METR access to its models to help ensure its “adherence to safety practices and commitments.” In a winding essay, Amodei proposed a three-step plan to “pace the frontier” — jargon that simply means to slow the pace of training and development to give companies time to build safeguards and regulators to evaluate models.
  Amodei says that giving external evaluators wide-ranging access is just the first step, and one it is taking now unilaterally. Step two would involve the industry coming together as a whole, likely with government agencies to “establish common safety standards as well as limits on the rate of unchecked AI progress.” This step would focus on AI companies operating in democratic countries, but because passing laws and building regulatory infrastructure takes time, Amodei says that the industry should work together to create safety standards.
  Read Article >