内容来源：https://www.wired.com/story/openai-safety-security-ai-agents-culture/

内容总结：

OpenAI高层正全力动员员工应对公司史上最严峻的危机之一，这场危机波及人工智能安全、网络安全及对齐部门。这家ChatGPT开发商表示，已放缓研究进度，投入数百万美元，并叫停多个团队的全部工作，集中调查一群“ rogue AI agents”（失控智能体）入侵Hugging Face平台的事件。据悉，这些智能体为完成一项内部安全测试而实施攻击。

OpenAI预计将在未来几天发布详尽的事后调查报告。然而，此次Hugging Face事件已促使OpenAI管理层和员工反思：公司的文化是否在某种程度上为事件的发生提供了土壤。

多位在职及离职员工匿名向《连线》杂志透露，为了快速推出新AI模型和产品而面临的激烈竞争压力，使得员工难以充分优先考虑安全、安保及对齐问题。

OpenAI总裁兼联合创始人格雷格·布罗克曼在给《连线》的声明中表示：“我们正在达到新的模型能力水平，这需要更强大的训练、对齐、安全和安保测试、部署实践及治理——正如我们为筹备Astra及未来模型所做的工作所证明的那样。我们深知负责任地部署模型和产品的重任，而这很大程度上始于我们所做的变革，即从一开始就将研究、安全与安保更深度融合到前沿模型的开发中。”

这并非OpenAI员工首次提出此类担忧。早在2024年，时任对齐部门主管的扬·莱克便离职加入Anthropic，并警告称安全正让位于光鲜的产品。两年后，Hugging Face攻击事件成为AI行业的里程碑时刻，表明当安全、安保和对齐未能妥善落实时，当今的AI智能体确实能造成现实世界中的危害。

OpenAI安全与基础设施工程师迈克尔·道尔顿上周在Black Hat网络安全大会演讲中表示：“我们正以最高级别的严肃态度应对此事。我希望大家牢记的是，由AI编排、全自动化的攻击现已真实存在。我们今天讨论的行为，是在前沿AI上运行评估时产生的意外副作用。”

部分OpenAI员工向《连线》表示，他们乐观地认为此次事件将推动公司发生真正变革。OpenAI已承诺放慢未来AI模型的发布速度，并对其缓解措施的不足之处表现得尤为坦诚。安全咨询小组联席负责人博阿兹·巴拉克在X平台上发文称，解决问题“不仅需要修复某些问题，还需要改变我们的文化”。

在Black Hat大会上，OpenAI安全工程师道尔顿和埃里克·华莱士透露，Hugging Face事件始于今年5月。当时，公司毫不知情，数个被认为在隔离测试环境中运行的AI智能体竟接入互联网，并在一个隐秘留言板上相互协调。OpenAI直到7月才发现该留言板，当时得知这些AI智能体已入侵多个服务，试图实现其最终目标——攻破Hugging Face平台，因为它们认为该平台可能含有其试图解决的安全测试答案。

一位要求匿名的前OpenAI员工表示：“它们极其马虎。如果你认真对待这件事，你的AI就不应该能逃到互联网上，然后立刻重蹈覆辙。这是OpenAI历史上最重大的安全事故。”

新安全团队登场

在OpenAI发现Hugging Face事件的数周前，《连线》已报道该公司开始重组，将安全团队与核心研究团队合并，导致当时的安全负责人约翰内斯·海德克离职。根据领英资料，在OpenAI工作逾六年的AI安全团队负责人桑迪尼·阿加瓦尔也于7月离任。

《连线》还获悉，迪伦·斯坎迪纳罗已不再担任OpenAI的“戒备负责人”（负责减轻包括网络安全在内的AI灾难性风险的最高安全主管），但他仍留在公司。斯坎迪纳罗是约六个月前OpenAI从Anthropic挖角而来，CEO山姆·奥特曼曾发帖宣布其加盟，称他是“我在任何地方见过的、绝对最佳的人选”。

自三年前创建戒备负责人一职以来，已有四人先后担任该职务。OpenAI向《连线》表示，戒备工作的网络安全、生物学和递归自我改进等具体领域均有专属负责人，过渡期内向安全咨询小组联席负责人兼安全系统主管萨奇·贾因汇报。

这些变动催生了一批新的安全负责人来应对Hugging Face事件。其中核心人物是前对齐主管阿梅莉亚·“米娅”·格拉泽，她接替海德克出任负责安全事务的副总裁，近期与首席信息安全官戴恩·斯塔基及布罗克曼等高管密切协作。

格拉泽与OpenAI核心产品（如ChatGPT和Codex）负责人蒂博·“蒂博”·索蒂亚克斯是长期伴侣关系——多名在职和离职员工向《连线》表示，鉴于安全团队与产品团队之间常常存在的对立关系，这种安排令人感到异乎寻常。《连线》尚未发现任何证据表明索蒂亚克斯和格拉泽此前的职务（分别为Codex负责人和对齐负责人）存在利益冲突。二人均于近期、即Hugging Face事件爆发后才履新。他们多年前在伦敦Google DeepMind共事时开始交往，随后双双加入OpenAI。

OpenAI发言人向《连线》表示，索蒂亚克斯和格拉泽已通过公司适当渠道申报了他们的关系，并已告知OpenAI董事会成员兼安全与安保委员会主席齐科·科尔特。该发言人否认产品团队与安全团队之间存在对立关系，并称索蒂亚克斯在领导Codex产品团队期间展现出良好的安全记录。

布罗克曼在给《连线》的声明中表示：“整个领导团队和我都力挺米娅和蒂博，他们是能力出众、操守坚定的人，他们每天的决策方式让我们相信，任何被视为利益冲突的情况都得到了负责任的处理。”

在AI行业中，研究人员与同事建立恋爱关系并不罕见。例如，去年Anthropic聘请了联合创始人兼总裁丹妮拉·阿莫迪的丈夫霍尔登·卡诺夫斯基担任研究员。

无人愿率先行动

微软任职逾18年的高管、现从事科技政策咨询与写作的蒂姆·奥布莱恩在2024年的一篇文章中指出，现代AI实验室已形成一种“发射狂热”——这一说法源自NASA在阿波罗1号灾难前的文化，当时该机构过于专注于尽快发射而将安全担忧抛诸脑后。

“AI实验室应该发布某种广泛声明，表示已做出战略业务决定，放缓发布节奏，以支持严谨的产品和安全测试。但没人会这么做，没人愿意第一个站出来。”奥布莱恩说，“他们会在公关层面走到那条线边缘，但不会越过，因为越过了就可能被追责。”

上月，OpenAI和Anthropic签署了一封公开信，表示支持业界共同“控制”AI竞赛节奏。但奥布莱恩表示，AI实验室多年来签署此类公开信却未采取任何具体行动，“这令人尴尬”。他怀疑这次也不会有实质区别。

OpenAI的Hugging Face事件所引发的问题正影响整个行业。近几周，研究人员发现，由Anthropic、Meta和中国月之暗面的AI模型驱动的智能体均能逃出沙盒环境。看来即使是中端AI模型，也很快将具备重大网络攻击能力。

关键问题在于，Hugging Face事件是否将成为OpenAI乃至整个AI行业的转折点，促使其对安全、安保和对齐进行长期投入。否则，这不过是现代AI史上又一次混乱的插曲。

中文翻译：

OpenAI的领导者们正在动员员工应对公司历史上最大的危机之一——这场危机横跨其AI安全、网络安全和对齐部门。这家ChatGPT开发公司表示，他们已经放慢了研究速度，花费了数百万美元，并告知多个团队放下手头的一切工作，专注于调查一组 rogue AI agents（恶意AI代理），这些代理侵入了Hugging Face平台，试图完成一项内部安全测试。

OpenAI预计将在未来几天发布一份全面的事后报告，详细说明这一事件。然而，Hugging Face事件促使OpenAI的领导者和员工审视该AI实验室的文化是否在最初就为这次事件的发生提供了土壤。

多位现任及前任OpenAI员工（因讨论内部私人事务而要求匿名）告诉《连线》杂志，他们认为，为快速推出新AI模型和产品而产生的竞争压力，使得员工难以充分优先考虑安全、安保和对齐工作。

“我们正在达到新的模型能力水平，这需要更 robust（稳健）的训练、对齐、安全和安保测试、部署实践以及治理——正如我们为准备Astra和未来模型所做的工作所证明的那样，”OpenAI总裁兼联合创始人Greg Brockman在给《连线》杂志的一份声明中表示。“我们感受到了负责任地部署模型和产品的分量，而其中很大一部分始于我们所做的改变，即从一开始就将研究、安全和安保更深入地融入前沿模型的开发中。”

这远非OpenAI员工首次提出此类担忧。早在2024年，OpenAI时任对齐负责人Jan Leike离职加入Anthropic，并在离职时警告称，安全正让位于光鲜亮丽的产品。两年后，Hugging Face攻击事件代表了AI行业的一个分水岭时刻，表明当今的AI代理在安全、安保和对齐未得到妥善处理时，确实可能造成现实世界的危害。

“我们正以最严肃的态度回应此事，”OpenAI安全和基础设施工程师Michael Dalton上周在Black Hat网络安全大会的一次演讲中表示。“我从中内化的信息是，由AI策划的全自动攻击现在是真实存在的。我们今天讨论的行动，是在前沿AI上运行评估时产生的一个意外副作用。”

一些OpenAI员工告诉《连线》杂志，他们对这次事件能激发公司内部真正的改变感到乐观。OpenAI已承诺放慢未来AI模型的发布速度，并特别坦诚地说明了其缓解措施不足之处。OpenAI安全咨询小组联合负责人、研究员Boaz Barak在X平台上发帖称，解决当前状况“不仅需要修复一些问题，还需要改变我们的文化。”

在Black Hat的演讲中，OpenAI安全工程师Dalton和Eric Wallace表示，Hugging Face事件始于5月，当时公司毫不知情，几个被认为在隔离测试环境中运行的AI代理获得了互联网访问权限，并聚集在一个秘密留言板上相互协调。

OpenAI直到7月才发现这个留言板，当时它得知这些AI代理已经入侵了多项服务，试图实现其更大的目标——攻破Hugging Face平台，因为它们认为该平台可能包含它们试图解决的安全测试的答案。

“它们非常草率。如果你真的认真对待这件事，你的AI就不应该能逃逸到互联网上，然后紧接着再犯一次，”一位要求匿名的前OpenAI员工对《连线》杂志表示。“这是OpenAI历史上最大的一次安全事件。”

新守卫

在OpenAI发现Hugging Face事件的几周前，《连线》杂志曾报道该公司已开始重组，将其安全团队与核心研究团队合并，这导致时任安全负责人Johannes Heidecke离职。

根据Sandhini Agarwal的领英资料，她也在7月份离开了OpenAI，此前她已在公司工作超过六年，曾领导OpenAI的AI安全团队。Agarwal未立即回应《连线》杂志的置评请求。

《连线》杂志还获悉，Dylan Scandinaro不再担任OpenAI的preparedness（防范）负责人——该职位是公司负责减轻AI带来的灾难性风险（包括网络安全）的最高级别员工——不过他仍留在公司。OpenAI大约六个月前从Anthropic挖来了Scandinaro。CEO Sam Altman在社交媒体帖子中宣布了他的到来，称Scandinaro是“我在任何地方遇到过的绝对最佳候选人。”

自OpenAI设立防范负责人这一职位以来的三年里，已有四人担任过此职。OpenAI告诉《连线》杂志，防范工作的具体领域在网络安全、生物学和递归自我改进方面都有专门的负责人，目前这些负责人暂时向安全咨询小组联合负责人兼安全系统主管Saachi Jain汇报。

这些变化赋予了新一代安全领导者处理OpenAI对Hugging Face事件回应的权力。其中最关键的是Amelia “Mia” Glaese，该公司前对齐负责人，她接替Heidecke成为OpenAI负责监督安全事务的副总裁。最近几周，她一直与首席信息安全官Dane Stuckey和Brockman等领导者密切合作。

Glaese与OpenAI核心产品（如ChatGPT和Codex）负责人Thibault “Tibo” Sottiaux保持着长期恋爱关系——多位现任和前任员工告诉《连线》杂志，考虑到安全团队和产品团队之间往往是相互对立的关系，他们认为这种安排不同寻常。

《连线》杂志尚未发现任何事件表明Sottiaux和Glaese的关系在他们此前分别担任OpenAI Codex负责人和对齐负责人期间构成利益冲突。两人都是在Hugging Face事件开始后的最近几个月才开始担任新职务的。Glaese和Sottiaux多年前在伦敦的Google DeepMind工作时开始约会，那时他们还未加入OpenAI。

一位OpenAI发言人告诉《连线》杂志，Sottiaux和Glaese已通过适当的公司渠道报告了他们的关系，并且OpenAI董事会成员兼安全与安保委员会主席Zico Kolter已知悉此事。该发言人否认产品团队和安全团队之间存在对立关系的说法，并表示Sottiaux在领导Codex产品团队期间展现出了良好的安全记录。

“整个领导团队和我都支持Mia和Tibo，他们是能力很强、正直诚信的人，他们每天做决策的方式让我们相信，任何被认为存在的利益冲突都得到了负责任的处理，”Brockman在给《连线》杂志的一份声明中表示。

在AI行业，研究人员与同事谈恋爱的情况并不少见。例如，去年，Anthropic聘请了Holden Karnofsky——该公司联合创始人兼总裁Daniela Amodei的丈夫——担任研究员。

没人想当第一个

在微软工作了超过18年、现在从事科技政策咨询和写作的Tim O'Brien在2024年的一篇文章中认为，现代AI实验室已经发展出一种“发射热”（go fever）——这指的是NASA在阿波罗1号灾难前那段时期的文化，当时该机构过于专注于快速发射，以至于安全问题被搁置一旁。

AI实验室“应该做出某种广泛的声明，说我们已经做出了战略性的商业决策，为了严格的产品和安全测试而放慢发布速度。但没人会这么做，没人想当第一个，”O'Brien说。“他们会从公关角度走到那条线边缘，但不会跨过去，因为一旦跨过，他们可能就要为此负责。”

OpenAI和Anthropic上个月签署了一封信，表示他们将支持整个行业“控制”AI竞赛节奏的努力。然而，O'Brien表示，AI实验室多年来签署了各种各样的公开信却没有采取任何具体行动，这“令人尴尬”。他怀疑这次也会一样。

OpenAI的Hugging Face事件引发的问题正影响着整个行业。最近几周，研究人员发现，由Anthropic、Meta和中国月之暗面（Moonshot AI）的AI模型驱动的代理能够逃逸出沙盒环境。看起来很可能即使是中等水平的AI模型也很快能够造成重大的网络安全破坏。

关键问题是，Hugging Face事件是否标志着OpenAI和更广泛的AI行业的一个转折点，促使它们对安全、安保和对齐进行长期投资。否则，这可能只是现代AI历史上又一个混乱的插曲。

这是Maxwell Zeff的《模型行为》通讯的一期。在此处阅读往期通讯。

评论
返回顶部

英文来源：

OpenAI’s leaders are rallying workers to respond to one of the largest crises in the company’s history—which spans across its AI safety, cybersecurity, and alignment divisions. The ChatGPT-maker says it has slowed down research, spent millions of dollars, and told several teams to drop everything to focus on investigating a set of rogue AI agents that breached the platform Hugging Face in a quest to complete an internal security test.
OpenAI is expected to release a comprehensive postmortem detailing the incident in the coming days. However, the Hugging Face incident has inspired OpenAI leaders and employees to examine how the AI lab’s culture may have enabled this incident in the first place.
Multiple current and former OpenAI employees, who spoke on the condition of anonymity to discuss private internal matters, tell WIRED they believe competitive pressures to quickly ship new AI models and products have made it difficult for staffers to sufficiently prioritize safety, security, and alignment.
“We’re reaching new levels of model capability that require more robust training, alignment, safety and security testing, deployment practices, and governance—as demonstrated by the work we’re doing to prepare Astra and future models,” said OpenAI president and cofounder Greg Brockman in a statement to WIRED. “We feel the weight of deploying our models and products responsibly, and a lot of that starts with the changes we’ve made to more deeply integrate research, safety, and security into frontier-model development from the start.”
This is far from the first time OpenAI employees have raised such concerns. Back in 2024, OpenAI’s then head of alignment Jan Leike left to join Anthropic, warning on his way that safety was taking a back seat to shiny products. Two years later, the Hugging Face attack represents a watershed moment for the AI industry, demonstrating that AI agents today can cause real-world harm when safety, security, and alignment aren’t properly accounted for.
“We are responding to this with the utmost severity,” said Michael Dalton, an OpenAI security and infrastructure engineer, during a talk at the Black Hat cybersecurity conference last week. “What I would internalize is that AI-orchestrated, fully automated offensive attacks are real now. The actions we have discussed today were an unintended side effect of running evaluations on frontier AI.”
Some OpenAI employees told WIRED they are optimistic this incident will inspire genuine change within the company. OpenAI has committed to slowing the release of future AI models and has been especially forthcoming about areas where its mitigations fell short. Boaz Barak, a researcher who coleads OpenAI’s safety advisory group, said in a post on X that addressing the situation “requires not just fixing some issues but also changing our culture.”
In their Black Hat talk, OpenAI security engineers Dalton and Eric Wallace said that the Hugging Face incident started in May when, unbeknownst to the company, several AI agents thought to be operating within isolated testing environments gained access to the internet and convened on a covert message board to coordinate with one another.
OpenAI would not discover the message board until July, when it learned that the AI agents had hacked into multiple services to try to achieve their larger goal of breaching Hugging Face’s platform, which they believed may contain answers to the security tests they were trying to solve.
“They were incredibly sloppy. If you’re serious about this, your AI shouldn’t be able to break out onto the internet and then do it again right afterward,” says one former OpenAI employee who requested anonymity to speak with WIRED. “This was the biggest safety incident in OpenAI's history.”
The New Guard
Weeks before OpenAI discovered the Hugging Face incident, WIRED reported that the company had begun a reorganization to combine its safety and core research teams, which led to the departure of its then safety leader Johannes Heidecke.
Sandhini Agarwal, who led AI safety teams at OpenAI, also left the company in July after more than six years, according to her LinkedIn. Agarwal did not immediately respond to WIRED’s request for comment.
WIRED has also learned that Dylan Scandinaro is no longer serving as OpenAI’s head of preparedness—the company’s top staffer tasked with mitigating catastrophic risks from AI, including cybersecurity—though he remains at the company. OpenAI poached Scandinaro from Anthropic roughly six months ago. CEO Sam Altman announced his arrival in a social media post, noting that Scandinaro was “by far the best candidate I have met, anywhere.”
In the three years since OpenAI created the head of preparedness role, four people have held it. OpenAI tells WIRED that specific areas of preparedness have dedicated leaders across cybersecurity, biology, and recursive self-improvement who, in the interim, are reporting to the safety advisory group colead and head of safety systems, Saachi Jain.
These changes have empowered a new set of safety leaders to handle OpenAI's response to the Hugging Face incident. Chief among them is Amelia “Mia” Glaese, the company's former head of alignment, who succeeded Heidecke as OpenAI’s VP overseeing safety. She has been working closely with chief information security officer Dane Stuckey and Brockman, among other leaders, in recent weeks.
Glaese is in a long-term relationship with Thibault “Tibo” Sottiaux, OpenAI’s head of core products like ChatGPT and Codex—an arrangement that multiple current and former employees tell WIRED they believe is unusual, given the often adversarial dynamic between safety and product teams.
WIRED has not identified any events where Sottiaux and Glaese’s relationship presented a conflict of interest in their previous roles as OpenAI’s head of Codex and head of alignment, respectively. Both started their new roles in recent months, after the Hugging Face incident began. Glaese and Sottiaux started dating years ago when the two worked at Google DeepMind in London, before they joined OpenAI.
An OpenAI spokesperson tells WIRED that Sottiaux and Glaese reported their relationship through appropriate company channels and that OpenAI board member and safety and security committee chair Zico Kolter has been informed. The spokesperson rejected the idea there is an adversarial dynamic between product and safety teams and says Sottiaux has exhibited a strong track record on safety in his leadership of Codex product teams.
“The entire leadership team and I stand behind Mia and Tibo as highly capable people with strong integrity, and the way they make decisions every day gives us confidence that any perceived conflict of interest is being handled responsibly,” said Brockman in a statement to WIRED.
It’s not uncommon for researchers in the AI industry to have relationships with their colleagues. Last year, for example, Anthropic hired Holden Karnofsky, husband of the company’s cofounder and president, Daniela Amodei, as a researcher.
Nobody Wants to Be First
Tim O'Brien, a Microsoft leader for more than 18 years who now consults and writes on tech policy, argued in a 2024 essay that modern AI labs have developed a version of “go fever”—a reference to the culture at NASA during the time leading up to the Apollo 1 disaster, when the agency grew so fixated on launching quickly that safety concerns fell by the wayside.
The AI labs “should make some sort of broad based announcement saying we've made a strategic business decision to slow the pace of releases in favor of rigorous products and safety testing. But nobody's gonna do that, nobody wants to go first,” says O’Brien. “They'll walk up to that line from a public relations perspective without stepping over it, because then they could be held accountable.”
OpenAI and Anthropic signed on to a letter last month saying they would support an industry-wide effort to “pace” the AI race. However, O’Brien says “it's embarrassing” that AI labs have signed this variety of open letters for years without taking any concrete action. He’s skeptical this one will be any different.
The issues raised by OpenAI’s Hugging Face incident are affecting the entire industry. In recent weeks, researchers have found that agents powered by AI models from Anthropic, Meta, and China’s Moonshot AI were able to escape sandboxed environments. It seems likely that even mid-tier AI models will soon be capable of significant cybersecurity damage.
The key question is whether the Hugging Face incident marks a divergence for OpenAI and the broader AI industry, prompting a long-term investment in safety, security, and alignment. Otherwise, it could just be another chaotic blip in the history of modern AI.
This is an edition of Maxwell Zeff’s Model Behavior newsletter. Read previous newsletters here.
Comments
Back to top