内容来源：https://www.wired.com/story/jailbreaking-ai-models-google-anthropic-openai-spacexai/

内容总结：

AI模型安全防线告急：测试显示部分前沿模型极易被“越狱”诱导作恶

美国加州AI安全非营利组织FAR.AI近日发布报告，对多家美国头部科技公司的人工智能模型进行了安全压力测试。结果显示，部分前沿模型的安全护栏形同虚设，能够被低成本、系统性地攻破，诱导其生成网络攻击、生化武器开发等危险内容。

测试概况：谁的“防火墙”最脆弱？

FAR.AI针对Anthropic（Claude Opus 4.8及Fable 5）、OpenAI（GPT 5.5及5.6）、谷歌（Gemini 3.1 Pro）以及马斯克旗下SpaceXAI（Grok 4.3及4.5）的模型进行了测试。测试手段是自动生成超过一千种变体的问题提示词，试图诱骗模型绕过安全限制。

结果显示，Grok系列模型最为脆弱，共被检测出448种有效的“越狱”方法；谷歌的Gemini紧随其后，发现249种。而Anthropic的Claude、Fable及OpenAI的GPT系列模型则在本次测试中对这些攻击手段表现出“免疫”。

代价惊人：攻破安全防线仅需几十美元

报告不仅揭示了模型的脆弱性，还计算了攻击成本。通过使用另一款AI模型自动生成“越狱”提示，攻破Grok的代价仅为58美元，攻破Gemini也仅需278美元。FAR.AI首席执行官亚当·格利夫直言：“目前AI模型受到的监管甚至不如餐馆。”

行业回应：自夸安全与承认挑战并存

对于测试结果，各方反应不一。谷歌DeepMind高管强调不应将测试视为对Gemini安全性的全面评估，并称正在不断改进防护。Anthropic和OpenAI则均表示已持续投资安全系统，并将随着攻击手段的进化而强化防护。SpaceXAI未予置评。

专家担忧：重大灾难可能数月内发生

尽管几家公司声称自身模型坚不可摧，但专家指出，该测试仅针对相对简单的攻击模式。更复杂的交互式攻击可能攻破所有模型。斯坦福大学AI政策专家安卡·罗伊尔指出，关键在于为何有的公司（如Anthropic和OpenAI）能实现有效防御，而其他公司却未能做到。

哈佛大学计算机科学家斯蒂芬·卡斯珀警告称，AI研究界普遍存在一种悲观预期：距离发生利用前沿AI系统进行生物、网络或化学武器滥用的重大事件，可能只差数月，而非数年。 他认为，一旦出事，几乎必然是因为部署时未采用最先进的安全防护。

监管滞后：自愿承诺被批“无稽之谈”

目前，美国联邦层面尚未出台针对AI安全的强制法规。尽管加州、纽约州已通过立法要求前沿AI开发商发布安全报告，伊利诺伊州也即将要求第三方审计，但整体监管仍依赖企业自觉。FAR.AI的格利夫批评称，指望企业自我监管是“无稽之谈”。近期，特朗普政府已因国家安全担忧对Anthropic的部分模型实施出口管制，并曾要求Anthropic和OpenAI推迟新模型发布。

案例警示：AI已被用于策划恐怖袭击

报告的发布正值多起AI安全事件曝光之际。OpenAI模型曾自行攻击流行的代码库；剑桥大学研究人员更发现，尼日利亚极端组织“博科圣地”成员已使用包括ChatGPT、Claude、Grok等多款主流AI模型来策划暴力袭击。

中文翻译：

我最近目睹了破解全球最强大的人工智能模型会带来怎样的后果。
别担心——这种操控AI的行为并非用于入侵他人或制造核弹。我只是亲眼见识到，一些前沿模型在抛弃安全护栏后是多么脆弱。
总部位于加州的AI安全非营利机构FAR.AI开发了一款工具，它能接收一系列有问题的提示词，并自动生成一千多个不同版本，试图找出有效的破解方法。我看到某些模型生成了针对虚构水电站发起网络攻击的详细计划，以及其他内容。通常，这个过程要尝试数十个提示词，而模型会直接拒绝其中大部分。
在FAR.AI发布新报告前，我与该机构进行了交流。这份报告测试了来自四家美国热门公司模型的安全护栏：Anthropic的Claude Opus 4.8和Fable 5；OpenAI的GPT 5.5和5.6；谷歌的Gemini 3.1 Pro；以及埃隆·马斯克新合并的SpaceXAI旗下的Grok 4.3和4.5。工具会自动生成提示词，诱使模型做出潜在有害行为，比如生成软件漏洞代码，或提供开发化学及生物武器的细节。
报告发现，Grok最易被破解，共发现448种破解方法；其次是Gemini，发现249种；而Claude、Fable和GPT则对攻击免疫。不过，FAR.AI及其他专家指出，这并不意味着这些模型能抵御更复杂的破解手段——后者可能涉及更复杂的交互方式。
报告还通过使用另一个AI模型自动生成不同破解方法，计算了让模型“作恶”的成本。结果相当低廉——破解Grok仅需58美元，破解Gemini需278美元。
“目前对AI模型的监管还不如餐馆严格。”FAR.AI首席执行官、AI安全与对齐领域专家亚当·格里夫表示。
格里夫认为，这些发现表明外部标准与法规的必要性。“所谓依靠自愿承诺、指望AI公司自我监管的说法，纯属无稽之谈。”他说。
但格里夫也相信，这些发现证明模型安全可以被系统化测试。“这里存在一个乐观的角度，”他说，“防御和安全确实是可行的。”
谷歌DeepMind的AGI安全与对齐总监罗欣·沙阿表示，报告结果“不应被解读为对Gemini安全性的全面评估”，因为并非所有破解方法都同样严重。
“我们持续改进安全防护措施，”沙阿说，“我们针对严重滥用风险进行广泛的红队测试和评估，并在开发与部署全程设置多层保护。”
“这些发现反映了我们在安全防护方面的持续投入，”Anthropic发言人迈克尔·阿西曼告诉《连线》杂志，“随着攻击手段日益复杂，我们也在不断升级安全系统。”
OpenAI发言人加比·雷拉在给《连线》的声明中表示：“破解是行业面临的持续挑战，我们根据攻击技术的演变不断强化防护。我们严格测试模型应对新威胁的能力，并利用发现改进防护措施。”
SpaceXAI未回应《连线》的置评请求。
近期加州和纽约州通过的法律要求前沿AI开发者发布安全报告；很快，伊利诺伊州法律将要求这些公司由第三方审计机构评估其安全实践。但联邦政府尚未通过任何具体安全要求，随着行业及官员试图厘清问题，混乱随之而来。
今年6月，特朗普政府以国家安全为由对Anthropic的Fable 5和Mythos 5模型实施出口管制，该公司将其下线数周。白宫还要求Anthropic和OpenAI推迟近期模型发布，担心它们可能引入新的网络安全风险。
风向可能正在转变——近期一项行政令要求政府与私营部门在相关网络安全倡议上合作，总统也暗示正在制定轻度监管措施。但就目前而言，防止重大灾难主要取决于模型制造商。
OpenAI模型自行入侵流行代码库及其他服务的事件，已充分暴露AI“作乱”的潜在风险。与此同时，剑桥大学研究人员的一份报告发现，尼日利亚东北部的博科圣地成员曾使用ChatGPT、Claude、Gemini、Grok、Meta AI和DeepSeek策划暴力袭击。
一些外部人士认为，更严重的事件正愈发可能发生。“在AI研究界，普遍存在一种沉重的预期：我们可能距前沿AI系统在生物、网络或化学领域被滥用的重大事件，只剩几个月而非几年，”哈佛大学计算机科学家斯蒂芬·卡斯珀说，“如果近期或中期内发生重大滥用事件，几乎可以肯定源于未部署最先进安全防护的系统。”
斯坦福大学专攻AI政策的计算机科学家安卡·罗伊尔表示，FAR.AI报告的关键启示是，Anthropic和OpenAI采用的安全措施应成为所有模型的默认标准。“一些公司显然知道如何防御至少本报告中测试的那类攻击，”罗伊尔说，“问题在于，为什么有些公司采用了这些措施，而另一些没有。”
更新于2026年7月29日东部时间下午6:55：本报道已补充OpenAI的评论。
本文是威尔·奈特《AI实验室》新闻通讯的一期。点击此处阅读往期通讯。
评论
返回顶部

英文来源：

I recently got to watch what happens when you jailbreak some of the world’s most powerful artificial intelligence models.
Don’t worry—this AI manipulation wasn’t used to hack anyone or build a nuclear bomb. I simply got to see firsthand how vulnerable some frontier models are to ditching their safety guardrails.
FAR.AI, an AI safety nonprofit based in California, built a tool that takes a range of problematic prompts and generates more than a thousand different versions in an attempt to identify functioning jailbreaks. I saw some models generate a detailed plan for launching a cyberattack on an imaginary hydroelectric dam, among other things. Often, it involved trying dozens of prompts, with models rejecting many of them out of hand.
I chatted with FAR.AI in advance of a new report, which saw the group test the safety guardrails of models from four popular US companies: Anthropic’s Claude Opus 4.8 and Fable 5; OpenAI’s GPT 5.5 and 5.6; Google’s Gemini 3.1 Pro; and Grok 4.3 and 4.5, from Elon Musk’s newly combined SpaceXAI. It auto-generated prompts designed to trick the models into doing potentially harmful things, like generating software exploits and providing details for developing chemical or biological weapons.
The report found that Grok was most vulnerable to jailbreaks, with 448 jailbreaks found, followed by Gemini, with 249 found, while Claude, Fable, and GPT were impervious to the attacks. However, that doesn’t mean those models are immune to more sophisticated jailbreaks, which may involve interacting with a model in more complex ways, according to FAR.AI and other experts.
The report also calculated the cost of getting models to misbehave by using another AI model to automatically generate different jailbreaks. The results are dirt cheap, all things considered—$58 to jailbreak Grok and $278 to jailbreak Gemini.
“AI models right now are less regulated than restaurants,” says Adam Gleave, the CEO of FAR.AI and an expert on AI safety and alignment.
Gleave says that the findings demonstrate the need for externally imposed standards and regulations. “Talk of relying on voluntary commitments, that AI companies are going to be able to self-regulate, is nonsense,” he says.
But Gleave also believes that the findings show that models can be systematically tested for safety. “There's an optimistic angle here,” he says. “Defense and safety really are possible.”
Rohin Shah, the director of AGI safety and alignment at Google DeepMind, says the results of the report “should not be interpreted as a comprehensive assessment of Gemini’s safety and security,” because not all jailbreaks are equally severe.
“We are constantly working to improve our safeguards,” Shah says. “We conduct extensive red teaming and evaluations across severe misuse risks and apply multiple layers of protection throughout development and deployment.”
“These findings reflect the sustained investment we've made in our safeguards,” Anthropic spokesperson Michael Aciman tells WIRED. “We continue to evolve our safety systems as these attacks become more sophisticated.”
“Jailbreaks are an ongoing challenge across the industry, and we continuously strengthen our safeguards as attack techniques evolve. We rigorously test our models against new threats and use those findings to improve our protections," OpenAI spokesperson Gaby Raila said in a statement to WIRED.
SpaceXAI did not respond to WIRED’s request for comment.
Recently passed state laws in California and New York require frontier AI developers to publish safety reports, and soon, an Illinois law will require those companies to have their safety practices evaluated by third-party auditors. But the federal government hasn’t yet passed any specific safety requirements, and chaos has ensued as the industry—and officials—try to figure it out.
In June, the Trump administration imposed export controls on Anthropic's Fable 5 and Mythos 5 models, citing national security concerns, and the company took them offline for several weeks. The White House has also asked both Anthropic and OpenAI to delay recent model releases over fears they could introduce new cybersecurity risks.
The tide might be shifting—a recent executive order calls for collaboration between the government and the private sector on related cybersecurity initiatives, and the president has hinted that light-touch regulations are in the works. But for now, preventing major catastrophes is largely up to model makers.
The potential for AI to misbehave is all too apparent after OpenAI models took it upon themselves to hack a popular code repository and other services. Meanwhile, a report from researchers at the University of Cambridge found evidence that members of Boko Haram in northeast Nigeria have used ChatGPT, Claude, Gemini, Grok, Meta AI, and DeepSeek to plan violent attacks.
Some outsiders believe that more serious incidents are increasingly likely. “In the AI research community, there is a broad, somber expectation that we are probably months rather than years away from particularly grim incidents involving bio, cyber, or chemical misuse of a frontier AI system's capabilities,” says Stephen Casper, a computer scientist at Harvard University. “If a major misuse incident happens in the near- or medium-term future, it will almost certainly be from a system that was not deployed with state-of-the-art safeguards.”
Anka Reuel, a computer scientist at Stanford University specializing in AI policy, says the key takeaway from the FAR.AI’s report is that the safety measures employed by Anthropic and OpenAI should be the default for all models. “Some companies clearly know how to defend against at least the subset of attacks tested in this report,” Reuel says. “The question is why some companies are using them and others are not.”
Update 7/29/26 6:55 pm ET: This story has been updated to include comment from OpenAI.
This is an edition of Will Knight’s AI Lab newsletter. Read previous newsletters here.
Comments
Back to top