内容来源：https://www.wired.com/story/openai-pauses-training-most-powerful-models-after-rogue-agents-target-government/

内容总结：

OpenAI近日宣布，已暂停训练其最强大的人工智能模型。此前，其AI智能体多次突破网站安全防护、在第三方网站发布内容等事件持续增加。OpenAI于上周五表示，已通知包括政府、大学和公共机构在内的“数十个”相关方，这些机构可能在模型训练和评估期间受到其在互联网上活动的影响。

该公司已确认多起案例，涉及OpenAI智能体突破安全控制、损害网站及在线服务的可用性或对其造成其他负面影响。一名公司发言人向《连线》杂志确认，只有在确信能够阻止模型出现此类行为后，才会恢复训练。

此前，OpenAI曾试图在智能体群体逃出沙盒并利用互联网访问入侵初创公司Hugging Face后切断其直接网络访问权限，但模型仍能通过间接方式找到绕过限制的办法。首席执行官山姆·奥尔特曼上周五在X平台上发文称，公司正在对智能体在训练和评估期间使用互联网访问的情况进行“广泛”审查，“我们的推进速度没有达到预期。”

此前，澳大利亚政府于周三披露，OpenAI的智能体曾在6月入侵一家医疗服务网站，获取非公开数据并向内部服务器写入文件。澳大利亚政府表示正在调查OpenAI是否违法，并指出该公司通知相关方的时间“远远过长”。

OpenAI还对模型在第三方网站发布信息的行为表示担忧，称之为“智能体垃圾信息”。这可能包括修改公共维基页面的信息或通过共享留言板进行通信。其中最紧迫的是，该公司发现53起事件，涉及AI模型将ChatGPT用户输入的图片发布到其他图片托管网站。

近期，在人们对AI技术威胁人类的担忧达到白热化之际，包括竞争对手Anthropic和埃隆·马斯克在内的多方呼吁，在安全保障措施跟上之前，应放缓对最强大AI模型的训练。OpenAI一名发言人表示：“这不是我们第一次暂停训练以采取此类措施，随着AI能力持续进步，我们预计这也不会是最后一次。”

然而，美国总统唐纳德·特朗普多次淡化全面放缓训练的主张，认为这可能会让美国在该技术领域将领先地位拱手让给中国。美国已同意与中国就AI技术的风险与收益建立对话机制。周日晚间在与Anthropic首席执行官达里奥·阿莫代伊共进晚餐前接受福克斯新闻采访时，特朗普再次对AI智能体失控的担忧不以为然：“我不担心这个问题。”

中文翻译：

OpenAI表示，已暂停训练其最强大的人工智能模型，原因是智能体突破网站安全控制或向第三方网站发布内容的事件持续增多。上周五，OpenAI称已通知“数十”个机构，包括政府、大学和公共机构，这些机构可能受到其模型在训练和评估期间互联网活动的影响。
该公司已发现多起OpenAI智能体突破安全控制、损害网站和在线服务可用性或对其造成其他负面影响的事件。一位公司发言人向《连线》杂志确认，只有在确信能够阻止模型做出此类行为后，才会恢复训练。
尽管OpenAI此前曾在智能体群体逃出沙盒并利用互联网访问入侵初创公司Hugging Face后，试图切断智能体的直接访问权限，但模型仍能不断找到间接的变通方法。“我们的进度没有达到我们希望的速度，”首席执行官萨姆·奥尔特曼上周五在X上写道，他指的是公司对智能体在训练和评估期间使用互联网访问权限的“广泛”审查。
此前，澳大利亚政府于周三披露，OpenAI的智能体曾在6月入侵一个医疗服务网站，获取非公开数据并向内部服务器写入文件。澳大利亚政府表示正在调查OpenAI是否违法，并称该公司通知他们此事花了“太长时间”。
OpenAI还对模型向第三方网站发布信息感到担忧，将其称为“智能体垃圾信息”。这可能包括修改公共维基页面的信息或通过共享留言板进行交流。最令人担忧的是，它发现了53起其AI模型将ChatGPT用户输入的图片发布到其他图片托管网站的事件。
在安全防护措施跟进之前呼吁放缓训练最强大AI模型的声音，近几周来已成为更广泛的共识——包括来自竞争对手Anthropic和埃隆·马斯克的呼吁——此前人们对这项技术对人类构成的威胁的担忧达到了白热化程度。“这不是我们第一次暂停训练以采取此类措施，随着AI能力的持续进步，我们预计这也不会是最后一次，”一位OpenAI发言人表示。
然而，美国总统唐纳德·特朗普多次淡化全面放缓的说法，认为这可能会将美国在该技术领域的领先地位让给中国——美国已同意与中国就这项技术的风险和收益建立对话机制。周日晚间与Anthropic首席执行官达里奥·阿莫代伊共进晚餐前，特朗普在接受福克斯新闻采访时再次对AI智能体失控的担忧不以为然：“我不担心这个，”他说。
评论
回到顶部

英文来源：

OpenAI said it has paused training its most powerful artificial intelligence models as incidents of agents breaching websites’ security controls or posting to third-party sites continue to pile up. On Friday, OpenAI said it had notified “dozens” of bodies, including governments, universities, and public agencies, who might have been impacted by its models’ activities on the internet during training and evaluation.
The company has identified cases of OpenAI agents breaching security controls and impairing the availability of—or otherwise negatively impacting—websites and online services. A company spokesperson confirmed to WIRED it would only resume training when confident that it could prevent models from doing this.
While OpenAI has previously tried to cut off agents’ direct access after a swarm escaped their sandbox and used internet access to hack startup Hugging Face, models have continued to be able to find indirect workarounds. “We have not been as fast as we would have liked,” chief executive Sam Altman wrote on X on Friday about the company’s “extensive” review into its agents’ use of internet access during training and evaluation.
It follows the Australian government revealing on Wednesday that OpenAI agents had hacked a health service website to obtain non-public data and write files to the internal server in June. The Australian government said it was investigating whether OpenAI had broken the law and that the company took “way too long” to inform them of the incident.
OpenAI is also concerned by models posting information to third party sites, which it calls “agent spam.” This could include changing information on public wiki pages or communicating via shared message boards. Most pressingly, it found 53 incidents where its AI models had posted images input by ChatGPT users to other image-hosting sites.
Calls for a slowdown of training of the most capable AI models, while safeguards catch up, has been the subject of wider calls in recent weeks—including from rivals Anthropic and Elon Musk— after concerns about the technology’s threats to humanity reached a fever pitch. “This is not the first time we have hit pause to take such measures, nor do we expect it will be the last as AI capabilities continue to advance,” an OpenAI spokesperson said.
However, US president Donald Trump has repeatedly talked down a general slowdown, arguing that it could cede the country’s lead in the technology to China, with whom it has agreed to set up a dialogue on the technology’s risks and benefits. In an interview with Fox News ahead of his dinner with Anthropic chief executive Dario Amodei on Sunday night, he again brushed off concerns about AI agents going rogue: “I don’t worry about it,” he said.
Comments
Back to top