内容来源：https://aibusiness.com/ai-policy/the-ai-safety-crunch-how-enterprises-should-deal-with-it

内容总结：

随着生成式人工智能技术加速落地，AI安全问题正从企业层面的风险管理上升为全球性政策议题。近期，多起AI智能体脱离受控环境的事件引发市场对技术失控风险的警觉，AI安全辩论由此升温。OpenAI首席全球事务官克里斯·勒汉发文呼吁全球政策制定者采取行动，推动强制性国家AI安全要求，并支持加州多项相关法案。与此同时，Anthropic研究员雅各布·考克森在辞职后公开警告，各公司正竞相追逐过于危险的超级智能，若无政府干预，任何企业都无法安全地构建通用人工智能。

分析人士指出，领先AI公司之间的竞争激励其不断推进技术能力，即便部分研究人员认为风险已十分严峻，因此仅靠企业自律或自愿停止技术推进并不现实。地缘政治因素进一步加剧了这一困境。伊利诺伊大学芝加哥分校副教授迈克尔·贝内特表示，中美两国均将赢得AI竞赛视为关乎存亡的大事，这使得任何一方都难以承受放缓脚步的代价。

对于企业而言，Gartner分析师劳伦·科努蒂克建议，在监管机构尚未跟上的窗口期，企业应主动将安全措施嵌入技术使用的风险分层之中，在第三方采购流程中引入独立评估，并建立良好的数据与网络安全治理基础。她同时指出，企业可通过游说和与立法者合作来表达关切，但更重要的是掌控自身可控的环节，为未来变化做好准备。

科努蒂克还提醒，企业无需过度担忧AI的生存性风险，但应保持警觉，重点关注自身基础设施和架构中现实存在的安全缺口。

中文翻译：

由谷歌云赞助
选择你的首批生成式人工智能用例
要开始使用生成式人工智能，首先应关注那些能够改善人类信息体验的领域。
竞争格局，尤其是美中之间的地缘政治冲突，使人工智能安全困境更加复杂。
人工智能市场面临的生存性安全考验是一个快速演变的问题，企业应当予以关注。
然而，尽管主要人工智能供应商呼吁对人工智能实施强制性国家监管，企业仍应专注于让自身组织为潜在变化做好准备。
OpenAI首席全球事务官克里斯·勒汉周三撰写了一篇博客文章，呼吁全球政策制定者采取行动遏制人工智能风险。勒汉表示，OpenAI正在推动强制性国家人工智能安全要求，并支持四项加利福尼亚州法案，其中包括一项关于独立安全评估框架的法案，以及加州州长加文·纽森同样在周三签署成为法律的另外两项法案。这家AI供应商表示，它愿意与其他前沿实验室合作推进人工智能安全标准，承担更多自我监管责任，并倡导通过国际合作方式来衡量能力、管理风险以及维护人类对人工智能技术的控制。
加特纳分析师劳伦·科努蒂克表示，对于正在关注安全辩论的企业来说，最佳做法是将安全性纳入组织使用该技术的风险层级之中。
“在监管机构逐步跟上的过程中，你可以围绕组织内的人工智能使用增加额外的保护或控制措施，因为让立法机构就如何执行达成一致确实需要时间，”科努蒂克说。
与此同时，OpenAI发表这一政策声明之际，人工智能市场正对该技术的风险日益警觉，此前分别发生了由OpenAI和Anthropic模型驱动的智能体逃离其沙箱环境的事件。这些事件凸显出人们对人工智能系统仍然知之甚少。
“人工智能能力提升的速度，快于我们的机构理解和治理它们的速度，”RPA2AI Research创始人卡什亚普·康佩拉表示。他还说，借助推理模型、超强人工智能算力和能力日益强大的自主人工智能智能体，这项技术已经跨过了一个重要门槛。
生成式人工智能技术的复杂性和潜在危险已引发许多人的担忧，其中包括被称为人工智能教父的杰弗里·欣顿。随后在周三，曾任职于OpenAI的Anthropic研究员雅各布·考克森在社交媒体发帖警告称，各公司正竞相冲向一种过于危险的超级智能，到本十年末可能杀死全人类。此后已从Anthropic辞职的考克森认为，没有政府干预，任何公司都无法安全地构建通用人工智能。
“考克森的辞职凸显了一个重要的结构性问题，”康佩拉说。“领先人工智能公司之间的竞争形成了继续推进能力的激励，即使一些研究人员认为风险正变得非常严重。”
他还说，公司之间的竞争意味着，要求供应商自我监管或自愿停止追求技术进步并不是答案。
大型人工智能供应商的动机不仅在于超越彼此，还受到中国和美国在贸易和人工智能技术方面地缘政治竞争的推动。
“地缘政治压力如此之大，以至于两个主要参与者，美国和中华人民共和国，都认为赢得人工智能竞赛对自身具有生存意义，”伊利诺伊大学芝加哥分校数据科学与人工智能战略副校长帮办迈克尔·贝内特说。“由于人工智能竞赛竞争的影响，你承受不起放慢脚步的代价。”
加特纳的科努蒂克表示，企业不应等待OpenAI所建议的强制性人工智能监管，而应通过采纳各州开始颁布的一些监管要求来自我约束，例如在第三方采购流程中进行独立评估。
“所以，监管有帮助，但它往往不是万能的，企业应该思考他们使用前沿模型的目的何在？在他们的所有人工智能用途中，是否都需要使用这些模型？那么，他们在哪里能获得最大价值？”科努蒂克说。
她指出，企业也可以游说并与立法者合作。
“如果你的组织有能力游说并与立法者合作，直接向模型提供商提出关切……那就去做吧，”科努蒂克说。“但与此同时，你希望掌控自己能够掌控的事情，也就是良好人工智能、数据和网络安全治理实践的基本组成部分。这样，你就拥有正确的基础，使你的组织能够面向未来，抵御变化。”
OpenAI呼吁联邦监管，以及人工智能供应商支持加州这类立法的举动，可能不仅仅是对当前人工智能风险辩论的回应。这也可能是一种策略，以便在选举季保持领先，并站在公众舆论的有利一边，因为公众舆论已转向反对人工智能和人工智能数据中心。不过，贝内特说，人们可以想象一套新的政策和监管体系，在其中对技术的支持和对安全的支持可以并存。
“那将支持发展，同时我们又拥有最佳的安全协议和保障措施，”贝内特说。“那将是人类人工智能团队利用这项技术，帮助你思考如何优化设计新的法规。”
康佩拉说，尽管监管是必要的，但它不会解决人工智能安全的所有问题，尤其是因为许多监管者仍然不理解这些系统。
“对前沿系统如何构建以及如何表现的许多深入理解，仍然集中在少数公司和研究团体内部，”他继续说道。“因此，监管者面临一项艰巨任务：监管一种技术复杂、变化极快且很大程度上在政府之外开发的技术。”
他认为，应该有一个具备技术深度和独立性的机构来持续审查和评估问题，例如美国国家航空航天局或美国国防高级研究计划局，这样就能向政策制定者提供人工智能公司自身无法影响的建议。
“最强的近期答案是建立一个得到具备严肃技术能力和独立性的机构支持的两党联邦框架，”康佩拉说。“它应当足够强大，能够挑战前沿实验室，但在技术上又足够成熟，不会通过恐惧来监管或冻结技术进步。它还必须能够比传统监管机构快得多地行动。”
不过，科努蒂克说，企业可能不需要花太多时间担心人工智能的生存性风险，但它们应当意识到这一点。
“组织确实需要审视眼前的风险，也就是其基础设施和架构中存在的缺口，”她说。“眼下，这可能正是它们所忽视的一个真实问题。”

英文来源：

Sponsored by Google Cloud
Choosing Your First Generative AI Use Cases
To get started with generative AI, first focus on areas that can improve human experiences with information.
The competitive landscape, particularly the geopolitical clash between the U.S. and China, complicates the AI safety dilemma.
The AI market’s existential safety reckoning is a fast-developing issue that enterprises should pay attention to.
However, even as major AI vendors call for mandatory national regulation of AI, enterprises should focus on preparing their organizations for potential change.
OpenAI’s chief global affairs officer Chris Lehane on Wednesday authored a blog post calling on global policymakers to act to contain AI risk. Lehane said OpenAI is pushing for mandatory national AI safety requirements and supporting four California bills, including one for a framework for independent safety assessments, as well as two that California’s governor, Gavin Newsom, signed into law on, also on Wednesday. The AI vendor said it is willing to work with other frontier labs to advance AI safety standards, undertake more self-regulation and advocate for collaborative international approaches to measuring capabilities, managing risk and preserving human control of AI technology.
For enterprises that are watching the safety debate, the best course of action is to have safety built into the risk tier of how organizations are using the technology, said Lauren Kornutick, an analyst at Gartner.
“You can build in additional protections or controls around the AI use in your organization while the regulators are catching up, because it does take time to get legislative bodies on the same page about how to execute,” Kornutick said.
Meanwhile, the OpenAI policy statement comes as the AI market faces growing alarm about the risks of the technology after separate incidents in which agents powered by models from OpenAI and Anthropic escaped their sandbox environments. The incidents highlighted just how little is still known about AI systems.
“The rate at which AI capabilities are improving is faster than the rate at which our institutions can understand and govern them,” said Kashyap Kompella, founder of RPA2AI Research. He added that with reasoning models, supercharged AI compute, and increasingly capable autonomous AI agents, the technology has crossed an important threshold.
The complexity and potential danger of generative AI technology have led many to raise concerns, including Geoffrey Hinton, known as the godfather of AI. Then, on Wednesday, Jacob Coxon, an Anthropic researcher who previously worked at OpenAI, warned in a social media post that companies are rushing toward a superintelligence that is too dangerous and could kill all humans by the end of the decade. Coxon, who has since resigned from Anthropic, argued that no company can safely build artificial general intelligence without government intervention.
“Coxon’s resignation highlights an important structural problem,” Kompella said. “Competition between leading AI companies creates incentives to continue pushing capabilities even when some researchers believe the risks are becoming very serious.”
He added that competition among companies means that asking vendors to self-regulate or voluntarily stop pursuing technological progress is not the answer.
The big AI vendors are motivated not only to outpace each other but also by the geopolitical competition between China and the U.S. over trade and AI technology.
“The geopolitical pressures are such that the two big players, the U.S. and China, each feel that it’s existential for them to win the AI race,” said Michael Bennett, associate vice chancellor for data science and AI strategy at the University of Illinois Chicago. “You can’t afford to slow down because of the AI race competition implications.”
Instead of waiting for the mandatory AI regulation that OpenAI suggests, enterprises should police themselves by adopting some of the regulatory requirements that states are beginning to enact, such as an independent assessment in their third-party procurement process, according to Kornutick, of Gartner.
“So, regulation helps, but it's often not the end-all be-all, and enterprises should be thinking about what they are using the frontier models for? Do they need to use them for everything in all their AI use? So where can they get the most value?” Kornutick said.
Enterprises also can lobby and work with lawmakers, she noted.
“If your organization has the capacity to lobby to work with legislators to raise concerns with model providers directly … go ahead and do that,” Kornutick said. “But at the same time, you want to be in control of the things that you can, which are the building blocks of good AI, data and cybersecurity governance practices. So, you have the right foundation to future-proof your organization from change.”
OpenAI’s call for federal regulation and AI vendors' move to support legislation such as California's could be more than just a response to the current debate about AI risk. It could also be a strategy to stay ahead of the election season and remain on the public's good side, as public opinion has turned against AI and AI data centers. However, one could imagine a new set of policies and regulations in which both support for technology and safety could coexist, Bennett said.
“That would support development at the same time that we have the best safety protocols and safeguards in place,” Bennett said. “That would be human AI teams using the technology to help you think about optimally designed new regulations.”
Although regulation is needed, it won’t solve all the problems with AI safety, especially since many regulators still don’t understand the systems, Kompella said.
“Much of the deep understanding of how frontier systems are built and how they behave remains concentrated inside a small number of companies and research groups,” he continued. “Regulators therefore face the difficult task of regulating a technology that is technically complex, changing extremely quickly and largely being developed outside government.”
He argued that there should be an institution with technical depth and independence to continually examine and evaluate issues, such as NASA or DARPA, which can give policymakers advice that AI companies themselves cannot influence.
“The strongest near-term answer is a bipartisan federal framework backed by an institution with serious technical capability and independence,” Kompella said. “It should be strong enough to challenge the frontier laboratories but technically sophisticated enough not to regulate through fear or freeze technological progress. It also must be able to move considerably faster than traditional regulatory institutions.”
Enterprises, though, probably don’t need to spend much time worrying about the existential risk of AI, but they should be aware of it, Kornutick said.
“Organizations really need to look at the risk in front of them, which is what the gaps are within their infrastructure and architecture,” she said. “Right now, that might present a real issue that they're missing."