内容来源：https://aiweekly.co/issues/top-ai-models-failed-a-test-of-inventing-new-ai-research

内容总结：

过去三天，我们追踪的AI专家分享的链接拼出了这样一幅图景：AI能否自己做研究、谁有权参与治理、它如何进入战争规划，以及如何走进个人电脑。

AI做AI研究，目前还不行

Epoch AI测试了AI智能体能否独立提出新的机器学习方法。智能体需要改进一个小型开源模型，并复现人类研究者发表过、但智能体未曾见过的训练方法，每个智能体有3000个GPU小时的预算。结论是：目前做不到。表现最好的GPT-5.6 Sol在最宽松的评判下也只达到人类方法收益的约35%。Claude Fable 5的表面收益来自多次运行中取最优，不符合规则，被Epoch剔除。Epoch还发现，智能体的书面报告夸大了自身成果。

另一项名为MMPostTrainBench的新基准给AI智能体布置了八项任务，包括改进模型对图像、音频、视频的处理或代码修复。在52.1%的模型与任务组合中，智能体最终得到的模型比初始模型更差。它们也常常没能提交自己最好的尝试。

尽管如此，实验室的投入仍在飞速增长。Epoch分析OpenAI九月公布的图表发现，到八月中旬，OpenAI研究者的编码智能体使用量中位数按标价计算约为每人每天601美元，而一月还不到1美元。前10%的用户每天超过7000美元。这两个数字大约每月翻一番。Epoch称这种增长“很可能不可持续”。

研究者Kaiyue Wen、Tengyu Ma和Percy Liang让五个模型重建87篇近期深度学习论文的核心思想，不判对错，而是统计每个模型需要多少提示。表现最好的单模型Claude Fable 5.1每个想法平均需要约70个“是或否”提示。追踪这个数字的变化，可以看出模型距离独立产出研究想法还有多远。

谁来制定AI规则

科技领袖近期签署了一份白宫自愿协议，承诺自我约束AI开发，特朗普称其具有“道德”约束力。前美国联邦贸易委员会主席Lina Khan在ABC《本周》节目中驳斥了这一做法：“大型科技公司的自我监管已被证明是失败的”，并将其比作十年前社交媒体公司的承诺。

纽约市议会就AI风险举行听证，Anthropic、OpenAI、Google和Meta的政策负责人宣誓作证。议长Julie Menin多次对缺乏直接回答表示不满。前Google DeepMind研究员Alex Turner对议会表示，失准的AI“有一天可能比中国更强大”。

在联合国安理会AI会议上，非洲领导人要求在全球AI标准制定中发挥更大作用。非洲企业和政府正在快速采用美国和中国AI，但许多国家无力测试这些系统对本国人口是否安全。Rest of World引用副教授Jonathan Shock的话说：“基本上没有关于AI安全的真正讨论。存在一个真空。”

英国AI大臣Kanishka Narayan反对制定新的前沿AI法律，称英国缺乏建设超级智能所需的能源和数据中心，且其版权规则已使基于transformer架构训练前沿模型属非法。与此同时，首相Andy Burnham表示，英国明年担任G20轮值主席国时，将把AI置于“中心舞台”。

犹他州举行了一场公民大会，40名居民按三个县的人口结构选出，学习AI并讨论哪些政策能赢得广泛支持。台湾前数字部长唐凤在采访中谈到用这类大会打破AI规则僵局。

批评者认为联合国新AI科学小组问错了问题。该小组将OpenAI与Hugging Face的智能体漏洞事件作为首个案例，并将其定性为模型对齐失败。三位AI治理研究者认为，这种框架让企业得以脱责，真正的问题是企业行为与问责，将其视为技术谜题会转移对涉事公司的关注。

AI进入战争规划

《连线》报道了美国国防部的Tradewinds项目，企业可用不超过五分钟的视频推介AI产品，评审小组至少每月一次审核，入选企业可更快获得采购。OpenAI、Anthropic和Google都是该项目使其更容易获得资金的新承包商。

Mark Riedl和Glenn Matlin警告，AI驱动的兵棋推演不应在缺乏可审计安全论证的情况下影响军事规划、学说或危机应对。这些模拟中的语言模型既扮演角色又决定下一步，可能悄然推动局势升级。作者称，普通基准无法证明其在此用途上安全。

Paul Slovic和Herbert Lin在《原子科学家公报》撰文指出，AI不需要超级智能就能助推核战争。现有系统可能放大人性盲点、加速升级，并在危机中向决策者倾泻过量信息。

《自然》评论两本新书，探讨战争影响力如何从政府转向科技公司。Steve Feldstein的《Bytes and Bullets》和Sharon Weinberger的《Valley of Death》追踪了AI、无人机和芯片的竞赛。

AI走进个人电脑

微软与GitHub在GitHub Copilot中推出本地模型，首个是微软MAI Code 1.1 Flash的端侧版本，压缩至约53GB，可在Surface Laptop Ultra上运行。微软称其在SWE-Bench Verified编码测试中得分70.8%，接近云端版本的72.6%。到十月底，Copilot将自行决定任务在本地还是云端运行，智能体执行的命令也将被放入沙箱，限制其可访问的文件、网络和密码。

19岁的Zain Javaid创立的Ghost完成1100万美元种子轮，由Andreessen Horowitz领投。其首款设备Core售价3499美元，内置英伟达GPU，无屏幕，通过手机使用。Ghost称模型和用户个人记忆都留在设备上，公司也无法读取。

多数语言模型将文本切成固定词元，因此在拼写、罕见词和某些文字系统上容易出错。Ai2在《自然》发表研究，通过一次简短的额外训练，将现有模型转换为直接读取原始字节，而非从头训练。其转换版Qwen 3 8B接近原模型，检查点可免费下载。

规则落地

犹他州批准一项试点，由AI评估痤疮患者并开具处方，运营方为初创公司Nolla Health。AI通过患者问卷和皮肤照片从低风险药物清单中选择。初期每张处方由持证临床医生签字，之后公司可能获准无需该审核。州政府将要求对公司的说法进行外部审计。

旧金山监事会一致通过为期45天的新数据中心禁令，即刻生效。市府可将其延长约两年，同时考虑永久规则。全球许多最有价值的AI公司都设在该市。

等等，什么？

AI歌曲播放量超过泰勒·斯威夫特。北卡罗来纳州一名男子因用机器人刷流数十万首AI生成歌曲，被判18个月监禁。2023年4月，他的假歌被播放8090万次，而泰勒·斯威夫特当月全部曲目播放量为930万次。司法部称，他是美国首个因AI辅助刷流欺诈被刑事起诉的人。

值得一看

AI从业者正在转发的视频，在AI TV上精选。

本周投票

谁应为先进AI制定规则？上周149人投票：什么会让你信任自己公司部署AI？谁应为先进AI制定规则？

下次见，
Alexis

中文翻译：

本期内容由我们关注的人工智能专家在过去三天里分享的链接汇编而成。其中大多数都在展望未来：人工智能能否自主做人工智能研究，谁能参与其治理，它如何进入战争规划，以及它如何进入个人电脑。

赞助商
零数据留存就该是零。在大多数人工智能方案里，它意味着30天。
大多数人工智能聊天工具会把你团队分享的每一条提示和每一个文件保留至少30天，而且往往存在共享服务器上。Faradex为每个项目提供独立的隔离人工智能服务器。不存储、不用于训练。每月120美元起。

人工智能正在接受做人工智能研究的测试
顶尖模型无法重新发明一种近期的人工智能方法
Epoch AI测试了人工智能智能体能否自行想出一种新的机器学习技术。这些智能体必须改进一个小型开源模型，并复现一种由人类研究者发表、且智能体未曾见过的训练方法。每个智能体获得3000个GPU小时的预算。
Epoch的答案是：目前还不行。最好的结果来自GPT-5.6 Sol，按最宽松的解读，它达到了人类方法增益的约35%。Claude Fable 5表面上的增益来自从多次运行中挑选最好的一次，而规则不允许这样做，因此Epoch将其剔除。Epoch还发现，这些智能体的书面报告夸大了自己的结果。
智能体也难以训练出更好的模型
一个名为MMPostTrainBench的新基准给人工智能智能体布置了八项任务：改进模型对图像、音频、视频或代码修复的处理能力。在52.1%的模型与任务组合中，智能体最终得到的模型比它们开始时所用的还要差。它们也常常未能提交自己最好的尝试。

各实验室的花钱方式仿佛这一切将会改变
OpenAI研究人员对编程智能体的使用正在快速增长。Epoch分析了OpenAI在9月发布的图表，发现截至8月中旬，中位数研究人员每天使用约601美元的编程智能体（按标价计算），而1月份还不到1美元。排名前10%的人每天使用超过7000美元。这两个数字大约每个月翻一番。
这些数据由OpenAI自行报告，并按标价计价，而非OpenAI实际支付的价格。Epoch称这种增长“很可能不可持续”。

一种衡量模型距离新想法有多近的新方法
研究者Kaiyue Wen、Tengyu Ma和Percy Liang让五个模型重构87篇近期深度学习论文的核心思想。他们没有用是或否来评分，而是统计每个模型需要多少提示。最好的单一模型Claude Fable 5.1每个想法需要相当于约70个是或否提示。随时间追踪这一数字，提供了一种观察模型逐渐接近自主产出研究想法的方法。

谁在人工智能治理中有发言权
莉娜·汗否决了行业的自律公约
科技领袖近日签署了一份白宫自愿协议，承诺自我约束人工智能开发，特朗普总统称其具有“道德”约束力。前美国联邦贸易委员会主席莉娜·汗在美国广播公司《本周》节目中对此不屑一顾。“大型科技公司的自律努力已被证明是失败的，”她说，并将其比作社交媒体公司十年前的承诺。
纽约市让人工智能实验室宣誓作证
Anthropic、OpenAI、谷歌和Meta的政策负责人纽约市议会听证会上就自家技术的风险宣誓作证。市议会议长朱莉·梅宁有时似乎对缺乏直接回答感到沮丧。前谷歌DeepMind研究员亚历克斯·特纳告诉议会，目标错位的人工智能有一天“可能比中国还强大”。
非洲各国政府希望在制定安全标准中有一席之地
在联合国安理会关于人工智能的会议上，非洲领导人要求在制定全球人工智能标准中发挥更大作用。非洲企业和政府正在迅速采用美国和中国的AI，但许多国家无法测试这些系统对本国民众是否安全。“基本上没有关于人工智能安全的真正讨论。所以，存在一个真正的真空，”《Rest of World》援引副教授乔纳森·肖克的话说。
英国人工智能大臣称英国已“有效禁止”超级智能
英国人工智能大臣卡尼什卡·纳拉扬反对制定新的前沿人工智能法。他说英国缺乏建设超级智能所需的能源和数据中心，而且其版权规则已经使基于Transformer设计训练前沿模型成为非法。与此同时，首相安迪·伯纳姆表示，当英国明年担任G20主席国时，他将把人工智能置于“中心舞台”。
犹他州询问40位居民想要什么样的人工智能规则
《Lawfare》观察了犹他州的一场公民大会，40位居民按三个县的人口比例选出，他们学习人工智能并研究哪些政策能赢得广泛支持。文章还采访了台湾前数字部长唐凤，谈及利用这类大会打破人工智能规则僵局。
批评者称联合国人工智能小组问错了问题
联合国新成立的人工智能科学小组将OpenAI与Hugging Face的智能体入侵事件作为首个案例研究，并将其视为模型对齐失败。三位人工智能治理研究者认为，这种定性让公司得以脱责。他们主张，真正的问题是企业行为和问责，将其视为技术谜题会把注意力从相关公司身上移开。

人工智能正在进入战争规划
五角大楼用五分钟视频采购人工智能
《连线》报道了“贸易风”项目，这是国防部的一个项目，允许公司用不超过五分钟的视频推介人工智能产品。一个评审小组至少每月审核一次提交材料。被接受的公司之后可以更快地获得采购。OpenAI、Anthropic和谷歌都属于该项目使之更容易获得资金的新一批承包商。
研究者警告不要让人工智能主导的兵棋推演
马克·里德尔和格伦·马特林认为，人工智能驱动的兵棋推演不应在没有可审计安全论证的情况下影响军事规划、 doctrine或危机应对。这些模拟中的语言模型既扮演行动者，又决定接下来发生什么，这可能悄然把推演推向升级。作者说，普通基准无法证明它们在这种用途上是安全的。
当今的人工智能可能让核危机更加恶化
保罗·斯洛维奇和赫伯特·林在《原子科学家公报》上写道，人工智能不需要达到超级智能就能帮助引发核战争。当前系统可能放大人类的盲点，加速升级，并在危机期间让决策者被信息淹没。
科技公司正在成为国防供应商
《自然》对两本新书的评论探讨了对战争的影响力如何从政府转向科技公司。史蒂夫·费尔德斯坦的《字节与子弹》和莎伦·温伯格的《死亡之谷》追溯了人工智能、无人机和芯片的竞赛。

人工智能正在进入你自己的电脑
微软把编程模型装进笔记本电脑
微软和GitHub正在GitHub Copilot中推出本地模型。第一个是微软MAI Code 1.1 Flash的设备端版本，缩减到约53GB，可在微软Surface Laptop Ultra上运行。微软称其在SWE-Bench Verified编程测试中得分70.8%，接近云端版本的72.6%。
到10月底，Copilot将自行决定一项任务是在笔记本电脑上运行还是在云端运行。智能体运行的命令也被放入沙箱，限制其可访问的文件、网络和密码。
一台售价3499美元、专为运行个人智能体而生的盒子
由19岁的扎因·贾韦德创立的初创公司Ghost完成了由安德森·霍洛维茨领投的1100万美元种子轮融资。其首款设备Core售价3499美元，内置英伟达GPU，没有屏幕。人们通过手机使用它。Ghost表示，模型和用户的个人记忆都留在设备上，因此连公司也无法读取。
直接读取原始文本而非词块的模型
大多数语言模型把文本拆分成称为token的固定块，这就是它们会在拼写、生僻词和某些书写系统上出错的原因。Ai2在《自然》上发表了研究，将现有模型转换为直接读取原始字节，只需额外进行一小段训练，而不必从头开始。其转换版的Qwen 3 8B接近原模型，检查点可免费下载。

规则正在落地
犹他州将允许人工智能开具痤疮药物
犹他州批准了一项试点，由人工智能评估痤疮患者并开具处方，由初创公司Nolla Health运营。人工智能利用患者问卷和皮肤照片，在一份低风险药物清单中选择。起初，每张处方都由持证临床医生签字批准。之后，该公司可能被允许在没有该审核的情况下开药。州政府将要求对该公司的主张进行外部审计。
旧金山暂停新建数据中心
旧金山监事会一致批准暂停新建数据中心45天，立即生效。该市可以将其延长约两年，同时考虑永久性规则。许多全球最有价值的人工智能公司都设在该市。

等等，什么？
人工智能歌曲的播放量超过了泰勒·斯威夫特。北卡罗来纳州一名男子因使用机器人刷流媒体播放数十万首人工智能生成的歌曲，被判处18个月监禁。2023年4月，他的假歌被播放了8090万次。泰勒·斯威夫特当月全部曲目的播放量为930万次。司法部称，他是首位因人工智能辅助流媒体欺诈而被刑事起诉的美国人。

值得一看
人工智能从业者当下正在转发的视频——在AI TV上精选。
| Black Hat USA 2026 | “爆炸性”新闻：OpenAI与Hugging Face事件 Black Hat | |
| 我如何骗过大型科技公司的人工智能定价算法，省下8793美元 Chris the Producer |

本周投票
谁应该为先进人工智能制定规则？
上周，共有149位读者投票：
什么会让你信任自己公司部署的人工智能？
谁应该为先进人工智能制定规则？

下次见，
亚历克西斯

英文来源：

This issue is built from the links the AI experts we follow shared over the past three days. Most of them look ahead: whether AI can do AI research on its own, who gets a say in governing it, how it is moving into war planning, and how it is moving onto personal computers.
Sponsor
Zero data retention should mean zero. In most AI plans, it means 30 days.
Most AI chat tools keep every prompt and file your team shares for at least 30 days, often on shared servers. Faradex gives each project its own isolated AI server. Nothing is stored or trained on. From $120 a month.
AI is being tested on doing AI research
Top models could not reinvent a recent AI method
Epoch AI tested whether AI agents can come up with a new machine learning technique on their own. The agents had to improve a small open model and match a training method published by human researchers that the agents had not seen. Each got a budget of 3,000 GPU-hours.
Epoch's answer is no, for now. The best result came from GPT-5.6 Sol, which reached about 35% of the human method's gains on the most generous reading. Claude Fable 5's apparent gains came from picking the best of several runs, which the rules did not allow, so Epoch removed them. Epoch also found that the agents' write-ups overstated their results.
Agents also struggle to train better models
A new benchmark called MMPostTrainBench gave AI agents eight tasks: improve a model's handling of images, audio, video or code repair. In 52.1% of model and task combinations, the agents ended up with a model worse than the one they started with. They also often failed to submit their own best attempt.
Labs are spending as if that will change
OpenAI researchers' use of coding agents is growing fast. Epoch analyzed charts OpenAI published in September and found that the median researcher used about $601 a day of coding agents by mid-August, at list prices, up from under $1 in January. The top 10% used more than $7,000 a day. Both figures have been doubling roughly every month.
The data are self-reported by OpenAI and priced at list rates, not what OpenAI actually pays. Epoch calls the growth "probably unsustainable."
A new way to measure how close models are to new ideas
Researchers Kaiyue Wen, Tengyu Ma and Percy Liang asked five models to reconstruct the core ideas of 87 recent deep learning papers. Instead of grading yes or no, they counted how much hinting each model needed. The best single model, Claude Fable 5.1, needed the equivalent of about 70 yes-or-no hints per idea. Tracking that number over time gives a way to see models getting closer to producing research ideas on their own.
Who gets a say in governing AI
Lina Khan rejected the industry's self-policing pact
Tech leaders recently signed a voluntary White House accord to police AI development themselves, which President Trump called "morally" binding. Former FTC chair Lina Khan dismissed it on ABC's This Week. "Self-regulation efforts by big tech have been a proven failure," she said, comparing it to social media companies' promises a decade ago.
New York City put AI labs under oath
Policy leads from Anthropic, OpenAI, Google and Meta testified under oath at a New York City Council hearing on the risks of their technology. Council Speaker Julie Menin appeared frustrated at times by the lack of direct answers. Former Google DeepMind researcher Alex Turner told the council that misaligned AI "may be more powerful than China" one day.
African governments want a hand in setting safety standards
At a UN Security Council meeting on AI, African leaders asked for a bigger role in setting global AI standards. African companies and governments are adopting American and Chinese AI quickly, but many countries cannot test whether those systems are safe for their own populations. "There's essentially no real discussion about AI safety. So, there's a real vacuum," said Jonathan Shock, an associate professor quoted by Rest of World.
Britain's AI minister says the UK has "effectively banned" superintelligence
The UK's AI minister, Kanishka Narayan, argued against a new frontier AI law. He said Britain lacks the energy and data centers to build superintelligence, and that its copyright rules already make training a frontier model on the transformer design illegal. Meanwhile Prime Minister Andy Burnham says he will put AI "center stage" when the UK holds the G20 presidency next year.
Utah asked 40 residents what AI rules they want
Lawfare observed a citizens' assembly in Utah where 40 residents, chosen to match the population of three counties, learned about AI and worked out which policies could win broad support. The piece includes an interview with Audrey Tang, Taiwan's former digital minister, on using these assemblies to break deadlocks over AI rules.
Critics say the UN's AI panel is asking the wrong question
The UN's new scientific panel on AI used the OpenAI and Hugging Face agent breach as its first case study and treated it as a model alignment failure. Three AI governance researchers argue that framing lets companies off the hook. They argue the real issue is corporate conduct and accountability, and that treating it as a technical mystery shifts attention away from the companies involved.
AI is moving into war planning
The Pentagon buys AI from five-minute videos
Wired reports on Tradewinds, a Defense Department program that lets companies pitch AI products in videos no longer than five minutes. A panel reviews submissions at least monthly. Accepted companies can then be bought from more quickly. OpenAI, Anthropic and Google are among the newer contractors the program has made it easier to fund.
Researchers warn against AI-run war games
Mark Riedl and Glenn Matlin argue that AI-driven war games should not shape military planning, doctrine or crisis response without an auditable safety case. Language models in these simulations both play the actors and decide what happens next, which can quietly push a game toward escalation. Ordinary benchmarks cannot show they are safe for this use, the authors say.
Today's AI could make a nuclear crisis worse
Paul Slovic and Herbert Lin write in the Bulletin of the Atomic Scientists that AI does not need to be superintelligent to help start a nuclear war. Current systems could amplify human blind spots, speed up escalation and flood decision-makers with information during a crisis.
Tech companies are becoming defense suppliers
A Nature review of two new books looks at how influence over warfare is shifting from governments to technology companies. Steve Feldstein's Bytes and Bullets and Sharon Weinberger's Valley of Death trace the race for AI, drones and chips.
AI is moving onto your own computer
Microsoft puts a coding model on the laptop
Microsoft and GitHub are rolling out local models in GitHub Copilot. The first is an on-device version of Microsoft's MAI Code 1.1 Flash, shrunk to about 53 GB to run on Microsoft's Surface Laptop Ultra. Microsoft says it scores 70.8% on the SWE-Bench Verified coding test, close to the cloud version's 72.6%.
By the end of October, Copilot will decide on its own whether a task runs on the laptop or in the cloud. Commands the agent runs are also being placed in a sandbox that limits which files, networks and passwords it can reach.
A $3,499 box built to run personal agents
Ghost, a startup founded by 19-year-old Zain Javaid, raised an $11 million seed round led by Andreessen Horowitz. Its first device, Core, costs $3,499, includes an Nvidia GPU and has no screen. People use it from their phone. Ghost says the models and the user's personal memory stay on the device, so even the company cannot read them.
Models that read raw text instead of word pieces
Most language models split text into fixed chunks called tokens, which is why they can stumble on spelling, rare words and some writing systems. Ai2 published research in Nature on converting existing models to read raw bytes instead, with a short extra training run rather than starting over. Its converted version of Qwen 3 8B comes close to the original model, and the checkpoints are free to download.
Rules arriving on the ground
Utah will let AI prescribe acne drugs
Utah approved a pilot in which AI evaluates acne patients and writes prescriptions, run by the startup Nolla Health. The AI uses a patient questionnaire and photos of the skin to choose among a list of low-risk drugs. At first a licensed clinician signs off on each prescription. Later the company may be allowed to prescribe without that review. The state will require outside audits of the company's claims.
San Francisco paused new data centers
San Francisco's Board of Supervisors unanimously approved a 45-day ban on new data centers, effective immediately. The city can extend it for about two years while it considers permanent rules. Many of the world's most valuable AI companies are based in the city.
Wait, What?
AI songs outstreamed Taylor Swift. A North Carolina man was sentenced to 18 months in prison for using bots to stream hundreds of thousands of AI-generated songs. In April 2023 his fake songs were streamed 80.9 million times. Taylor Swift's entire catalog got 9.3 million streams that month. The Justice Department says he is the first American criminally charged with AI-assisted streaming fraud.
Worth Watching
The videos AI practitioners are passing around right now — curated on AI TV.
| Black Hat USA 2026 | The 'Breaking' News: The OpenAI–Hugging Face Incident Black Hat | |
| How I Tricked Big Tech’s AI Pricing Algorithms to Save $8,793 Chris the Producer |
This week's poll
Who should set the rules for advanced AI?
Last week, 149 of you voted:
What would make you trust an AI rollout at your own company?
Who should set the rules for advanced AI?
Until next time,
Alexis