我在一只追逐我猫的虚拟鸭子身上看到了机器人的未来

qimuai 发布于 阅读:41 一手编译

我在一只追逐我猫的虚拟鸭子身上看到了机器人的未来

内容来源:https://www.wired.com/story/i-trained-a-microduck-to-be-my-cats-butler/

内容总结:

AI技术正大幅降低机器人开发门槛。开发者借助OpenAI的GPT-6 Sol等最新模型,仅用数小时便完成了虚拟机器狗的编程与训练,使其能识别孟加拉猫并规划递送零食的动作。这一效率在以往可能需要数周时间。OpenAI最新模型在理解三维物理世界、将其转化为代码及机器人训练指令方面表现出色。

在学术界,加州大学伯克利分校教授Ken Goldberg的团队发现,OpenAI的Astra模型能在不到一小时内根据真实机器人视频生成包含相机位置、光照、材质等全部参数的高精度三维仿真场景,并可自动完成大量繁琐的训练实验与结果分析工作。英伟达研究员Jim Fan与Goldberg等人本周开源了名为Aspire的机器人智能体框架,使机器人能够自主发现并学习新技能,例如用双臂解开魔方。Fan在奥斯汀举行的机器人学习会议上展示,Astra控制机械臂完成了绘制金门大桥的任务。

尽管让机器人稳定高效运行仍需大量工程专业能力,但AI辅助训练机器人的便捷与速度有望对制造业和电商产生重大影响。已有工业机器人与自动化企业对此表现出兴趣。随着爱好者与工程师意识到编程和训练机器人变得如此简单,“氛围机器人”时代或将到来。

中文翻译:

请允许我向你介绍我借助AI开发的机器猫管家。
利用OpenAI的GPT-6 Sol,我教会了一个虚拟版的Microduck——一个来自Pollen Robotics和Hugging Face的可爱机器人——在房间里走动,并识别附近是否有孟加拉猫。(上方视频片段显示机器人摇摇晃晃地走向目标猫咪;它的第一视角画面显示在右上角。)
我还在耐心等待一台实体Microduck——拜托快点发货吧,各位——但模拟环境中的行为相对简单,应该可以迁移到实物上。我计划让机器人自动为我的孟加拉猫Leona投放零食,比如通过低头从3D打印的头戴式储粮仓中释放一颗零食。
我只用了几个小时就完成了虚拟机器人的编程和训练,说实话,这简直不可思议。以前,这种事情可能要花我好几周才能搞明白。之所以能做到,是因为OpenAI的最新模型非常擅长理解和操控物理世界中的3D表征,并将其转化为有用的代码、智能体行为和机器人训练方案。
真正的机器人专家正在利用最新模型做出一些令人惊叹的成果。去年9月OpenAI发布Astra时,加州大学伯克利分校教授Ken Goldberg和他的学生们看到了展示其3D图像渲染能力的演示。他们想知道它能否帮助完成训练各类机器人所需的仿真工作。这些仿真需要极其精细且高度精确,才能确保复杂行为——比如会跳舞的人形机器人或能抓握的机械手——忠实地迁移到现实世界中。
Goldberg实验室的一名成员让Astra接入了一款名为Newton的英伟达工具,给它看了一段真实机器人运行的视频片段,然后问它能否创建该场景的仿真。
“让我们所有人都震惊的是,不到一个小时它就生成了一个完整的3D仿真,所有参数都设置好了——相机位置、光照、背景、所有材质,”Goldberg告诉我。
Goldberg说,Astra还能通过运行实验并分析结果,自动完成仿真训练机器人中大量枯燥繁琐的工作。
这正是我在训练虚拟Microduck时的切身体验。除了GPT-6 Sol,我还使用了Nous Research的智能体框架Hermes和英伟达DGX Spark——一台搭载强大AI芯片的个人电脑。
我之前写过关于使用智能体和编程模型来配置、编程和调试实体机器人的文章。通常,这类工作需要相当的专业知识;现代编程模型让这件事变得轻而易举。但OpenAI的最新模型远不止于写代码。
它使用了一款名为Isaac Sim的英伟达软件,向控制虚拟机器人的模型发送指令,观看其运行视频以评估效果,并对其训练方案提出调整建议。
这个过程让我成功训练Microduck行走而不摔倒。我最初尝试让GPT-6 Sol教机器人走太空步——我给它喂了一段迈克尔·杰克逊在舞台上滑行的GIF,它试图模仿——但经过数小时的努力,只练成了一种非常轻快的后退小跑。(机器人还没有完全打败我们。)
与Goldberg合作过的英伟达研究员Jim Fan表示,AI现在控制机器人的方式与软件智能体的运作方式如出一辙。大语言模型可以使用经过训练来控制物理动作的模型——即视觉语言动作(VLA)模型——就像它们在电脑上访问不同软件一样。
本周,Fan、Goldberg等人开源了一个名为Aspire的机器人智能体框架,它能让机器人发现并使用新技能。一个例子是:GPT-6 Astra学会了如何用一对机械臂还原魔方。
“它拿起魔方,尝试扭转,但角度不对,”Fan说。“然后它进行了非常非常长时间的推理,但最终能够解开,因为它随着时间的推移学会了这些技能。”
本周在得克萨斯州奥斯汀举行的机器人学习会议(CoRL)上,Fan发表了主题演讲,谈到了AI模型能够自动化大量让机器人完成复杂任务所需工作的惊人发现。Fan展示了一个演示:Astra被赋予控制一条机械臂的权限,并被要求画一幅金门大桥的画。它利用附近的颜料和画笔,顺利完成了任务。
当然,让机器人可靠高效地工作仍然极其困难,需要大量的工程专业知识。我那台机器鸭的视觉系统还不足以部署到真实产品中。
不过,用AI训练机器人的便捷性和速度可能很快就会对制造业和电子商务产生巨大影响,Goldberg说已有不少工业机器人和自动化公司对这一方法表现出了兴趣。
就像几年前的“氛围编程”一样,随着爱好者和工程师意识到编程和训练机器人有多么简单,我们很可能会看到“氛围机器人”领域的突破。而且,假设我能完善我的机器人助手,Leona或许再也不用断零食了。
本文出自Will Knight的“AI Lab”新闻通讯。点击此处阅读往期通讯。
评论
返回顶部

英文来源:

Allow me to introduce you to the robot cat butler I’m developing with help from AI.
Using OpenAI’s GPT-6 Sol, I taught a virtual version of Microduck—a cute robot from Pollen Robotics and Hugging Face—to walk around a room and identify any bengal cats in its vicinity. (The clip above shows the robot waddling toward a target feline; its eye view is shown top right.)
I’m still waiting patiently for a physical Microduck—please hurry up and ship it, guys—but simulated behavior is relatively simple, so it should transfer over to the real thing. I plan to have the robot automatically dispense a treat for my pet bengal, Leona, perhaps by bowing its head to release one from a 3D-printed, head-mounted storage chamber.
I was able to code and train the virtual robot in just a few hours, which is, quite frankly, incredible. Previously, this kind of thing might have taken me weeks to figure out. It was possible because OpenAI’s latest models are remarkably good at understanding and manipulating 3D representations of the physical world and then translating that into useful code, agentic actions, and robot training.
Real roboticists are using the latest models to do some amazing stuff. When OpenAI released Astra this past September, University of California, Berkeley professor Ken Goldberg and his students saw demos that showcased its ability to render 3D images. They wondered if it could help with the simulations required to train different kinds of robots. These simulations need to be incredibly detailed and extremely accurate to ensure that sophisticated behavior—a humanoid that can dance or a robot hand that can grasp—will transfer faithfully to the real world.
One member of Goldberg’s lab gave Astra access to an Nvidia tool called Newton, showed it a video clip of a real robot in action, and asked if it could create a simulation of the scene.
“What blew us all away was that within an hour it had produced a full 3D simulation with all the parameters set—the position of the camera, the lighting, the background, all the materials,” Goldberg tells me.
Astra is also able to automate much of the drudgery involved with training a robot in simulation, Goldberg says, by running experiments and examining the results.
This is precisely what I found while training the virtual Microduck. Along with GPT-6 Sol, I used an agentic harness from Nous Research called Hermes and a Nvidia DGX Spark, a personal computer that contains a powerful AI chip.
I’ve previously written about using agents and coding models to configure, program, and debug physical robots. Normally, this kind of work requires significant expertise; modern coding models make it a cinch. But OpenAI’s newest model went far beyond just writing code.
It used a piece of Nvidia software called Isaac Sim to feed instructions to a model that controlled the virtual robot, watch videos of it in action to assess how well things were working, and suggest tweaks to its training regime.
This process allowed me to train the Microduck to walk around without toppling over. I initially tried getting GPT-6 Sol to teach the robot to moonwalk—I fed it a GIF of Michael Jackson sliding across the stage, which it tried to copy—but after hours of effort, it only resulted in a very jaunty backward jog. (The robots don’t have us totally beat quite yet.)
Jim Fan, a researcher at Nvidia who has collaborated with Goldberg, says the way AI can now control robots mirrors how software agents function. LLMS can use models trained to control physical actions—visual language action (VLA) models—just as they access different software on a computer.
This week, Fan, Goldberg, and others open sourced an agentic framework for robots called Aspire, which allows a robot to discover and use new skills. One example: GPT-6 Astra figured out how to use a pair of robot arms to solve a Rubik’s cube.
“It picks it up, tries to twist it, but has the wrong angle,” Fan says. “Then it reasons for a very, very long time, but it eventually is able to solve it because it learns the skills over time.”
At the Conference on Robot Learning (CoRL), held in Austin, Texas, this week, Fan gave a keynote speech in which he talked about the incredible realization that AI models can automate a lot of the work needed to have a robot do sophisticated things. Fan showed off a demo in which Astra was put in control of a robot arm and told to paint a picture of the Golden Gate Bridge. Using some paint and brushes nearby, it duly completed the task.
Of course, getting robots to work reliably and efficiently is still extremely difficult, requiring significant engineering expertise. The vision system on my robot duck does not work well enough to be deployed in a real product.
Still, the ease and speed of training robots with AI could soon have an enormous impact on manufacturing and e-commerce, and Goldberg says a number of industrial robotics and automation firms have shown an interest in the approach.
Much like vibe coding a few years ago, we may well see breakthroughs in “vibe robotics” as hobbyists and engineers realize how easy it is to program and train robots. And, assuming I can perfect my robot helper, Leona may never have to go without treats again.
This is an edition of Will Knight’s AI Lab newsletter. Read previous newsletters here.
Comments
Back to top

连线杂志AI最前沿

文章目录


    扫描二维码,在手机上阅读