脑植入物如何将思想转化为语言

qimuai 发布于 阅读:28 一手编译

脑植入物如何将思想转化为语言

内容来源:https://www.sciencenews.org/article/brain-implants-thoughts-speech-podcast

内容总结:

渐冻症患者借助脑机接口重建沟通能力。凯西·哈雷尔因肌萎缩侧索硬化症(ALS)失去说话能力后,作为临床试验志愿者参与测试一种脑机接口系统。植入其大脑的电极阵列将神经信号传送至附近计算机,再由计算机将这些信号翻译成他想要表达的词语。凯西每天使用这一系统与妻子、女儿、朋友和同事交流。他回忆首次意识到系统奏效时的心情是“纯粹的喜悦”,尽管当时研究人员预先设定了测试语句,他仍称那是“一个意义深远的时刻”,“意识到近在眼前的种种可能性,对我和全人类而言都令人无比振奋”。

这期《The Deep End: Talk to Me》节目追踪了凯西以及让他得以开口说话的快速发展的神经科学。节目第四集文字记录显示,颈部以下瘫痪的患者已能用思维操控机械臂;视频中的女性正用思维控制机器人。所谓BrainGate系统依靠植入大脑运动皮层的电极传感器工作。研究人员希望,通过在大脑植入电极读取负责说话肌肉运动的脑信号,将人的说话意图转化为计算机生成的可理解语音。

莱拉·桑德斯:用思维控制事物听起来像科幻小说,但确确实实是真的。今天我们将探讨用脑植入设备解码思维并转化为语言进行交流是什么体验。[录音]“嘿妈妈,我能跟你说件事吗?”声音绝对是身份不可或缺的一部分。“你好?”“加上‘是你’这一层。”“你想说话吗?”“你的声音。”“嗨。”“你的声音就是你。”桑德斯:我是《科学新闻》的莱拉·桑德斯,这里是《The Deep End: Talk to Me》。那么BrainGate是什么?它是一个实验系统,将植入人脑的电极与附近计算机连接,称为脑机接口(BCI)。目标很宏大:帮助瘫痪、脑损伤或肢体缺失者重新掌控自己的世界。

利·霍赫伯格:BrainGate临床试验至今已进行近22年,目前是21年多一点。桑德斯:利·霍赫伯格是BrainGate联盟及临床试验主任,同时在麻省总医院、布朗大学和退伍军人事务部任职。霍赫伯格:对于因脑干中风突然丧失运动和说话能力的人,或逐渐丧失运动和说话能力的ALS患者,人们长期希望特别是可植入式脑机接口能恢复沟通和行动能力。桑德斯:2004年,BrainGate设备首次植入人脑,受试者是在事故中瘫痪的马修·纳格尔。借助BrainGate,他能用思维控制电脑光标。以下是IT World 2008年的报道:技术给了他希望。他是测试BrainGate的四名受试者之一,该技术将芯片植入大脑拾取电脉冲,计算机再将脉冲解读为动作。霍赫伯格:早期研究非常令人鼓舞,说明了该领域的潜力。桑德斯:多年来有许多令人惊叹的案例:人们用它浏览互联网、玩电子游戏,一位瘫痪女性甚至能用机械臂喝肉桂拿铁。霍赫伯格:她能够想象使用自己的手臂,伸出机械臂拿起保温杯送到嘴边,倾斜杯子,把吸管放入口中喝一口。虽然这不是她近15年来第一次喝东西,却是第一次完全出于自主意愿做到。我记得她当时的笑容。桑德斯:几十年来BrainGate在多方面取得大量研究成功。但说话呢?霍赫伯格说,解码语音的科学仍非常非常年轻。霍赫伯格:要非常明确地说,我们还处于早期,这是该领域的开端。世界上只有极少数人植入过任何类型的能解码语音的脑机接口。桑德斯:在解码说话意图方面科学进展如何?前几集我们听到了失去说话能力者的故事,其中一位是凯西·哈雷尔,他大脑中也植入了BrainGate设备。凯西·哈雷尔:我叫凯西·哈雷尔,47岁。桑德斯:凯西早晨醒来后插上设备。凯西:起床后我通常做的第一件事就是接入“矩阵”,打个比方。桑德斯:他指的是将两根线连接到头顶伸出的两个基座上,两根黑色线缆蜿蜒连接到附近计算机。凯西大脑中的电极、头上线缆和计算机合称BrainGate 2神经接口系统。凯西启动系统后就开始使用。凯西:除非有医疗预约等需要出门的事,我大部分时间都坐在这里,中间短暂休息。桑德斯:全程凯西在想着要说的词时产生脑信号,这是他的神经元特别是言语运动皮层细胞发出的微小精确电脉冲。记得我们说过说话涉及100多块肌肉吗?这些就是控制那些肌肉的脑区。凯西:过去近两年我每周7天、每天约10到14小时使用解码器,是世界上使用脑机接口时间最长的重度用户。桑德斯:和朱尔斯一样,凯西患有ALS,疾病逐渐夺走了他的运动和说话能力。凯西:2020年确诊后不久我开始失去说话能力,然后像慢动作车祸一样持续多年,直到几乎没人能听懂我。手术时只有我的助手和妻子莱瓦娜能完全听懂我,少数常在身边的朋友能听懂部分。我依赖助手和妻子充当翻译,没有她们我哪儿都不去,连视频通话都不行。桑德斯:2023年7月,凯西自愿参加名为BrainGate 2的临床试验。加州大学戴维斯分校神经外科医生将四个小型电极阵列植入凯西大脑左侧。我想了解更多,于是采访了为凯西植入设备的神经外科医生。大卫·布兰德曼:我叫大卫·布兰德曼,是加州大学戴维斯分校神经假肢实验室联合主任,开发下一代神经技术帮助瘫痪者。脑机接口是一种记录瘫痪者脑信号并将其转化为控制环境物体的设备。桑德斯:凯西的植入物由四个阵列组成,每个方形约胡椒粒大小,每个阵列有许多尖端伸入大脑,看起来有点像金属猫梳。四个阵列共含256个电极,全都在凯西想着说词时监听其大脑活动。布兰德曼:该设备的工作原理是记录他试图说话时大脑的电活动。桑德斯:凯西头上的线缆将神经信号传到附近计算机,机器学习算法在此进行复杂的匹配,将凯西的脑活动模式与特定语音(音素)对应的可预测模式匹配。布兰德曼:我们将他的脑信号翻译成39种英语音素之一。音素是语言的基本构件。比如“hello”可拆成“ha、uh、uhl、oh”。桑德斯:接下来算法将这些音素组装成词,尽力猜测凯西想说什么。布兰德曼:我把脑活动翻译成音素,再用人工智能将音素变成词、词变成句、句变成段。桑德斯:这就是流程,让凯西能在Slack上打字、给朋友发短信、发邮件。有时他给我的邮件底部有标签:“发自我的大脑”,配一个可爱的粉色大脑表情。他的设备还能将打字信息转为语音,文字显示在屏幕上供他校对,确认无误后系统大声说出。布兰德曼告诉我,凯西自然说话时每分钟约能说6个可懂词,相当慢。布兰德曼:使用这个假肢,他大约每分钟50词。我现在说话大约每分钟120到150词,取决于兴奋程度。我们让他恢复到健全时说话速度的三分之一左右。桑德斯:采访前几天我给凯西发了问题,但他没用邮件回复,而是用脑机接口实时想着要说的话回答我。自我介绍时我看到了时间尺度被拉长。凯西:这很有挑战,因为我们可以对话,但即使在我们长大的地方,我也比普通人慢。桑德斯:凯西和我都成长于南方,他在亚特兰大地区,我在田纳西中部。但他在湾区住了20多年,口音和我一样淡了。凯西:是的,我没多少南方口音了。但我爱我的根,很想带女儿看看我成长的地方。但无法行动时出行极其困难。桑德斯:行动受限前凯西做过很多酷事。凯西:我大部分职业生涯是气候活动人士,走遍全球,与破坏地球的公司斗争,为地球及其居民争取正义。我很幸运能做热爱的事,并与其他杰出人士建立深厚友谊。桑德斯:对凯西的采访与我以往任何采访都不同。我习惯灵活应对时区问题、语言障碍、用符号公式回答问题的数学家。但登录Zoom与凯西交谈时,我很快意识到这次不同。开始时我很不自在,手忙脚乱,问了问题又在他回答前问下一个,几次不小心打断他。有时不确定他是在组织答案还是在等我说话。我表现不佳对凯西来说并不罕见。凯西:处于你这种位置的人很难保持专注,因为你不习惯这么慢的对话节奏。桑德斯:他说得对,我不习惯。对话和生活中时机就是一切。以下是UCLA的格雷格·布莱恩特。布莱恩特:对话轮次最长以数百毫秒计,200毫秒就算长轮次,即五分之一秒,非常快。所以只要你慢一点就很明显。我们必须协调说话节奏,不能太慢也不能太快回应,必须知道何时停止和开始。我们进入一种流动的动态,这对我们对对话、对方和自己的感受影响巨大。一切都是时机。桑德斯:再说朱尔斯。他的话也始于大脑,但走不同路径:他通过注视屏幕上的字母打字,然后语音克隆说出。过程也慢且耗大量脑力。对话中MA描述了朱尔斯需要付出的努力。MA:你可以想象他大脑里做着各种脑力体操,现在他听着我们,用眼睛逐字母写字来参与对话,还要处理他预先写好的对你问题的回答。幕后发生很多事,不必要想自己声音的人理所当然。我暂停让他说完。桑德斯:MA和我等朱尔斯发言。你已听到10秒停顿,完整停顿为1分18秒。朱尔斯:与不每天互动的人交流很有挑战。我从想到实际说出有重大瓶颈。桑德斯:和凯西一样,停顿会带来困难。言语病理学家杰西卡·彼得罗夫斯基进一步解释。彼得罗夫斯基:人类对沉默不太舒服,尤其在交流中。我们交谈时会填补沉默以减少尴尬。使用设备交流的人可能很习惯延迟和沉默并感到舒适,而对方却想“继续啊,我想帮忙填补空白,不想尴尬”,结果可能更尴尬。桑德斯:朱尔斯讨厌别人在他准备好之前读屏幕,这催生了他脱口秀的一个笑话。[朱尔斯录音]我打字时人们读我屏幕并试图替我说完,就像不断应付一个过度争胜的你画我猜搭档。桑德斯:朱尔斯和凯西 constantly 遇到别人的不耐烦,虽是好意但无益。更糟的是别人对他们能力的误解。凯西是有趣、风趣、引人入胜的对话者,但不慢下来听他说话的人可能看不出来,这种假设让他很困扰。凯西:医疗系统这样快速对待我,极其有失尊严,后果可怕。桑德斯:布兰德曼也见过这种轻视和不当对待,发生在他参与BrainGate 2项目的志愿者身上,凯西也在该试验中。布兰德曼:作为医疗工作者,我们把无法沟通与缺乏理解混为一谈。我亲眼见医生护士与我的BrainGate参与者互动时把他们当无效者或不存在,因为他们无法表达。事实上我在重症监护室和病房特意对所有人说:他们在里面,他们在那儿,能理解发生的一切,只是不能说话,请给予应有的尊重和尊严。这对参与者是持续挑战,是我们面对的偏见。桑德斯:凯西和其他ALS患者并不孤单。许多说话困难或需要更多时间的人都会被这样评判。以下是Bridging Voice的卡琳娜·纳金。纳金:这关乎别人怎么看你。患ALS后无法行动、没有面部表情、连着机器,人们就认为你不在。我们每天看到人们要么不跟ALS患者说话,假设他们不能回应,要么大声慢速说话。他们的头脑完全正常。桑德斯:目前凯西的话可能慢,但尽管有缺点,仍是奇迹。记得本集开头的霍赫伯格吗?他是与凯西合作的神经科学家之一,他讲了凯西脑机接口植入后最初几天测试的故事。霍赫伯格:第一天我们给他50个词的词汇表,即使第一天他就能用这50个高价值词说出一些关键句子。桑德斯:测试中凯西被要求说预设句子以测试系统。凯西:大多数句子是所谓的闭环。桑德斯:即研究人员已知凯西想说的词对应什么脑活动模式,测试系统是否工作。凯西:解码器能验证我说对还是错,给我简短的预载句子。我被要求说“那有什么好处?”桑德斯:“那有什么好处?”不是最动人的话,但在房间和远程视频参会的研究人员中引起巨大反响。凯西:“那有什么好处?”我成功后大家都在哭。桑德斯:后来凯西才有时间真正表达自己。凯西:头几次会话后我才有点真正时间在相对私密下说想说的话。即便如此,那是纯粹的喜悦,尽管那一刻我没说想说的。对我来说是深刻时刻,意识到近在眼前的种种可能性,对我和全人类都令人无比振奋。桑德斯:之后越来越好。这次会话中霍赫伯格在办公室,凯西在家客厅,霍赫伯格通过屏幕看凯西。霍赫伯格:第二次会话回来,给他完整的12.5万+词英语词典供选择,即解码器和AI语言模型基本可从完整词典中选择。他说的第一句话大概是“这玩意儿开着吗?”挺有趣。第二天脑机接口解码的第二句话是“我在找一只猎豹”。才第二天,我们知道有各种解码错误可能。我看到时想,不可能第二天第二句就说什么猎豹,显然是解码错误,没关系。桑德斯:霍赫伯格能看到屏幕上的凯西,但看不到房间里刚发生的重要事。霍赫伯格:他四岁的女儿刚穿着猎豹服装走进房间。桑德斯:凯西回忆同一时刻。凯西:是的,我记得很清楚。那是2024年8月初第二次说话会话。我女儿穿猎豹装非常可爱,那是她最喜欢的动物之一。他们正处于什么都超快的阶段,从一个东西跑到另一个。像利这样不在房间的人都以为肯定是错误,太随机了。在场的研究人员能证实我确实说了,因独特情境我确实是那个意思。桑德斯:这是许多父母都有的时刻:孩子做可爱的事,父母配合。但对凯西和孩子而言,这一刻绝不寻常。霍赫伯格:她出生时正是他确诊ALS前后。可以想象,计算机传出的声音是他自己声音的版本,其余互动是父女之间、亲子之间的。桑德斯:脑机接口给了凯西无价之物:一分钟与可爱的小猎豹玩耍。凯西:我内心确信它会像最终那样奏效。我做了很多研究,技术正处于突破边缘。我问自己,为什么不是我?桑德斯:研究早期凯西和妻子被告知不要期望成功,这是实验,结果远不确定。但凯西很乐观。霍赫伯格:对我来说,看到它如此快速有力地奏效,真的强调了这些技术不仅能恢复日常生活活动,不仅关键工具性日常活动,还能恢复亲子沟通、朋友家人照护者和医疗提供者之间的沟通。桑德斯:因为这是关于人声的播客,我一直问受访者声音对他们意味着什么。凯西的回答:凯西:意义重大。我认为这是我在这场病中失去的最重要的东西。桑德斯:即使以有限形式找回也意味着一切。凯西:从一开始就能被理解、能说话回应,尤其能回应女儿,太棒了,因为此前我生活中完全缺乏这些。桑德斯:凯西恢复的声音改变了生活。凯西:这是养育女儿的能力,与妻子连接的能力,与朋友说俏皮话的能力。时机就是一切,这就是为什么从无到有完全是革命性的。回看我之前的回答,这确实还能改进。桑德斯:实验初期研究团队告诉凯西夫妇每周只能 casual 聊天几小时。当时觉得是巨大礼物,因为很久无法有意义交谈。布兰德曼:我们每周去他家两三次,情况越来越好,直到凯西想用系统说所有想说的话。桑德斯:早期只有研究人员能为他设置系统、接入“矩阵”。布兰德曼:研究技术员每周来他家三到五次穿戴和取下系统,让他能沟通、过一天。后来我们看到他很想用,能否简化?我们向FDA申请修改试验方案,培训他的照护伙伴在我们不在时穿戴取下系统。此后他几乎每天使用。桑德斯:这与每周几小时天差地别。我读一些布兰德曼和同事2026年6月发表在《自然神经科学》关于凯西脑机接口的论文数字:“近两年,参与者(凯西)在无研究人员在场的情况下在家中使用了BCI超过3800小时,与家人朋友保持丰富人际沟通,独立控制个人电脑,尽管瘫痪仍维持全职工作。他交流了183,060句,共1,960,163词,平均每分钟56词。他标记92%的句子至少大部分被正确解码。”这些数字以精简科学方式呈现显得枯燥,但细想每句、近两百万词都是连接凯西生活的纽带:给女儿的信息、与伴侣的玩笑、给朋友的短信、与妻儿玩Wordle。以下是加州大学戴维斯分校健康中心拍摄的凯西一家视频。[视频]“好,没有A,但有个E。M-I-O-M-I。干得好,团队。我们快到了。”桑德斯:我们谈了很多词以及BCI恢复说话能力的根本方式。但凯西的BCI还更进一步,能加些额外效果。凯西:它首先是通信设备,而非我用来讲笑话的工具。但我也有这本事。比如我能给说的话加音效。桑德斯:例如凯西有悲伤长号音效。[长号声]凯西:如果我觉得某事好笑想说LOL,会播放《宋飞正传》主题曲。[主题曲]桑德斯:就是这个意思。这些玩笑让我想到朱尔斯,于是问凯西是否也能用系统骂人。凯西:回答你能否用这设备骂人,我当然[哔]能。桑德斯:满足好奇的各位。展望未来,很多事在发生,科学发展快,许多公司在做类似系统,我们将在几集里讨论。但目前凯西是孤例,科学家说的N=1,唯一如此重度使用脑机接口沟通的人。凯西:我希望很快改变,因为我觉得每个想要的人都该有机会,尤其是我这种病的人。桑德斯:霍赫伯格告诉我,这种帮助他人的动力在临床试验志愿者中很常见。霍赫伯格:每位参与者都非常慷慨,清楚研究尚早,但告诉我们想帮助其他瘫痪者,让脑干中风、颈髓损伤、ALS或其他瘫痪疾病或损伤者改善功能,重新控制电脑或恢复流畅说话沟通。这种慷慨精神定义了BrainGate研究的参与者。桑德斯:凯西的慷慨和助人动力在谈话中闪耀。但我想回到他本人。我问凯西把这项技术放在家里、大脑里是什么感觉。他告诉我声音多么珍贵,我们多容易忘记。凯西:我们理所当然。跑步时想发泄压力能尖叫,能在伴侣耳边说甜言蜜语。这些是生活的意义。被剥夺时你不惜一切想找回。这就是为什么目前这样够好,我们应该加速批准这类设备。桑德斯:凯西的BCI比正常对话慢,偶尔出错。凯西:抱歉,我在试着回答你,它在改变我想说的话,我固执地不换方式说,现在换。桑德斯:我不急,你尽管固执。桑德斯:它需要联网和电源,别忘了凯西头顶有转接器。但这些都是可解决的问题,科学家正在努力。所以不完美,但如凯西所说,目前够好。恢复声音的科学努力对朱尔斯、凯西等人有变革性。但将声音与身体分离也为社会和生活带来新可能。MA:他的声音会活在视频和我们捕捉的东西里,他有意留给我们的,就这些。桑德斯:下一集我们走出实验室,进入声音可能不似表面的奇怪新数字世界。哈尼·法里德:我们生活在既激动又恐惧的时代,两者并存。桑德斯:声音可轻易克隆的世界会带来什么变化?法里德:今天只需键盘和网络,你就能打字得到任何图像、声音、视频,任何人做任何事说任何话。桑德斯:《The Deep End》是《科学新闻》制作,基于我的原创报道。本集由艾拉·罗文制作和混音,项目经理阿什利·耶格尔,主编南希·舒特,音乐蓝点会话。播客部分由伯勒斯·韦尔科姆基金资助,PRX支持。本播客由伯勒斯·韦尔科姆基金资助并与PRX合作制作。播客标志由尼尔·韦布创作。

中文翻译:

大脑植入物如何将思想转化为语言

凯西·哈雷尔因渐冻症(ALS,即肌萎缩侧索硬化症)失去了声音。作为临床试验的志愿者,他正在帮助科学家测试一种脑机接口。植入凯西大脑的电极阵列将信号发送到附近的计算机,计算机将这些信号转化为他想说的话。

凯西每天都使用这个系统与妻子、女儿、朋友和同事交谈。他意识到它起作用的那个时刻是“纯粹的喜悦”,凯西说。尽管研究人员为第一次测试预设了他的话语,“那仍然是一个深刻的时刻,”他说。“意识到近在眼前的可能性——对我以及全人类而言——令人震撼。”

本期《深渊:跟我说话》跟随凯西以及让他得以开口说话的前沿神经科学。

第4集文字记录

颈部以下瘫痪的患者可以用思想操控机械臂。视频中的这位女性正在用大脑控制一个机器人。所谓的BrainGate系统依赖于植入大脑运动皮层的电极传感器。研究人员希望,通过将电极植入大脑,能够读取那些意图驱动说话相关肌肉的脑信号,并将此人说话的意图转化为由计算机生成的完整语音。

劳拉·桑德斯:用思想控制事物。听起来像科幻小说,但它绝对是真实的。今天,我们将探索与一个能解码你的思想并将其转化为语言的脑植入物进行交流是什么感觉。

[声音]嘿妈妈,我能跟你说件事吗?声音绝对是你身份不可分割的一部分。喂?那一层额外的“是你”的东西。你想说话吗?你的声音。嗨。你的声音就是你。

桑德斯:来自《科学新闻》,我是劳拉·桑德斯,这里是《深渊:跟我说话》。好,那么什么是BrainGate?它是一个实验性系统,将植入人脑的电极与附近的计算机连接起来。它被称为脑机接口,简称BCI。其目标很宏大:帮助瘫痪、脑损伤或肢体缺失的人重新掌控他们的世界。

利·霍克伯格:BrainGate临床试验现在已经进行了近22年。目前是21年多一点。

桑德斯:利·霍克伯格是BrainGate联盟和临床试验的主任。他还在麻省总医院、布朗大学和退伍军人事务部任职。

霍克伯格:对于那些可能因脑干中风而突然失去运动和说话能力的人,或者对于渐冻症患者——他们正在逐渐丧失运动和说话的能力——人们长期以来一直希望,特别是可植入的脑机接口能够恢复交流和恢复行动能力。

桑德斯:2004年,BrainGate设备首次被植入人脑。那个人名叫马修·纳格尔,他在一次事故中瘫痪了。借助BrainGate,他可以用思想控制电脑光标。以下是IT World 2008年的一则报道。

技术给了他希望。他是测试BrainGate的四个人之一,这是一项新技术,将芯片植入大脑以捕捉电脉冲。然后计算机将这些脉冲解读为动作。

霍克伯格:那些早期研究确实令人鼓舞,我认为这说明了该领域的潜力。

桑德斯:多年来,人们使用BrainGate的例子令人惊叹。人们用它来浏览互联网、玩电子游戏,一位瘫痪的女性甚至能够用机械臂喝一杯肉桂拿铁。

霍克伯格:她能够想着使用自己的手臂,伸出机械臂,拿起那个保温杯,送到嘴边,倾斜向嘴巴,然后把吸管放进嘴里,喝下那一口。虽然这不是她近15年来第一次喝东西,但这是她第一次真正完全出于自己意愿做到这件事。我记得她做到时脸上的笑容。

桑德斯:几十年来,BrainGate在很多方面取得了大量研究成功。但说话呢?霍克伯格说,在解码语音方面,这里的科学还非常非常年轻。

霍克伯格:说清楚一点,我们还处于早期阶段。这是这个领域的开端。世界上只有极少数人拥有过任何类型的可植入BCI来解码语音。

桑德斯:在解码说话意图方面,科学处于什么阶段?在过去的几集中,我们听到了那些失去说话能力的人的故事。其中一个人就是凯西·哈雷尔,而凯西恰好大脑中也植入了BrainGate设备。

凯西·哈雷尔:我叫凯西·哈雷尔,今年47岁。

桑德斯:凯西早上醒来,然后插上电。

凯西:我起床后通常做的第一件事就是接入矩阵,可以这么说。

桑德斯:他的意思是,将两根线连接到从头顶伸出的两个基座上。这两根黑色线缆然后蜿蜒向下连接到附近的计算机。凯西大脑中的这些电极、他头部的线缆以及计算机,统称为BrainGate 2神经接口系统。于是凯西把它全部启动,然后就开始了。

凯西:除非我有医疗预约或其他需要出门的事,否则我大部分时间都会坐在这个位置,中间只有短暂的休息。

桑德斯:在整个过程中,凯西在想着说出词语时产生脑信号。这些是凯西的神经元——特别是他言语运动皮层中的细胞——产生的微小而精确的电脉冲。还记得我们了解到有超过100块肌肉参与说话吗?这些就是控制那些肌肉的大脑区域。

凯西:我每天使用解码器大约10到14个小时,每周七天,已经持续了近两年,这使我成为世界上已知最重度使用脑机接口的用户。

桑德斯:和朱尔斯一样,凯西患有渐冻症。这种疾病慢慢夺走了他运动和说话的能力。

凯西:我在2020年确诊后不久就开始失去说话能力。然后它变成了一场持续多年的慢动作车祸,直到几乎没有人能听懂我。

到我手术的时候,只有我的助理和我的妻子莱瓦娜能真正完全听懂我。其他几个经常在身边的朋友能听懂我说的一部分话。我依赖我的助理和妻子基本上充当我与他人之间的翻译。没有他们——其中一个或两个——我哪里都不去,甚至视频通话也不行。

桑德斯:2023年7月,凯西自愿参加了一项名为BrainGate 2的临床试验。加州大学戴维斯分校的一位神经外科医生将四个小型电极阵列放置在凯西大脑的左半球。我想更多地了解这些阵列的工作原理,于是我和那位将凯西的设备植入他大脑的神经外科医生聊了聊。

大卫·布兰德曼:我叫大卫·布兰德曼,是加州大学戴维斯分校神经假体实验室的联合主任,致力于开发下一代神经技术来帮助瘫痪患者。脑机接口是一种记录瘫痪患者脑信号的设备,然后翻译这些脑信号,使人们能够用思想控制环境中的物体。

桑德斯:凯西的植入物由四个阵列组成。每个都是方形的,大约有胡椒粒那么宽。每个阵列有一堆尖端向下伸入他的大脑。它们看起来有点像那些金属猫梳。四个阵列总共包含256个电极。当凯西想着说出一个词时,这些电极都在监听他的大脑活动。

布兰德曼:这个设备的工作原理是记录他尝试说话时大脑的电活动。

桑德斯:凯西头部的线缆将这些神经信号传送到附近的计算机。在那里,机器学习算法尝试进行一种非常复杂的匹配。算法将凯西的脑活动模式与某些被称为音素的特定声音可预测地出现的模式进行匹配。

布兰德曼:我们将他的脑信号转化为39种不同英语音素中的一种。音素是语言的构建单元。所以如果我说“hello”这个词,“hello”可以分解为ha、uh、uhl和oh。Hello。

桑德斯:接下来,这些算法将这些音素组装成词语,对凯西想说的话做出最佳猜测。

布兰德曼:所以我将这种脑活动转化为音素,然后使用人工智能将音素转化为词语,词语转化为句子,句子转化为段落。

桑德斯:这就是整个过程,它让凯西能在Slack上输入消息、给朋友发短信、发送电子邮件。有时在他给我的邮件中,他会在底部加一个签名:“发自我的大脑”,配上那个可爱的粉色小大脑表情符号。他的设备还让他能将这些输入的消息转化为口语。这些词语会显示在他面前的屏幕上,凯西进行校对。如果一切看起来没问题,凯西就发出确认,系统便大声说出这些话。布兰德曼告诉我,当凯西尝试自然说话时,他每分钟大约能说出六个可理解的词。这相当慢。

布兰德曼:使用这个假体,他大约能达到每分钟50个词。而我现在跟你说话时,根据我有多兴奋,我的语速大约是每分钟120到150个词,对吧?所以我们把他恢复到了他作为健全人时语速的大约三分之一。

桑德斯:在我们谈话前几天,我通过电子邮件给凯西发了一些问题。但他没有用电子邮件回答。他使用他的脑机接口,仅仅通过想着他想对我说的话,就实时回答了我的问题。当我们互相介绍时,我看到了时间尺度被拉长了。

凯西:这是一个挑战,因为我们可以进行这场对话,但我比普通人慢,即使在我们长大的地方也是如此。

桑德斯:凯西和我都在南方长大,凯西在亚特兰大地区,我在田纳西州中部。但他现在已经在湾区生活了20多年。和我一样,他的口音已经淡了。

凯西:是的,我已经没有太多南方口音了。但我热爱我的根,我很想带女儿看看我在哪里以及如何长大。但当你不能移动时,旅行是极其困难的。

桑德斯:在行动能力受限之前,凯西做过很多很酷的事情。

凯西:我职业生涯的大部分时间都是一名气候活动家,走遍全球,与那些破坏地球的公司作斗争,努力为我们的星球和居住其上的人们带来一些正义。我非常幸运,因为我能够做我热爱的事情,同时也与其他杰出的人建立了深厚的友谊和关系。

桑德斯:我对凯西的采访与我做过的任何采访都不同。我习惯了在采访中保持灵活。有时区问题、语言障碍、数学家有时用符号和公式回答我的问题。但当我登录Zoom与凯西交谈时,我很快意识到这次对话会不一样。谈话开始时我很不自在。我手忙脚乱地说了太多。我问凯西一个问题,然后在他还没来得及回答之前又问了一个。我不小心打断了他几次。有时我不确定他是在组织答案还是在等我说点别的。我这种糟糕的表现对凯西来说一点也不陌生。

凯西:像你这样的情况很难保持专注,因为你不习惯这种慢节奏的对话。

桑德斯:他说得对。我不习惯。而时机,在对话中以及在生活中,就是一切。以下是加州大学洛杉矶分校的格雷格·布莱恩特。

布莱恩特:话轮转换,对话中的话轮转换,最长也就是几百毫秒的级别。两百毫秒已经是一个很长的话轮了。那是五分之一秒。我是说,那真的很快。所以如果你哪怕有一点慢,就很明显。我们必须协调我们的谈话,使我们的话语有意义。我不能花太长时间,也不能太快地回答你的问题。我必须知道何时停止、何时开始。我们进入这种流动状态,一种动态,这对我们对对话、对对方、对自己的感受有着巨大的影响。你知道,一切都是时机。

桑德斯:让我们转到朱尔斯。他的话也始于他的大脑,但之后走了不同的路径。他通过注视屏幕上的字母来输入,然后他的声音克隆说出来。这也是一个缓慢的过程,而且也需要大量的脑力。在我们对话的某个时刻,MA描述了这需要朱尔斯付出多大的努力。

MA:在他的大脑里,你可以想象他在做所有这些脑力体操,现在,他在听我们说话。他在用眼睛写字,一个字母一个字母地为对话做贡献。再加上他有预先写好的你那些问题的答案。所以真的就像,幕后有一大堆事情在发生,当然任何不需要考虑自己声音的人都理所当然地认为这些是理所当然的。所以我会暂停一下,让他说完他想说的话。

桑德斯:MA和我在这里等朱尔斯发表意见。到目前为止,你们已经听到了十秒钟的停顿。完整的停顿长度是一分十八秒。

朱尔斯:与不每天和我互动的人交流是一个挑战。我从思想到实际说话之间有一个重大的瓶颈。

桑德斯:和凯西一样,这些停顿会带来困难。以下是言语病理学家杰西卡·皮特罗夫斯基对此的更多说明。

皮特罗夫斯基:人类对沉默不太舒服,尤其是在交流中。当我们互相交谈时,我们会填补沉默,这样不那么尴尬。所以,一个使用设备交流的人可能非常习惯那些延迟期和更多的沉默,他们对此感到舒适。而与他们交流的人则会想,让我们继续吧,我想帮忙,我想填补空白,我不想让这感觉尴尬,但反过来,可能会让他们想象得更尴尬一些。

桑德斯:朱尔斯讨厌别人在他还没准备好说出想说的话之前就读他的屏幕。这种厌恶激发了他在脱口秀表演中的一个笑话。

[朱尔斯音频]我在打字的时候人们读我的屏幕,试图替我说完。就像不得不 constantly 应付一个过度好胜的你画我猜搭档。

桑德斯:朱尔斯和凯西经常遇到别人的不耐烦。不是说这不是出于好意,但仍然,这没有帮助。更糟糕的是,他们还要面对别人对他们能力的误解。凯西是一个有趣、风趣、引人入胜的对话者。这对所有不慢下来听他说话的人来说可能并不明显。而这种假设真的让他很困扰。

凯西:被这样草率地对待是极其没有尊严的,也是医疗系统的一个可怕后果。

桑德斯:布兰德曼也见过这种轻视、这种不当对待。这发生在他与我前面提到的BrainGate 2项目的一些志愿者合作时。那就是凯西参加的同一个试验。

布兰德曼:我认为作为医疗专业人员,我们把无法交流与缺乏理解混为一谈。我亲眼目睹了这一点,我看着医生和护士与我的BrainGate参与者互动,然后把他们当作无效的人对待。或者因为他们无法表达自己而当作他们不在那里。而事实上,我在重症监护室和病房里,在与任何与我参与者打交道的人交涉时,都会特意说,看,你知道,他们在里面。他们就在那里。他们能理解发生的一切。他们只是恰好不能说话。所以请以他们应得的尊重和尊严对待他们。这对与我合作的参与者来说是一个持续的挑战。这就是我们面临的现实,这就是我们对不能为自己说话的人的偏见。

桑德斯:凯西和其他渐冻症患者并不孤单。许多说话有困难或只是需要更多时间的人都会被这样评判。以下是Bridging Voice的卡琳娜·纳金。

纳金:这关乎人们如何看待你。所以当你患有渐冻症,你不能动,甚至没有面部表情,还被连接到机器上时,人们会认为你不在那里。我们每天都看到这种情况,人们要么不与渐冻症患者说话,认为他们无法回应。他们说话很大声很慢。而他们的头脑完全在运转。

桑德斯:目前,凯西的话语可能很慢。但尽管有所有这些缺点,它们仍然是一个绝对的奇迹。还记得本期节目开头的霍克伯格吗?他是与凯西合作的神经科学家之一,他给我讲了一个凯西的BCI植入后头几天测试的故事。

霍克伯格:第一天我们给了他50个词的词汇量。即使在第一天,他就能用这50个高价值词汇说出一些关键句子。

桑德斯:在测试的这个阶段,凯西被要求说一些预先指定的句子,以便研究人员测试他们的系统。

凯西:大多数句子是我们所谓的闭环。

桑德斯:这意味着研究人员已经知道凯西想着要说的词语应该对应什么样的脑活动模式。这是一个测试系统是否在工作的实验。

凯西:解码器能够验证我是对还是错,给我简短的预载句子来说。我被要求说:“那有什么用?”

桑德斯:“那有什么用?”并不是有史以来最动人的句子,但它在房间里以及对于远程通过视频加入的研究人员都产生了巨大影响。

凯西: “那有什么用?”我成功之后所有人都在哭。

桑德斯:后来,凯西有了真正表达自己的时间。

凯西:只有在最初几次会话之后,我才有真正的时间在相对私密的环境中在相当长的时间里说我想说的话。话虽如此,那是纯粹的喜悦,即使在那个时刻我没能说出我想说的话。那对我来说是一个深刻的时刻。意识到近在眼前的可能性——对我以及全人类而言——令人震撼。

桑德斯:从那以后事情只会越来越好。让我来设定一下场景,在这次会话中,霍克伯格在他的办公室里,凯西在他家中,坐在客厅里。霍克伯格通过屏幕看着凯西。

霍克伯格:第二次会话时,回来给了他完整的12.5万词以上的英语词典以供选择。也就是说,解码器和AI以及机器学习语言模型基本上能够从一整本词典中进行选择。他说的第一句话大概是:“这东西开着吗?”挺好笑的一句话。

他在使用脑机接口的第二天说的下一句话,或者说我们解码的第二句话是:“我在找一只猎豹。”这才第二天。我们知道有各种各样的解码错误可能出现。当我看到那个时,我说,你知道,不可能在使用植入式脑机接口的第二天第二句话会说什么猎豹。你知道,那显然是一个解码错误,没关系。

桑德斯:霍克伯格能在屏幕上看到凯西,但他看不到房间里刚刚发生的一件非常重要的事情。

霍克伯格:他四岁的女儿刚刚走进房间,打扮成一只猎豹。

桑德斯:以下是凯西对同一时刻的回忆。

凯西:是的,我非常清楚地记得这一次。那是我在2024年8月初经历的第二次说话会话。我女儿穿着她的猎豹服装,可爱极了。这是她最喜欢的动物之一。他们当时正处于一个做什么都超快的阶段,从一个东西跑到另一个东西。所有人都认为这一定是个错误,我不可能使用这个语言。太随机了。这只是来自像利这样不在我们房间里的人的想法。在场的研究人员可以证实,是的,我说了这句话。我是认真的,因为独特的语境。

桑德斯:这是许多父母都有过的时刻之一。孩子做了可爱的事,父母配合着玩。但对凯西和他的孩子来说,这个时刻绝非寻常。

霍克伯格:她大概是在他被诊断出渐冻症前后出生的。可以想象,因为从电脑里传出的声音是他自己声音的一个版本,然后其余的互动就是爸爸和女儿之间的。是父母和孩子之间的。

桑德斯:脑机接口给了凯西无价的东西——一分钟时间和他可爱的小猎豹玩耍。

凯西:我内心平静地相信它会像最终对我起作用那样起作用。我做了大量研究,关键是这项技术正处于突破的边缘。那为什么不是我呢?我问自己。

桑德斯:在研究项目的早期,凯西和他的妻子被告知不要期望它会起作用。这是一个实验,结果远不确定。然而,凯西很乐观。

霍克伯格:所以对我来说,看到它如此之快地、如此之强地起作用,真的真的强调了这些技术将不仅能够恢复日常生活活动,不仅能够恢复关键的工具性日常生活活动,而且它们将恢复父母与孩子之间的交流。它们将恢复朋友、家人、护理伙伴和医疗保健提供者之间的交流。

桑德斯:因为这是一个关于人声的播客,我一直在问我采访的人他们的声音对他们意味着什么。以下是凯西的回答。

凯西:它意味着太多太多。我认为这是我在整个疾病中失去的最重要的东西。

桑德斯:把它找回来,即使是以有限的形式,就是一切。

凯西:能够从一开始就被理解,能够说话和回应,回应我的女儿,这太神奇了,尤其是因为在那之前我的生活中完全缺乏这些。

桑德斯:凯西恢复的声音改变了他的生活。

凯西:这是能够养育我女儿的能力。这是能够与我妻子连接的能力。这是能够与朋友开一个挖苦玩笑的能力。时机就是一切,这就是为什么拥有这一切——不,这是完全革命性的。回顾我之前的回答,这确实是仍然可以继续改善的事情。

桑德斯:在实验开始时,研究团队告诉凯西和他的妻子,他们每周只能尝试使用BCI进行随意聊天几个小时。考虑到他们已经很久无法进行有意义的交谈,那些时间在当时感觉像是一份巨大的礼物。以下是布兰德曼。

布兰德曼:随着我们每周去他家两三次,事情变得越来越好。到了某个程度,凯西想用这个系统说出所有他想说的话。

桑德斯:早期,只有研究人员能为凯西设置系统,把他接入矩阵,正如他所说。

布兰德曼:所以我们的研究技术人员开始每周去他家三四五次,为他穿戴和卸下系统,使他能够交流,以便他能度过一天。所以在某个时候,我们看到,嗯,他真的很想用这个系统。我们能为他简化这个流程吗?于是我们向FDA申请,实际上允许我们稍微修改临床试验方案,以便我们可以培训他的护理伙伴在没有我们在场的情况下为他穿戴和卸下系统。从那时起,他几乎每天都使用这个系统。

桑德斯:这与每周几个小时相去甚远。我要给你们读一些布兰德曼和他的同事于2026年6月在《自然神经科学》上发表的关于凯西BCI的研究论文中的数字。

“在近两年的时间里,参与者——也就是凯西——在家中在没有研究人员在场的情况下使用BCI超过3800小时,与家人和朋友保持丰富的人际交流,独立控制他的个人电脑,并维持全职工作,尽管他瘫痪了。他交流了183060个句子,总计1960163个词,平均每分钟56个词。他将92%的句子标记为至少大部分被正确解码。”

这些数字以非常精简的科学方式来说,说实话,很枯燥。但当你停下来想一想,每一个句子,那近两百万个词中的每一个,都是连接凯西与他生活的绳索。它们是给女儿的信息,与伴侣的玩笑,给朋友的短信。它们是与妻子和孩子玩Wordle的游戏。以下是凯西和他的家人在加州大学戴维斯分校健康中心拍摄的一段视频中。

好的,没有A,但我们有一个E。M-I。O。M-I。做得好,团队。我们快到了。

桑德斯:我们在这里谈了很多关于词语的事情,以及凯西的BCI恢复他说出词语能力的根本方式。但凯西的BCI比这更进一步。它给了他添加一些额外东西的能力。他还有一些花哨的功能。

凯西:它首先且 foremost 是一个 communication 设备,而不是我用来发表有趣言论的工具。但我的 repertoire 中确实也有那个。例如,我有能力为我说的内容添加音效。

桑德斯:比如,凯西有一个悲伤长号的音效。

[悲伤长号音效]

凯西:所以如果我发现什么好笑的想说“LOL”,电视剧《宋飞正传》的主题曲就会播放。

[主题曲]

桑德斯:这就是他的意思。这些玩笑让我想到朱尔斯,这让我好奇凯西是否也能像朱尔斯一样用他的系统骂人。

凯西:回答你关于我能否用这个设备骂人的问题,我绝对[哔]可以。

桑德斯:所以对于所有好奇的 minds,答案就在这里。稍微展望一下未来,地平线上有很多事情在发生。这里的科学进展迅速,许多公司正在开发类似的系统。我们将在几集中讨论这些。但就目前而言,凯西是孤独的。用科学家的话说,他是一个N=1——唯一一个如此重度使用脑机接口进行交流的人。

凯西:我希望很快能改变这一点,因为我觉得每个想要这个的人都应该有机会拥有它,特别是那些和我患同样疾病的人。

桑德斯:霍克伯格告诉我,这种帮助他人的动力在自愿参加这类临床试验的人中很常见。

霍克伯格:试验中的每一位参与者都非常慷慨,非常清楚地认识到研究还处于早期阶段,但他们告诉我们,他们想帮助其他瘫痪的人。他们希望能够让脑干中风、颈椎脊髓损伤、渐冻症或其他致瘫疾病或损伤的人改善功能,能够重新控制那台电脑,或恢复他们以前拥有的流利言语和交流能力。所以正是这种慷慨的精神真正定义了我们的BrainGate研究中的参与者。

桑德斯:凯西的慷慨,他帮助他人的动力,在我们交谈中真的闪耀出来。但我想回到他本身。我问凯西,把这项技术放在家里、放在大脑里是什么感觉。他告诉我我们的声音有多么珍贵,以及我们许多人多么容易忘记这一点。

凯西:我们理所当然地认为它们存在。跑步时想释放压力时尖叫的能力。在伴侣耳边说甜言蜜语的能力。这些才是生活真正意义所在。当你被剥夺了这些时刻,你会不惜一切代价想要把它们找回来。这就是为什么这对现在来说已经足够好了,我们应该对这样的设备进行加速审批。

桑德斯:与正常对话的节奏相比,凯西的BCI很慢。而且它偶尔会出错。

凯西:抱歉,我在试图回答你。它在改变我想说的话。而我在固执,而不是试着用另一种方式说,我现在就试试。

[桑德斯音频]我不着急。你想固执多久就固执多久。

桑德斯:而且它需要互联网连接和电源。别忘了凯西头顶伸出一个转接头。但这些都是可以解决的问题,科学家们正在积极研究。所以不完美,但正如凯西所说,对现在来说已经足够好了。

恢复声音的科学努力对朱尔斯、凯西和其他人来说是变革性的。但将声音与身体分离的能力也为我们的社会和我们的生活带来了新的可能性。

MA:他的声音将在我们拍摄的任何视频或事物中继续存在,无论他有意给我们留下了什么,那就是全部了。

桑德斯:在下一集中,我们将走出实验室,进入一个奇怪的新数字世界,在那里声音可能不是它们看起来的样子。

哈尼·法里德:所以我们正生活在一个,我认为,同等程度令人兴奋和令人恐惧的时代。而不知怎的,这些同时存在。

桑德斯:在一个声音可以如此容易被克隆的世界里,将会发生什么变化?

法里德:今天,只需一个键盘和一个互联网连接,你就可以打字,然后得到任何东西、任何人在做或说任何事情的图像、声音、视频。

桑德斯:《深渊》是《科学新闻》的制作。它基于我——劳拉·桑德斯——的原创报道。本集由埃拉·罗文制作和混音。我们的项目经理是阿什利·耶格尔。南希·舒特是我们的主编。我们的音乐由Blue Dot Sessions提供。本播客部分得益于伯勒斯·惠康基金会的资助以及PRX的支持。

本播客在伯勒斯·惠康基金会的资助下并与PRX合作制作。播客标志由尼尔·韦布设计。

英文来源:

How brain implants turn thoughts into speech
Casey Harrell lost his voice to ALS or amyotrophic lateral sclerosis. As a volunteer in a clinical trial, he’s helping scientists test a brain-computer interface. Electrode arrays that are implanted in Casey’s brain send signals to nearby computers, where those signals are translated into words he intended to say.
Casey uses this system every day to talk with his wife, daughter, friends and work colleagues. The moment he realized that it worked was “pure joy,” Casey says. Even though the researchers scripted his words for that first test, “it was a profound moment,” he says. “The possibilities for the near future for me and the rest of humanity were overwhelming to realize.”
This episode of The Deep End: Talk to Me follows Casey and the fast-moving neuroscience that enables him to speak.
Episode 4 transcript
Patients paralyzed from the neck down can manipulate robotic arms with their thoughts. The woman in this video is controlling a robot with her mind. The so-called BrainGate system relies on a sensor of electrodes implanted in the motor cortex of the brain. Researchers hope by implanting electrodes in the brain it can read the brain signals meant to move the muscles involved in speaking, and translate the person’s intention to speak into comprehensive speech produced by a computer.
Laura Sanders: Controlling things with your mind. It sounds like science fiction, but it’s definitely real. Today, we’re going to explore what it’s like to communicate with a brain implant that decodes your thoughts and turns them into words.
[voices] Hey mom, can I tell you something? The voice is absolutely an integral part of your identity. Hello? That added layer of it being you. Do you want to talk? Your voice. Hi. Your voice is you.
Sanders: From Science News, I’m Laura Sanders, and this is The Deep End, Talk to Me. Okay, so what is BrainGate? It’s an experimental system that connects electrodes that are implanted in a person’s brain to a nearby computer. It’s called a brain-computer interface, or BCI. The goal here is big: to help people with paralysis or brain injuries or missing limbs regain control over their world.
Leigh Hochberg: BrainGate clinical trials have now been going on for nearly 22 years. It’s 21 point something at the moment.
Sanders: Leigh Hochberg is the director of the BrainGate Consortium and Clinical Trials. He also has a job at Massachusetts General Hospital, Brown University and the VA.
Hochberg: People who may have had a brainstem stroke and suddenly lost the ability both to move and to speak, or for somebody with ALS who is progressively losing the ability to move or to speak, there’s long been a hope that particularly implantable brain-computer interfaces could restore communication and could restore mobility.
Sanders: In 2004, a BrainGate device was implanted into a person’s brain for the very first time. That person was a man named Matthew Nagle, and he had been paralyzed in an accident. With BrainGate, he could control a computer cursor with his thoughts. Here’s a 2008 report from IT World.
Technology gave him hope. He was one of four testing BrainGate, a new technology where a chip is implanted in the brain that picks up electrical impulses. A computer then interprets those impulses as actions.
Hochberg: Those early studies have been really encouraging, and I think speaks to the potential for the field.
Sanders: Over the years, there’s been amazing examples of people using BrainGate. People have used it to click around on the internet, play video games, one woman who was paralyzed was even able to drink a cinnamon latte with a robotic arm.
Hochberg: And she was able to think about using her own arm and reach out with a robotic arm and pick up that thermos and bring it to her mouth, and tip it towards her mouth, and then put the straw in her mouth and then to take that sip. And while it wasn’t the first time in nearly 15 years that she had had something to drink, it was the first time that she was really able to do so solely of her own volition. I remember the smile on her face when she did that.
Sanders: Over the decades, BrainGate has had a lot of research success in a lot of ways. But what about talking? Hochberg says that when it comes to decoding speech, the science here is still very, very young.
Hochberg: We are at early days, just to be super clear. This is the very beginning of this field. There are only a small handful of people in the world who have had any type of implantable BCI that has been decoding speech.
Sanders: Where does the science stand when it comes to decoding the intention to talk? In the past few episodes, we’ve heard the stories of people who have lost their ability to speak. One of those people was Casey Harrell, and Casey also happens to have a BrainGate device implanted in his brain.
Casey Harrell: My name is Casey Harrell and I am 47 years old.
Sanders: Casey wakes up in the morning, and plugs in.
Casey: The first thing I will usually do after getting out of bed is to plug into the Matrix, so to speak.
Sanders: By that, he means attaching two wires to two pedestals that come out of the top of his head. These two black wires then snake down to nearby computers. Collectively, these electrodes in Casey’s brain, wires from his head, and computers are called the BrainGate 2 neural interface system. So Casey gets it all fired up and then he’s off.
Casey: Unless I have a medical appointment or something that will move me out of the house, I will sit in this position for most of the day, with specifically small breaks in between.
Sanders: All the while, Casey is generating brain signals as he thinks about saying words. These are small, precise electrical pulses made by Casey’s neurons, specifically the cells in his speech motor cortex. Remember how we learned there are over 100 muscles that are involved in speech? These are the parts of the brain that control those muscles.
Casey: I am using a decoder for about ten to 14 hours a day, seven days a week, for the past almost two years, making me the world’s biggest power user of the brain-to-computer interface known to man.
Sanders: Like Jules, Casey has ALS. The disease has slowly taken away his ability to move and to talk.
Casey: I began to lose my ability to talk shortly after my diagnosis in 2020. Then it became a slow motion car crash of multiple years before almost no one would be able to understand me.
By the time of my surgery, only my assistant and my wife, Levana, could really understand me fully. A few other friends who were around often would be able to understand some of what I was saying. I relied upon my assistant and my wife to basically serve as my translators to other people. I did not go anywhere, even a video call without them, either one or both.
Sanders: In July of 2023, Casey volunteered for a clinical trial called BrainGate 2. A UC Davis neurosurgeon put four small electrical arrays onto the left side of Casey’s brain. I wanted to know more about how these arrays worked, so I talked to the neurosurgeon who put Casey’s devices into his brain.
David Brandman: My name is David Brandman, and I am the co-director of the UC Davis neuroprosthetics lab, developing next-generation neurotechnology to help people with paralysis. A brain-computer interface is a device that records brain signals from people that are paralyzed and then translates those brain signals to allow people to control objects in their environment using their minds.
Sanders: Casey’s implant is made of four arrays. Each of them are square and they’re about as wide as a peppercorn. Each array has a bunch of pointy ends that go down into his brain. They look a little like those metal cat brushes. Altogether, the four arrays contain 256 electrodes. These electrodes are all listening to Casey’s brain activity when he thinks about saying a word.
Brandman: The way this device works is by recording the electrical activity of the brain as he tries to speak.
Sanders: The wires from Casey’s head carry those neural signals to nearby computers. That’s where machine learning algorithms attempt to do a very complex sort of matching. The algorithms match Casey’s brain activity patterns to the patterns that predictably occur with certain sounds called phonemes.
Brandman: We translate his brain signals into one of 39 different English phonemes. Now, a phoneme is a building block of language. So if I say the word hello, the word hello can be broken down into ha, uh, uhl, and oh. Hello.
Sanders: Next up, those algorithms assembles these phonemes into words by making its best guess about what Casey wants to say.
Brandman: So I translate this brain activity into a phoneme and then, using artificial intelligence, turn phonemes into words, words into sentences, and sentences into paragraphs.
Sanders: So that’s the process, and it allows Casey to type messages on Slack, text his friends, and send emails. Sometimes in his emails to me, he includes a tagline at the bottom: “Sent from my brain,” with that cute little pink brain emoji. His setup also lets him convert these typed messages into spoken words. These words show up on a screen for him where Casey proofreads them. If everything looks good, Casey gives the go-ahead and the system speaks the words out loud. Brandman tells me that when Casey tries to speak naturally, he can say about six understandable words per minute. That’s pretty slow.
Brandman: Using this prosthesis, he’s in the ballpark of 50 words per minute. Now, as I’m speaking to you right now, I’m speaking on the order of 120 to 150 words per minute, depending on how excited I am, right? So we’ve got him back to about a third of how quickly he spoke as an able-bodied person.
Sanders: I had emailed Casey some questions a few days before our talk. But he didn’t answer them by email. He used his brain-computer interface to answer my questions in real time just by thinking about the words he wanted to say to me. As we introduced ourselves, I saw how the timescale had stretched.
Casey: It’s a challenge because we can have this conversation, but I am slower than the average Joe, even where we grew up.
Sanders: Casey and I both grew up in the south, Casey in the Atlanta area and me in middle Tennessee. But he’s lived in the Bay Area for over 20 years now. And like me, his accent had faded.
Casey: And yes, I do not have much of a Southern accent anymore. But I love my roots, and I would love to show my daughter where and how I grew up. But it is exceedingly difficult to travel when you cannot move.
Sanders: Before his mobility was limited, Casey did a lot of cool things.
Casey: I spent most of my professional career as a climate campaigner and have went all over the globe, fighting companies who are destroying the Earth and trying to bring some justice to our planet and the people who inhabit it. I am very fortunate because I have been able to do something that I love and also develop some deep friendships and relations with other phenomenal people.
Sanders: My interview with Casey was unlike any I’ve ever had. I’m used to being flexible for interviews. There’s time zone problems, language barriers, mathematicians who sometimes answer my questions in symbols and formulas. But when I logged on to Zoom to talk with Casey, I quickly realized this conversation would be different. I was self-conscious at the beginning of our talk. I fumbled around too much. I’d ask Casey a question and then another one before he had a chance to answer. I accidentally interrupted him a few times. Sometimes I wasn’t sure if he was composing an answer or just waiting for me to say something else. This poor performance on my part wasn’t unusual for Casey at all.
Casey: It is hard for people in your position to stay focused because you were not used to the cadence of a conversation this slowly.
Sanders: He’s right. I was not used to it. And timing, in conversations and in life, is everything. Here’s Greg Bryant of UCLA.
Bryant: Turns, conversational turns, happen at the level of hundreds of milliseconds at the longest. Two hundred milliseconds is a long turn. That’s a fifth of a second. I mean, that’s really fast. So if you are slow at all, it’s noticeable. We have to coordinate our talk so our terms make sense. I can’t take too long or be too quick to respond to your questions. I have to know when to stop and start. We get into this flow, kind of a dynamic, and that is hugely consequential for how we feel about the conversation, the other person, ourselves. You know, it’s all timing.
Sanders: Let’s turn to Jules. His words start in his brain too, but then they take a different route. He types them by looking at letters on a screen, then his voice clone says them. It’s also a slow process and one that takes a lot of mental power, too. At one point in our conversation, MA describes the effort this requires of Jules.
MA: And in his brain, you know, you can imagine he’s doing all these mental gymnastics, to, right now, he’s listening to us. He’s writing something with his eyes, like letter by letter to contribute to the conversation. Plus, he has the answers that he pre-wrote, of the questions that you asked. So it really is like, there’s a whole lot happening behind the scenes that of course anybody who doesn’t have to think about their voice takes for granted. So I’ll pause to let him complete what he’s saying.
Sanders: MA and I wait here for Jules to weigh in. So far you’ve heard ten seconds of the pause. The full length of the pause was a minute and 18 seconds.
Jules: It is a challenge to communicate with people who don’t interact with me on a daily basis. I have a major bottleneck from my thoughts to actually speaking.
Sandes: And as with Casey, these pauses can introduce difficulties. Here’s speech pathologist Jessica Pietrowski with more on that.
Pietrowski: Humans are not very comfortable with silence, especially communication. And when we’re talking to one another, we fill in the silences, it’s less awkward. And so, someone who’s communicating with a device is probably very used to those latency periods and some more silence and they’re comfortable with it. Whereas the person they’re communicating with is like, let’s keep it going, like this is, I want to help, I want to fill in the blanks, I don’t want this to feel awkward, but in turn, might make it a little more awkward than they think.
Sanders: Jules hates when people read his screen before he’s ready to say what he wants. This particular hate inspired a joke in his stand-up routine.
[Jules audio]: People read off my screen while I’m typing and try to finish what I am saying. It’s like having to constantly deal with an over-competitive charades partner.
Sanders: Jules and Casey encounter other people’s impatience constantly. Not that it isn’t well-intentioned but still, it does not help. Even worse, they face other people’s misconceptions about what they’re capable of. Casey is an interesting, funny and compelling conversationalist. That might not be apparent to everyone who doesn’t slow down to hear him. And that’s an assumption that really bothers him.
Casey: It is incredibly undignified and a terrible consequence of the health care system to treat me like this very quickly.
Sanders: Brandman has seen this dismissal, this mistreatment too. It happens with some of the volunteers he works with on the BrainGate 2 project I mentioned earlier. That’s the same trial Casey’s in.
Brandman: I think as healthcare professionals, we conflate the inability to communicate with the lack of understanding. And I’ve witnessed this firsthand as I’ve watched doctors and nurses interact with my BrainGate participants and then treat them as if they are invalid. Or they’re not there because they’re unable to express themselves. And in fact, I go out of my way in the intensive care unit on the wards and dealing with anyone who’s dealing with my participants to say, look, you know, they’re inside. They’re there. They can understand everything that’s happening. They just happen to not be able to speak. And so please treat them with the respect and dignity that they deserve. And it’s a constant challenge for the participants that I work with. And it’s the reality of which we face and it’s the bias that we have against people that can’t speak for themselves.
Sanders: Casey and the other people with ALS are not alone. Many people who have trouble talking or who just need more time can be judged this way. Here’s Karina Nagin from Bridging Voice.
Nagin: It’s how people see you. And so when you have ALS and you cannot move and you don’t even have facial expressions and you’re hooked up to machines, people assume you’re not there. And we see this every day where people either don’t talk to the person with ALS, assuming that they can’t respond. They speak really loudly and slow. Their mind is completely working.
Sanders: For now, Casey’s words may be slow. But despite all the drawbacks, they are still an absolute marvel. Remember Hochberg from the start of this episode? He’s one of the neuroscientists who works with Casey, and he told me a story from the first few days of testing after Casey’s BCI was implanted.
Hochberg: We gave him a 50-word vocabulary on day one. And even on day one, he was able to say some key sentences from those 50 high-value words.
Sanders: At this point in the testing, Casey was asked to say some pre-specified sentences so researchers could test their system out.
Casey: Most of the sentences were what we called closed loops.
Sanders: That means that the researchers already knew what brain activity patterns should go with the words Casey thought about saying. It was a test to see if the system was working.
Casey: The decoder would be able to verify whether I was right or wrong, giving me short preloaded sentences to say. I was asked to say, “What good is that?”
Sanders: “What good is that?” is not the most poignant sentence ever uttered, but it had a big effect both in the room and for researchers joining remotely by video.
Casey: “What good is that?” Everyone is crying after I am successful.
Sanders: Later on, there was time for Casey to actually express himself.
Casey: It was only after the first few sessions that I had any real time to say what I wanted for a good amount of time in relative privacy. That said, it was pure joy, even though I wasn’t getting to say what I wanted to say in that moment. It was a profound moment for me. The possibilities for the near future, for me and the rest of humanity were overwhelming to realize.
Sanders: Things only got better from there. To set the scene here, during this session, Hochberg was in his office and Casey was at his home, sitting in his living room. Hochberg was watching Casey on a screen.
Hochberg: On the second session, came back and gave him the full 125,000 plus word English dictionary to be able to select from. That is, the decoders and the AI and the machine learning language models would be able to select from basically a full dictionary. And the first thing that he said was something along the lines of, “Is this thing on?” Kind of a funny statement.
The next thing that he said on the second day of using brain-computer interface, or the second thing that we decoded was, “I’m looking for a cheetah.” And it’s the second day. We know there’s all kinds of opportunities for decoding errors. And when I saw that, I said, you know, there’s no way that the second sentence on the second day of using an implanted brain-computer interface is going to say anything about a cheetah. You know, that’s obviously a decoding error and that’s fine.
Sanders: Hochberg could see Casey on the screen, but he couldn’t see something really important that had just happened in the room.
Hochberg: His four-year-old daughter had just walked into the room dressed as a cheetah.
Sanders: Here’s Casey remembering the same moment.
Casey: Yes, I remember this time very well. It was the second session of talking that I experienced early in August 2024. My daughter was incredibly cute in her cheetah outfit. This was one of her favorite animals. They were going through a phase where everything they did was super fast, running from thing to thing. Everyone was thinking that it must have been a mistake that I could not use this language. It was so random. This was only from the people like Leigh who were not in the room with us. The researchers who were there could validate that, yes, I said this. I meant it because of the unique context.
Sanders: It’s one of those moments that many parents have. A kid does something cute, a parent plays along. But in this moment for Casey and his child, it was anything but ordinary.
Hochberg: And she was born around the time that he was diagnosed with ALS. And one could imagine that because the voice that was coming out of the computer was a version of his own voice, and then the rest of the interaction was between dad and daughter. And it was between parent and child.
Sanders: The BCI had given Casey something priceless, a minute to play with his sweet little cheetah.
Casey: I was quietly confident that it would work the way it ended up working for me. I had done a lot of research and the thing was that the technology was on the cusp of a breakthrough. And why not me? I asked myself.
Sanders: Early on in the research project, Casey and his wife had been told not to expect it to work. This was an experiment and the outcome was far from certain. Casey, however, was optimistic.
Hochberg: So to me, seeing it work that quickly and that powerfully really, really emphasized that these are technologies that are going to be able to restore not only activities of daily living, not only key kind of instrumental activities of daily living, but they’re going to restore communication between parent and child. They’re going to restore communication between friends and family and care partners and health care providers.
Sanders: Because this is a podcast about the human voice, I’ve been asking people I interview what their voice means to them. Here’s Casey’s answer.
Casey: It means so much. I think it is the most important thing that I have lost in this whole disease.
Sanders: Getting it back, even in a limited form, is everything.
Casey: It was so amazing to be able to be understood right from the get-go and to be able to talk and respond, respond to my daughter, especially because that was lacking completely from my life up to that point.
Sanders: Casey’s restored voice changed his life.
Casey: It is the ability to parent my daughter. It is the ability to connect with my wife. It is the ability to make a wry joke with my friends. Timing is everything, and that is why having nothing, this is completely revolutionary. And to look back on my previous answer, it is something that really can still improve.
Sanders: At the beginning of the experiment, the research team had told Casey and his wife that they’d be able to try using the BCI for casual chats just a few hours a week. Those hours felt like a huge gift at the time, considering that they hadn’t been able to talk meaningfully for a long time. Here’s Brandman.
Brandman: As we kept coming to his house two or three times a week, things kept getting better and better. And it got to a point where Casey wanted to use the system to say all the things that he wanted to say.
Sanders: Early on, only the researchers could set Casey up with the system, plugging him into the Matrix, as he puts it.
Brandman: And so our research technologist would start coming to his house three, four, five times a week to don and doff the system to enable him to communicate so that he could go about his day. And so at a certain point, we saw, well, he really wants to use the system. Can we streamline this for him? And so we petitioned the FDA to actually allow us to change the clinical trial protocol slightly so that we could train his care partners to don and doff the system without us being there. And so since then, he’s been a near daily user of the system.
Sanders: That’s a far cry from a few hours a week. I’m going to read you some numbers from a research paper Brandman and his colleagues published about Casey’s BCI in Nature Neuroscience in June of 2026.
“Over nearly two years, the participant, that’s Casey, used the BCI for more than 3,800 hours at home with no researchers present to maintain rich, interpersonal communication with his family and friends, independently control his personal computer, and sustain full-time employment, despite being paralyzed. He communicated 183,060 sentences, totaling 1,960,163 words, at an average rate of 56 words per minute. He labeled 92 percent of sentences as being decoded at least mostly correctly.”
Those numbers in this very bare-bones science kind of way are dry, if we’re being honest. But when you stop to think about it, each sentence, each of those nearly two million words, is a tether that connects Casey to his life. They’re messages to his daughter, jokes with his partner, texts to his friends. They’re games of Wordle with his wife and kid. Here’s Casey and his family in a video shot by UC Davis Health.
Okay, so no A, but we got an E. M-I. O. M-I. Good job, team. We are almost there.
Sanders: We’ve talked a lot about words here and the fundamental ways that Casey’s BCI has restored his ability to say them. But Casey’s BCI goes a little bit farther than that. It gives him the ability to add some extras. He’s got some bells and whistles up his sleeve.
Casey: It is really a communication device, first and foremost, and less of a tool that I use to make funny remarks. But I do have that as well in my repertoire. For example, I have the ability to add sound effects to what I am saying.
Sanders: For instance, Casey has a sad trombone sound.
[sad trombone sound]
Casey: So if I find something funny and want to say, LOL, the theme from the TV show, Seinfeld, will play.
[theme song]
Sanders: This is what he means. These jokes make me think of Jules, which made me wonder if Casey, like Jules, could also swear with his system.
Casey: To answer your question of whether I can curse with this device, I most certainly [beep] can.
Sanders: So for all your inquiring minds, there you have it. Looking ahead a bit, there’s a lot happening on the horizon. The science here is moving fast and many companies are working on similar systems. We’re going to cover that in a few episodes. But for now, Casey is alone. He is an N of 1, as the scientists say – the only person using a brain-computer interface for communication this heavily.
Casey: I hope to have this changed very soon because I feel like everyone who wants this should have the opportunity to have it, especially those who are with my disease.
Sanders: Hochberg tells me that this drive to help others is common among people who volunteer for these sorts of clinical trials.
Hochberg: Each participant in the trial has been just so generous and so clear that they recognize that it’s early days in the research, but they’ve told us that they want to help other people with paralysis. They want to be able to allow people with brainstem stroke or cervical spinal cord injury or ALS or other paralyzing illnesses or injuries to have improved function, to be able to regain control over that computer or regain the fluent speech and communication that they had previously. So it’s that generosity of spirit that has really defined the participants in our BrainGate studies.
Sanders: Casey’s generosity, his drive to help other people, really shines through as we talk. But I want to get back to him. I ask Casey what it’s been like to have this technology in his home, in his brain. He tells me how precious our voices are, and how easy it is for many of us to forget that.
Casey: We take them for granted. The ability to scream when you are running and want to get out some stress. The ability to say sweet nothings into your partner’s ear. These are the things that life is all about. And when you are robbed of those times, you want them back at all costs. And this is why this is good enough for now, and we should have accelerated approval for devices like this.
Sanders: Casey’s BCI is slow compared to the pace of a normal conversation. And it gets things wrong occasionally.
Casey: Sorry, I am trying to answer you. And it is changing what I want to say. And I am being stubborn instead of trying to say it another way, which I will do now.
[Sanders audio] I’m not in any rush. You take as much stubborn time as you want.
Sanders: And it needs an internet connection and power. And let’s not forget that Casey has an adapter coming out of the top of his head. But these are solvable problems, one that scientists are actively working on. So not perfect, but as Casey puts it, good enough for now.
Scientific efforts to restore voices have been transformative for Jules, Casey, and others. But the ability to separate a voice from a body also raises new possibilities for our societies and for our lives.
MA: His voice will live on in whatever videos or things we’ve captured, whatever he’s left us intentionally, and that’s as far as that goes.
Sanders: On the next episode, we’re stepping out of the lab and into a strange new digital world where voices might not be what they seem.
Hany Farid: And so we are living in, I think, in equal parts an exciting time and a terrifying time. And somehow those are coexisting.
Sanders: What changes are coming in a world where voices can be so easily cloned?
Farid: Today, with nothing more than a keyboard and an internet connection, you can type and you can get an image, a voice, a video of anything, anybody doing or saying anything.
Sanders: The Deep End is a production of Science News. It’s based on original reporting by me, Laura Sanders. This episode was produced and mixed by Ella Rowen. Our project manager is Ashley Yeager. Nancy Shute is our editor in chief. Our music is by Blue Dot Sessions. The podcast is made possible in part by the Burroughs Wellcome Fund with support from PRX.
This podcast was produced with support from the Burroughs Wellcome Fund and in partnership with PRX. The podcast logo was created by Neil Webb.

AI科学News

文章目录


    扫描二维码,在手机上阅读