内容来源：https://www.sciencenews.org/article/ai-voice-restore-identity-the-deep-end

内容总结：

人工智能技术帮助失语者重获“原声”

对于因疾病而逐渐丧失说话能力的人来说，如何继续与他人交流成为一个严峻的挑战。近年来，一种名为“声音银行”的技术为患者提供了保留声音的可能，但传统方法合成的声音往往听起来不够自然。如今，借助人工智能技术，语音克隆正在改变这一局面——它能够高度还原人声中的呼吸、语调甚至口吃等细节，让合成声音听起来与本人几乎无异。

朱尔斯·罗德里格斯因患肌萎缩侧索硬化症（ALS，又称运动神经元病），自身声音逐渐退化。他此前通过传统“声音银行”录制的合成声音并不令他满意，甚至曾在脱口秀中调侃自己的备份声音像是“把Daft Punk的歌以四分之一速度播放”。2024年6月，朱尔斯利用自己过去的录音，通过人工智能公司ElevenLabs创建了语音克隆。第一次听到克隆声音时，他感觉“自己重新出现了”；他的妻子则因声音与丈夫失声前太过相似而落泪。

伦敦大学学院言语治疗师理查德·凯夫曾主导英国的声音银行推广工作。他表示，传统声音银行合成的语音“不算糟糕，但也不够好，并不能真正代表那个人”。已故物理学家斯蒂芬·霍金曾对自己标志性的合成语音表示满意，称其“虽有些单薄且带爱尔兰口音，但几乎是人类的声音”。然而对许多患者而言，声音是否像自己，远比“能否说话”更为重要。

非营利组织Bridging Voice负责人卡琳娜·纳金指出，团队最初认为声音是否像本人只是“锦上添花”，但很快意识到这正是核心所在——声音关乎身份认同和“做自己”的感受。她表示，有患者在获得语音克隆后改变了医疗预嘱，选择继续活下去，因为“如果我能用像自己的声音交流，我就愿意活得更久”。还有祖母在拿到语音克隆后立即打电话给从未听过她真实声音的孙辈。

语音克隆对患者家庭的影响同样深远。朱尔斯的妻子MA回忆，丈夫播放克隆声音的第一句话是告诉她“我有多爱你”，她听后浑身发麻、泪流满面，“感觉失去了很久的那部分他又回来了”。她也坦言，这种体验在情感上令人困惑——一部分鲜活的他又回来了，但其他一切并未改变。

2024年9月，朱尔斯经历了一次危及生命的住院危机。此前他明确表示不愿接受气管切开术（一种在喉部插管连接呼吸机以延长生命的手术，但会导致永久失声）。然而在那次危机过后，他改变主意，决定接受手术。MA认为，正是AI语音克隆给了他继续生活的动力：“我坚信，如果没有克隆声音，他不会选择气管切开。”朱尔斯的儿子斯凯勒对此的态度则很简单——“我们有电脑（声音），挺好的。”在他眼中，克隆声音就是父亲的声音。

2024年10月1日朱尔斯接受气管切开术后，仅过了一个半月，他便重返当地喜剧俱乐部的舞台，用克隆声音表演脱口秀。他还能用克隆声音大笑、低语甚至唱歌。

凯夫表示，新一代AI语音克隆是他听过最真实的声音，“听起来就像我服务的那些人本人”。令他尤为振奋的是，许多人反馈社交网络因此得到加强，更多人愿意来与他们交谈、回顾往昔。他说：“这才是声音银行应该做的——维系社交和工作网络。”

同样因ALS失声的佩吉·梅里尔使用语音克隆为孙辈朗读、教授瑜伽和带领引导冥想。她写道：“使用并听到自己的声音，让我重新找回了自我。”作曲家和音乐人帕特里克·达林则用克隆声音创作和演唱新歌。他说首次用AI声音演出时，惊讶地感到身体上的连接感，“你觉得自己又成了一个表演者”。

目前，朱尔斯仍需通过眼动追踪技术在屏幕上逐字输入想说的话，再由克隆声音读出，这一过程耗时且需要高度专注。科学家正在研究通过更早期的言语信号加快输出速度，甚至尝试直接从大脑中提取词语。

中文翻译：

AI能让你找回自己的声音

当人们面临失去说话能力时，他们需要替代的沟通方式。人们保留声音的一种方式是将其存入语音库。近年来这一流程有所改进，但合成出来的声音听起来仍可能不自然。朱尔斯·罗德里格斯在肌萎缩侧索硬化症（ALS）削弱了他的声音后就做了这件事。但他对合成出来的声音并不满意。

利用人工智能的新技术让朱尔斯得以创建一个声音克隆，听起来就像他以前的声音。这些声音克隆带有呼吸声、语调甚至口吃，捕捉到了人声的独特之处。朱尔斯的声音克隆如此逼真，以至于他妻子第一次听到时哭了。

在本期节目中，你将听到AI驱动的声音克隆如何改变人们建立联系的对话方式。其中包括一位名叫帕特里克·达林的音乐人，他用自己声音的克隆来演唱和表演歌曲。

第三集文字稿

劳拉·桑德斯：朱尔斯·罗德里格斯的声音正在消失。

朱尔斯·罗德里格斯：对我来说，感觉就像我在逐渐消逝，就像《回到未来》里马蒂·麦克弗莱看到照片中的家人逐渐消失一样。

桑德斯：朱尔斯患有ALS，和许多患这种病的人一样，这种消逝感觉是不可避免的。但朱尔斯对此并非无能为力。我是劳拉·桑德斯，这里是《深渊：跟我说话》第三集，一档《科学新闻》播客，探讨人类的声音以及它所承载的一切。

[声音片段]嘿妈妈，我能跟你说件事吗？声音绝对是你身份不可分割的一部分。你想说话吗？那层属于你的附加维度。喂？你的声音。嗨。

桑德斯：在今天的节目中，我们将听到朱尔斯声音的回归。

[声音]你的声音就是你。

我们在上一集中提到过这一点，但提醒一下，当人们面临失去声音时，有几种传统的保留沟通方式的方法。一种是趁还能说话时，录下自己说常见或特别的短语。这叫做信息语音库。还有一种技术叫做语音库。它要求一个人录下自己说非常特定的词语。然后这些声音被输入计算机程序，程序可以将它们拼接起来，说出一个人想说的任何话。理论上听起来不错，但实践中，输出结果听起来不太像真人。朱尔斯在确诊后不久就建立了自己的语音库。MA描述了当时的情况。

MA：我记得他录音的时候我们俩互相看着对方，基本上就是老一套的方法，而且他们现在除了克隆声音之外仍然推荐用这种方式建立语音库，但你会收到一大堆需要录制的短语，大概350个短语，然后存进一个文件里，当你需要的时候，公司会把它调出来插到你的设备里。

桑德斯：我采访了理查德·凯夫。他是伦敦大学学院的言语治疗师，与ALS患者合作。ALS也被称为运动神经元病，简称MND。

理查德·凯夫：我的工作是推动英国各地的语音库计划。那是2017年的事。我记得……在最初，我不知道，六个月、九个月之后，我非常自豪地让大约100人完成了语音库，平均每人完成全部1600个短语大约需要三个月。为此非常自豪。那些声音不算糟糕，但也不好。它们并没有真正代表那个人。它们代表了那个人的一部分，一点点。但那是我们当时能做到的最好的了。

在解码了来自某种输入设备（比如开关）的信号之后，人们还需要以一种他人能理解的形式将其传播出去。这可以是视觉上的，比如屏幕上的文字或图表。但如果你想像其他人一样交流——这正是处于我这种位置的人所希望的——你需要说话。

桑德斯：已故物理学家斯蒂芬·霍金拥有也许是最具标志性的电脑合成声音之一。以下是他在1994年在麻省理工学院演讲的录音。

[霍金音频]电脑语音合成器近年来有了很大的进步，这很重要。人不仅希望被理解，还不希望听起来像米老鼠或戴立克。

桑德斯：他在描述自己的合成声音，事实证明他相当喜欢它。

[霍金音频]我的这个声音可能有点单薄，还带点爱尔兰口音，但它几乎是人类的声音。

桑德斯：霍金对那个声音很满意。朱尔斯则不然。朱尔斯在一段单口喜剧表演中形容他的语音库声音听起来就像把Daft Punk的歌以四分之一的速度播放。

更难、更好、更快、更强。

[桑德斯声音克隆]好，快进一下，人工智能登场了。现在，只需 already 录制好的短小声音片段，这些强大的程序就能创造出合成声音，带有恰当的嗯嗯声、呼吸声、语气和节奏，非常接近原声。这些声音如此逼真，有时你甚至听不出区别。

桑德斯：也许你刚注意到了什么，也许没有。前面那三句话根本不是我说的。它们是由我制作的声音克隆说出来的。好，再来一次。这是我在说话。

[桑德斯声音克隆]而这是我的声音克隆。

桑德斯：我使用ElevenLabs来克隆我的声音。这是一家利用AI技术创建逼真数字声音复制品的公司。为了做到这一点，我使用了专业选项，需要30分钟的音频素材。所以作为输入，我录了自己朗读几段播客脚本。

朱尔斯·罗德里格斯的声音正在消失。

桑德斯：还有一些我妈妈发来的超长短信。

我在Food Lion超市。我在买腌黄瓜和半奶半奶油。

桑德斯：《绿山墙的安妮》第九章。

雷切尔·林德被数落了一顿真是件好事。她是个爱管闲事的老嚼舌根。

桑德斯：几分钟后，噗，我的声音克隆就出来了。它听起来像我到令人毛骨悚然。我花了一个有趣的下午玩它。你可以控制很多不同的方面。

[桑德斯声音克隆]它可以说得很快。也可以说得很慢。我的声音克隆可以笑。可以显得极其无聊。它还可以在对某事不以为然时发出嗤笑。它可以替劳拉发出威胁。

桑德斯：我甚至能说法语。

[桑德斯声音克隆]它甚至能说其他语言。

桑德斯：这项技术在不同阶段使用了几种不同类型的人工智能。它提取我声音的特征，比如我的音高和音调、我的韵律，甚至我的呼吸模式。这使它能够建立我声音的模型，然后这个模型可以被释放出来说出我打出的任何内容。这个主意来自言语病理学家凯特·尼尔森，她自己也尝试过。

尼尔森：我们甚至玩过这样的游戏——到底是我在说话还是电脑在说话？电脑版的我的声音逼真到有点吓人。声音克隆过程真正了不起的地方在于，你拿录音来提交。不是你现在录，而是拿旧的录音。所以对于你认识的人，比如ALS患者，如果他们的言语是最先受影响的，而且他们的言语已经改变了，那可能来不及做录音了，但如果他们有旧的语音留言，或者做过演讲，或者上过播客，类似这样的东西，那我们就可以拿那些录音来提交，制作一个声音。所以这真的为很多人打开了很多扇门。

桑德斯：2024年6月，朱尔斯用自己声音的录音创建了一个AI生成的声音克隆，同样是通过ElevenLabs公司。

朱尔斯：当我第一次听到我克隆的声音时，感觉就像我重新出现了。当我妻子第一次听到时，她哭了，因为它听起来太像失去声音之前的我了。

桑德斯：在几个月里逐渐越来越少听到朱尔斯的声音之后，他的家人已经习惯了新的常态。声音克隆是一个巨大的变化。

朱尔斯：他们一开始都很震惊，但现在感觉很自然了。挺疯狂的。

桑德斯：你刚开始使用声音克隆时，还记得你说的第一句话是什么吗？有没有什么你特别想说的话？

朱尔斯：嗯，可能是最肉麻的那句。我记得告诉妻子我有多爱她。

桑德斯：以下是他的妻子MA的回忆。

MA：他给我播放的第一句话，我全身起鸡皮疙瘩。那真是不可思议地美，不可思议，就像，哇。而且我也哭了，因为感觉他的一部分——我很久以前就失去的那部分——又回来了。

桑德斯：在如此多的失去之中，把这一样东西找回来，出乎意料地艰难。

MA：把那一部分找回来，而他的其他部分却没有回来，情感上让人困惑。就像，我现在说这件事的时候身体都能感觉到。ALS的病情变化太快了，而声音是那么亲密的东西，你一直在一点一点地让自己适应，然后突然间他那个真正充满活力的部分回来了，但其他一切都没变。但那个充满活力的声音部分回来了，就像，是的。那是一种奇异的体验，但最终对我们、对他儿子来说都非常非常特别。

桑德斯：朱尔斯立刻卸掉了辅助轮。

朱尔斯：我只想探索它的极限。

桑德斯：事实上，朱尔斯的声音克隆听起来太好了，以至于他决定自己做单口喜剧的梦想是可能的——而在此之前他早已放弃了这个想法。

朱尔斯：听到它之后不久我就开始用它写段子了。

桑德斯：于是朱尔斯开始准备一套节目。

MA：当他测试AI声音时，就像一个小灯泡亮了，他说，我觉得我可以重新做喜剧了。我记得听到这句话时心想，天哪，这真是件了不起的事，就像你以新的方式活过来了，这是我从他眼中看到的。他说，我觉得我能重新做喜剧了。我当时想，哇。

朱尔斯：哦，天哪，谢谢大家。我都不确定自己能不能来到这里。太多事情可能阻止我出现在这里了。住院、气管切开、电梯坏了。最大的障碍是任何超过两英寸的台阶。我很高兴能和大家在一起。

桑德斯：人们很容易认为这种个性化不应该有多重要，认为有一个声音——任何声音——就够了。但事实并非如此。拥有一个听起来像你真实声音的声音，确实至关重要。

卡琳娜·纳金：你需要表达你在想什么、你的医疗指示。你需要告诉你爱的人你爱他们。但我想我们之前没有意识到，你的声音听起来像你，这对身份认同有多么巨大的影响。

桑德斯：卡琳娜·纳金是“ bridging voice”的负责人，这是一家非营利组织，帮助ALS患者找到并使用沟通技术，包括ElevenLabs的声音克隆。正是这个组织帮助朱尔斯克隆了他的声音。她听说过一些故事：声音克隆以及它们带回的联系让人们重新考虑了自己的医疗指示。

纳金：人们一旦获得了这些声音克隆，它如此改变人生，以至于他们在改变自己的医疗指示，改变他们想要做的事情。很多人都改变了医疗指示。他们说，如果我有这个声音，我能听起来像我自己，我能继续沟通、感觉自己还是自己，那我愿意活得更久。这让我觉得活着值得。我想我们团队之前没有意识到它会有多大的影响，但我们有帮助人们用这些声音克隆念婚礼誓词。前几天我收到一位刚获得声音克隆的祖母的反馈，她说，我立刻打电话给我的孙辈们，其中最小的那个从来没听过我真正的声音。这是他们第一次听到我的声音。

桑德斯：纳金说，她的团队不仅对声音被恢复的人受到的影响感到惊讶，对其家人受到的影响也感到惊讶。

纳金：我们之前太专注于帮助人们获得沟通能力。至于声音是否听起来更像他们，我们说，哦，那是个不错的附加功能，但不是核心。但我们一把它给到人们，就意识到那正是核心——你的身份认同，感觉像你自己。而且这不仅对患者如此，对他们的家人也是如此。

桑德斯：一个个性化的声音，一个听起来像以前的声音，不仅仅是一个附加功能。

纳金：无数的人们录制的视频。我是说，我们有一个视频是一个大学生年纪的男孩两年来第一次听到他父亲的声音，他直接崩溃了。就像，你知道，他们正在经历这个诊断带来的可怕经历。而能够真正与家人联系、再次听到他们的声音，我觉得这给家人带来的快乐和安慰，不亚于给被诊断为ALS的人带来的。

桑德斯：ALS迫使家庭面对关于生死的艰难医疗决策。朱尔斯和MA曾非常仔细地考虑过他们将如何面对未来。他愿意接受什么样的医疗干预，不愿意接受什么样的？从大局来看，什么样的生活值得过？朱尔斯一直靠一个外部氧气面罩来维持身体运转。他已经决定不想迈出下一步——做气管切开术，这是一种延长生命的程序，会在喉咙里留一根小管子连接到呼吸机。有了气管套管，朱尔斯需要24小时护理，而且他将永久失去声音。

MA：他是一个不想做气管切开术的人。就像，他选择的是，一旦我不行了，那就这样吧。那就是我们的计划。

桑德斯：2024年夏天，在他克隆声音之后不久，朱尔斯到了一个非常艰难的关口。

MA：九月，我们又去了医院，有一个晚上我们真的觉得就是那一次了，他那天晚上就要走了。我记得给他家人发短信说，你知道，我不知道是不是现在就发生，但我们，我觉得我们真的很接近了。他经历了这个危机时刻。然后在另一边，当他们能够再次稳定他的时候，他看着我说，宝贝，如果气管切开术能帮我活下去，那我想我会改变主意。我想做。我们跟医生谈谈吧。

桑德斯：决定在喉咙里放置气管套管的这个决定意味着朱尔斯的声音——本已衰弱——将彻底消失。记住，这对夫妻的关系是建立在长时间深入交谈之上的。他们亲密、有趣、有深度的对话对他们如此重要，以至于他们以此为基础创办了一整档播客。但这个决定——一个主动选择完全且永久地失去说话能力——代表着一个硬性停止。说话的明确终结。你即将听到的是一段艰难的对话。MA在跟我讲述手术前的一个时刻。

MA：我记得那天晚上，那天，是的，他的手术是在晚上做的，我录了最后一个我们说话的小视频，因为我想，我只想再捕捉一次你的声音。他说，哦宝贝，你知道，就是那样，我到现在都还没看过那个视频，但它在那里。所以那是一个很大的，一个很大的转变。

桑德斯：是的。

桑德斯：你当然看不到，但也许你能听出来。我们说话的时候，我们两个都在哭。

MA：我想我上次我们谈话时提到过，从一开始我就想，哦，他会失去这个能力、那个能力、这个、这个、这个。我当时想，天哪，当他失去说话能力的时候，我真的真的真的真的真的会很难过。所有那些突然的变化和你必须适应它们的方式，某种程度上让你为下一件事做好了准备。所以通常对事情发生的恐惧比事情真正发生时更甚。然后你到了那一步，你说，好吧，我们到了，我们会适应的。几乎每件事都是这样。但是，是的，那件事很大。

桑德斯：但有一件事让朱尔斯最终完全失去声音变得容易了一些。朱尔斯的新数字声音。这是一个非常逼真的AI生成的对他旧声音的再造——那个他的家人熟悉并喜爱的声音。这个新声音的存在影响了朱尔斯做气管切开的决定。

MA：我真的相信，AI声音以及他能够真正以那种方式沟通的能力，是帮助他决定继续活下去的因素之一。

桑德斯：当然，这不是全部。这样的决定取决于太多变数。但它很重要。

MA：我坚信它对他考虑这个决定产生了巨大的积极影响，比如作为一种方式——你知道，拥有某些目标和经历，他觉得如果没有克隆声音，那些就不会存在。所以我甚至说过，如果没有克隆声音，我不认为他会选择气管切开。是的，然后那个时刻什么都有可能发生，但他一直是个很——很像那种，不，我不想要气管切开的人，从一开始就是。然后所有这些都变了。所以我认为那是一个巨大的推动力。

桑德斯：在朱尔斯住院和手术期间，他们跟儿子斯凯拉谈了正在发生的事情。

MA：整个后续的沟通，因为我说，你知道，现在他要完全失去声音了。他说，我们有电脑。那很好啊。我说，好吧，是的，没错。他随时间告诉我们的那些事情简直不可思议。他看事物的方式、他对事情的见解，令人惊叹。所以那真是一个非常重要的关键时刻——关于用你的声音意味着什么，保住你的声音，为你的声音而战，以你之前甚至没想到的方式表达自己。这需要很大的勇气和意愿。

桑德斯：而你的声音是什么？我是说，对斯凯拉来说，那不重要，因为他爸爸的声音还在。

桑德斯：对朱尔斯的儿子来说，声音克隆就是他爸爸的声音。真的就这么简单。

MA：他在十月做了气管切开术，十月一号，然后他在医院又待了两周。然后，十一月中旬，他就在我们当地一家喜剧俱乐部的即兴表演中登台了。

朱尔斯：说正经的，谢谢大家来。我叫朱尔斯，因为AI的进步，我现在有能力重新站在这里，用我原来的声音做这件事。

桑德斯：朱尔斯回来了。朱尔斯，带着他的声音克隆，可以笑、可以唱歌，可以跟妻子一起惹麻烦。

朱尔斯：特别值得一提的是它如何在词语上加入语调和语气变化。有时候我妻子会生气，因为我的回答可能带着点傲娇。

桑德斯：我问朱尔斯能不能用它说脏话。

朱尔斯：[哔]可以。用最新版本的这个技术，我可以通过提示词更好地控制我的声音。包括笑、耳语，甚至唱歌。我以前从来不会唱歌，尽管我和妻子是在卡拉OK酒吧认识的。

桑德斯：我觉得你能唱。任何想唱的人都能唱。

MA：他完全能唱。

桑德斯：凯夫做言语治疗师已经几十年了，但就连他也被这些更新的AI生成声音克隆的能力所折服。

凯夫：我真的很惊喜。这些声音是我从这项技术中听到过的最真实的。它真的听起来就像我在合作的那些人。所以它真的改变了我的世界。因为我在语音库方面有些名气，我收到了很多咨询。我最终花了大量时间回答关于ElevenLabs的问题。真正让我兴奋的是，很多人反馈说他们的社交网络再次得到了加强，更多的人来跟他们聊天，回顾过去的时光和现在的时光。这对我来说就是金子。这就是语音库的意义所在，或者说应该有的意义——维持社交和工作网络。

桑德斯：我想说，我们现在已经远远超越了霍金所说的“几乎是人类的声音”。今天的声音克隆流畅、个性化，在很多方面无缝衔接。我的合成声音说某些词的方式和我说的一模一样。

[桑德斯声音克隆]我的真实声音说某些词的方式和我说的一模一样。真瘆人。

桑德斯：我听到的那些关于人们如何使用这些声音的故事证实了这一点。我和佩吉·梅里尔通了邮件，她因ALS失去了声音。她用声音克隆给孙辈读书、教瑜伽、带领引导式冥想。

佩吉：我叫佩吉，我很荣幸与你们分享这段引导式瑜伽睡眠冥想。

桑德斯：佩吉给我写道：“使用并听到自己的声音，让我重新找回了自我感。”还有帕特里克·达林，你在之前某一集里听到过他的声音。他是一位作曲家和音乐人，一直在用声音克隆写新歌并表演。

帕特里克·达林：第一次用AI声音表演时，我很惊讶自己感到与它在身体上有那么强的联系，尽管我实际上并没有在唱。

[音乐]

帕特里克：很难解释。就是有一种奇怪的能量，你感觉自己又成了一个表演者。

桑德斯：帕特里克描述的那种与声音在身体上相连的感觉，我觉得可能是一个重要的线索，说明为什么这项声音克隆技术如此强大。这些逼真的声音能够在人内心以及人与人之间建立联系。这些联系曾经失去了，然后又重新建立起来。它们之所以重要，不仅因为那些沉重的大对话，也因为日常的小事。

朱尔斯仍然需要使用眼动追踪技术来打出他想说的话。他有一个设备，能感应他在屏幕上看哪里，用他的目光来书写。只有到那时，他的声音克隆才会说出他想说的话。这需要大量时间和专注。科学家们正在研究通过利用言语过程中更早期的信号来更快地输出词语的方法。

在下一集中……研究人员想直接从大脑中提取词语。

凯西·哈雷尔：我读得越多就越想，为什么不是我？为什么不是现在？

桑德斯：把自己的想法变成词语是什么感觉？

利·霍赫伯格：他说的第一句话大概是，你知道，这玩意儿开着吗？然后我们解码出的第二句话是，我在找一只猎豹。而这才是第二天。你知道，使用植入式脑机接口第二天的第二句话，不可能说什么猎豹的事。你知道，那显然是解码错误，那没关系。只不过他四岁的女儿刚穿着猎豹服装走进了房间。

桑德斯：这里是《深渊：跟我说话》。我是劳拉·桑德斯。如果你喜欢这档播客，告诉你的朋友，给我们留个评价。这对节目帮助很大。

把你们的问题和评论发到podcasts@sciencenews.org。

《深渊》是《科学新闻》的制作。它基于我——劳拉·桑德斯的原创报道。本集由艾拉·罗恩制作和混音。我们的项目经理是阿什利·耶格尔。南希·舒特是我们的主编。我们的音乐由Blue Dot Sessions提供。本播客部分得益于伯勒斯·韦尔科姆基金会的资助以及PRX的支持。

本播客在伯勒斯·韦尔科姆基金会的资助下制作，并与PRX合作。播客标志由尼尔·韦布设计。

英文来源：

AI can give you your voice back
When people face losing the ability to talk, they need alternative ways to communicate. One way people preserve their voice is by banking it. The process has improved in recent years, but it still produces a voice that can sound unnatural. Jules Rodriguez did this as amyotrophic lateral sclerosis, or ALS, left his own voice diminished. But he wasn’t happy with the sound.
New technology that uses artificial intelligence allowed Jules to create a voice clone, one that sounds like his old voice. With breaths, intonations and even stutters, these voice clones capture the quirks of a human voice. Jules’ voice clone was so realistic that his wife cried the first time she heard it.
In this episode, hear how AI-powered voice clones are changing the conversation around how people connect. That includes a musician named Patrick Darling who uses his voice clone to sing and perform songs.
Episode 3 transcript
Laura Sanders: Jules Rodriguez’s voice was disappearing.
Jules Rodriguez: For me, it felt like I was fading away, like when Marty McFly was seeing his family fade away in the picture he had in “Back to the Future.”
Sanders: Jules has ALS, and like many people with that disease, this fading felt inevitable. But there was something Jules could do about it. I’m Laura Sanders, and this is episode three of The Deep End: Talk To Me, a Science News podcast on the human voice and all it can hold.
[voices] Hey mom, can I tell you something? The voice is absolutely an integral part of your identity. Do you want to talk? That added layer of it being you. Hello? Your voice. Hi.
Sanders: On today’s episode, we’re going to hear about the return of Jules’ voice.
[voice] Your voice is you.
We touched on this in the last episode, but as a reminder, there are a few traditional ways people preserve their ways to communicate when they’re faced with losing a voice. One is for people to record themselves saying common or special phrases if they can still talk. That’s called message banking. And then there’s a technique called voice banking. It requires a person to record themselves saying very specific words. And then these sounds get fed into a computer program that can stitch them together to make any word a person wants to say. Sounds good in theory, but in practice, the output didn’t sound very human. Jules banked his voice soon after his diagnosis. MA describes what that was like.
MA: I remember both of us looking at each other while he was recording, basically the old school way, and they still recommend to bank your voice this way, besides your cloned voice, but you’re sent all these phrases that you’re just recording, like 350 phrases, and that goes into, you know, a file, and then when you do need it, the company will pull it up and plug it into your device.
Sanders: I spoke with Richard Cave. He’s a speech therapist at University College London who works with people with ALS. That’s also called motor neuron disease, or MND.
Richard Cave: My role was to build out the voice banking initiative across the UK. This was back in 2017. And I remember… After the first, I don’t know, six months, nine months, being very proud that I managed to put through 100 people or so at an average duration to complete the voice banking, all 1,600 phrases in around three months each. Very proud of that. They weren’t awful, but they weren’t good. And they didn’t really represent the person. They represented kind of part, a little bit, of the person. But that was the best we had.
Having decoded the signals from some input device, such as the switch, one then has to broadcast it in a form people can understand. This could be visually, with text or diagrams on the screen. But if you want to communicate like others, which is what people in my position would like, you need to speak.
Sanders: The late physicist Stephen Hawking had one of perhaps the most iconic computerized voices. Here he is giving a talk in 1994 at MIT.
[Hawking audio] Computerized speech synthesizers have improved a great deal in recent years, and this is important. Not only does one want to be understood, but also one doesn’t want to sound like Mickey Mouse or a Dalek.
Sanders: He’s describing his synthesized voice, which it turns out, he quite liked.
[Hawking audio] This voice of mine may be a bit tinny and Irish, but it is almost human.
Sanders: Hawking was happy with that voice. Jules was not. Jules described his banked voice in a stand-up routine as sounding like if you played Daft Punk at quarter speed.
Harder, better, faster, stronger.
[Sanders voice clone] Okay, fast forward a bit, artificial intelligence came onto the scene. Now, with just short snippets of already recorded sound, these powerful programs can create synthetic voices with appropriate umms, breaths, sounds, and cadence that are very close to the original. These voices are so realistic that sometimes you can’t even hear the difference.
Sanders: Maybe you just noticed something, but maybe you didn’t. Those previous three sentences weren’t spoken by me at all. They were said by a voice clone I made. Okay, one more time. This is me talking.
[Sanders voice clone] And this is my voice clone.
Sanders: I used ElevenLabs to clone my voice. That’s a company that uses AI technology to create lifelike, digital replicas of voices. To do it, I used the professional option, which required 30 minutes of audio to make. So for input, I recorded myself reading a few podcast scripts.
Jules Rodriguez’s voice was disappearing.
Sanders: Some really long texts from my mom.
I’m at Food Lion. I’m getting pickles, half and half.
Sanders: And chapter nine of Anne of Green Gables.
It’s a good thing Rachel Lynde got a calling down. She’s a meddlesome old gossip.
Sanders: After a few minutes, poof, there it was, my voice clone. It was spooky how much it sounded like me. I spent a fun afternoon playing with it. You can control so many different parts.
[Sanders voice clone] It can speak quickly. And it can speak slowly. My voice clone can laugh. It can be utterly bored. It can also scoff when it thinks something is ridiculous. It can be menacing on Laura’s behalf.
Sanders: I could even speak fluent French.
[Sanders voice clone] Il peut même parler d’autres langues.
Sanders: The technology uses a few different types of artificial intelligence at different stages. It extracts features of my voice, such as my pitch and tone, my prosody, even my breathing patterns. This lets it build a model of my voice, and then that model can be let loose to say anything I type. I got the idea from speech pathologist Kate Nilson, who has tried it herself.
Nilson: We’ve even played the game of like, is it really me talking or is it the computer? And it’s kind of scary how realistic the computerized version of my voice is. The really awesome part about the voice cloning process is that you take recordings and submit that. It’s not like you record it right now. You take old recordings. And so for people you know for instance people living with ALS, if their speech is the first thing that’s affected and their speech has already changed, it might be too late to do the recordings, but if they have old voicemails or a speech that they gave or a podcast that they were on, something like that, then we can take those recordings and submit them to make a voice. And so that’s really opened a lot of doors for a lot of people.
Sanders: In June of 2024, Jules used recordings of his own voice to create an AI-generated voice clone, also through the company ElevenLabs.
Jules: When I first heard my cloned voice, it felt like I reappeared. When my wife first heard it, she cried because of how close it sounded like me before I lost my voice.
Sanders: After months of gradually hearing Jules’ voice less and less, his family had gotten used to the new normal. The voice clone was a big change.
Jules: They were all shocked at first, but now it just feels natural. It’s pretty crazy.
Sanders: When you first started using the voice clone, do you remember what the first thing you said was? Were there things you really wanted to say?
Jules: Well, it was probably the cheesiest thing. I remember telling my wife how much I love her.
Sanders: Here’s how his wife MA remembers it.
MA: The first thing he played for me, I was like chills all over. And it was, it was incredibly beautiful and incredible and like, wow. And also I cried because it felt like there was a part of him that I was getting back that I had lost a long time ago.
Sanders: It was surprisingly hard to get this one thing back in the middle of so much loss.
MA: It felt confusing emotionally to get that part back and not the rest of him back. Like, I just I feel it right now in my body as I talk about it. Things with ALS change so quickly and the voice is something so intimate you’re sort of like, you’re training yourself little by little to adapt and then all of a sudden this like really vibrant part of him came back, but everything else was the same. But that vibrant voice part came back and it was like, yeah. It was a bizarre experience but ultimately really, really special for us and for his son too.
Sanders: Jules immediately took off the training wheels.
Jules: I just wanted to explore the limits of it.
Sanders: In fact, Jules’ voice clone sounded so good that he decided his dream of doing stand up comedy was possible, long after he had given up the idea.
Jules: I started to write jokes with it pretty soon after hearing it.
Sanders: So Jules started working on a set.
MA: When he tested the AI voice, it’s like a little light bulb went off and he was like, I think I could do comedy again. And I remember hearing that and going like, oh, my God, like that’s such a huge like thing to, to, it’s like you come alive in a new way, is what I saw through his eyes. He was like, I think I can do comedy again. I was like, wow.
Jules: Oh, man, thank you. I wasn’t even sure if I was going to make it out here. So many things could stop me from being here. Hospital stays, tracheostomy, broken elevator. And the biggest thing would be any step larger than two inches. I’m so happy to be here with you all.
Sanders: It’s tempting to think that this personalization shouldn’t matter much, that having a voice, any voice, would do the job. But it’s not true. Having a voice that sounds like your real voice does matter a tremendous amount.
Karina Nagin: You need to communicate what you’re thinking, your health directives. You need to tell your loved ones you love them. But I think we didn’t realize what a huge impact your voice sounding like you, what a big piece of identity that was.
Sanders: Karina Nagin directs Bridging Voice, a nonprofit organization that helps people with ALS find and use communication technology. That includes voice clones from ElevenLabs. It’s the group that helped Jules clone his voice. She’s heard stories about how voice clones and the connections they bring back led people to reconsider their medical directives.
Nagin: And people, once they received these voice clones, it was so life-altering that they were changing their health directives of what they wanted to do. Many people changed their health directives. They said, if I have this voice and I can sound like me, I can continue to communicate and feel like myself, I want to live longer. This is something that makes it worth living for me. I think our team didn’t realize how impactful it was going to be, but we’ve had people use these voice clones to tell their wedding vows. I got a testimonial from a grandmother the other day who just got her voice clone, and she said, I immediately called up my grandchildren, the youngest of who had never heard my real voice. And so this is the first time they were hearing my voice.
Sanders: Nagin says her team was surprised by the impact on not just the person whose voice is restored, but on their families too.
Nagin: We were so focused on people and giving them the ability to communicate. The idea that it sounds more like them or not, we said, oh, that’s a nice add-on, but that’s not really the core of it. But as soon as we gave that to people, we realized that is the core of it, your identity and feeling like yourself. And it’s not just for the patients, but it’s their family members.
Sanders: A personalized voice, one that sounds like the old one, is more than an add-on.
Nagin: Endless videos of people. I mean, we have one of a college-aged boy who’s hearing his father’s voice for the first time in two years, and he just breaks down. Like, this is, you know, they’re going through this horrific experience of this diagnosis. And to be able to actually connect with a family member and hear their voice again, I think it equally brings joy and comfort to the families as much as it does to the person who’s been diagnosed with ALS.
Sanders: ALS forces families to confront tough medical decisions about life and death. Jules and MA had thought very carefully about how they’d approach the future. What kind of medical interventions would he be OK with, and what kind would he not want? Big picture, what sort of life was worth living? Jules had been getting by with just an external oxygen mask to help keep his body going. He had decided that he didn’t want to take a next big step of having a tracheostomy, a life-extending procedure that leaves a small tube in his throat connected to a ventilator. With a trache tube, Jules would need round-the-clock care, and he would lose his voice permanently.
MA: He was someone who didn’t want to do a tracheostomy. Like, he was opting for, once I’m done, like, that’s it. And that was our plan.
Sanders: In the summer of 2024, not too long after he had cloned his voice, Jules had reached a very tough spot.
MA: September, we went to the hospital again and we had one evening where we really thought like that was it, like he was going to, he was going to pass away that night. I remember texting his family and saying, you know, I don’t know that it’s going to happen right now, but we’re, I feel like we’re really close. And he went through this like crisis moment. And then on the other side, when they were able to like stabilize him again, he looked at me and he was like, babe, well, if the tracheostomy can help me live, then I think I’d change my mind. I want to do it. Let’s talk to the doctors about it.
Sanders: This decision to have a trache tube placed in his throat meant that Jules’ voice, diminished as it already was, would be gone. Remember, this is a couple who built their relationship on long, deep talks. Their intimate, funny, thoughtful conversations were so meaningful to them that they started an entire podcast based on them. But this decision, a proactive choice to fully and permanently lose the capacity to talk, represented a hard stop. A definitive end to talking. What you’re about to hear is a tough conversation. MA is telling me about a moment before the surgery.
MA: So I remember the night, the day, yeah, he was, the operation was like in the evening when they did it, and me recording like a last little video of us talking because I was like, I just want to capture your voice one more time. And he was like, oh babe, you know, being all like, and I haven’t even looked at that video, but it’s there. And so that was a big, it was a big transition.
Sandes: Yeah.
Sanders: You can’t see it, of course, but maybe you can hear it. As we talk, we’re both crying.
MA: I think I mentioned when we spoke last time how of all the things from the very beginning that I was like, oh, he’s going to lose the ability to this and this and this and this and this. I was like, oh, my God, when he loses the ability to speak, like, I’m going to really, really, really, really, really have a hard time. And all of the abrupt changes and the way you have to adapt to them kind of prepare you for the next things. And so usually the fear of the thing happening is more than when it actually happens. And then you’re there and you’re like, okay, we’re here and we’ll adapt to it. That’s usually been the case for pretty much everything. But yeah, it was big.
Sanders: But there was something that made the final and complete loss of Jules’ voice easier. Jules’ new digital voice. This was a very realistic AI-generated recreation of his old voice, the one that his family knew and loved. The existence of this new voice factored into Jules’ decision to get a trache.
MA: I really do believe that the AI voice and the ability for him to actually communicate in that way is one of the things that helped him decide to keep living.
Sanders: It wasn’t everything, of course. A decision like this depends on so many moving pieces. But it was big.
MA: I firmly believe that it had an enormous positive impact in him considering it, like, as a way of you know having certain goals and experiences that he thought, you know, wouldn’t have been there, had it not been for the cloned voice. So I’ve I’ve gone as far to say that if it wasn’t for the clone voice, I don’t think he, I don’t think he would have chosen the trache. Yeah, then that moment anything could happen, but he was pretty, a pretty like no, I don’t want the tracheostomy kind of guy from the beginning. And then all of that changed. And so I think it was a huge driver.
Sanders: At the time of Jules’ hospitalization and surgery, they talked to their son Skylar about what was going on.
MA: The whole like communicating with him afterwards, because I said, you know, now he’s going to lose his voice completely. He’s like, we have the computer. That’s great. And I was like, okay, yeah, true. The things he’s told us over time are like bonkers. How he sees, the perspective he has on things, is incredible. So that was a really big, pivotal time for like what it means to like use your voice, to keep your voice, to fight for your voice and to express yourself in ways that you didn’t even think you would before. That takes a lot of courage and willingness.
Sanders: And what is your voice? I mean, to Skylar, it didn’t matter because his voice was still there.
Sanders: To Jules’ son, the voice clone was his dad’s voice. That’s really all there was to it.
MA: He had his tracheostomy October, October 1st, and he was in the hospital for another two weeks. And then, mid-November, he was performing at, you know, an improv at one of our local comedy clubs.
Jules: In all seriousness, thanks for coming out. My name is Jules and because of the advancements in AI, I now have the ability to come up here and do this again with my original voice.
Sanders: Jules was back. Jules, with his voice clone, can laugh and sing and start trouble with his wife.
Jules: One thing in particular is how it inserts tone and inflections on words. Sometimes my wife will get mad because my response can come off with some sass.
Sanders: I asked Jules if he can swear with it.
Jules: [Beep] yeah, I can. With the latest version of this technology, I have more control of how I sound with prompts. That includes laughing, whispering, and even singing. I could never sing, even though my wife and I met at a karaoke bar.
Sanders: I think you can sing. Anyone who wants to sing can sing.
MA: He could totally sing.
Sanders: Cave has been a speech therapist for decades, but even he was won over by the capabilities of these newer AI-generated voice clones.
Cave: I was really pleasantly surprised. These voices were the most authentic sounding I’d ever heard from this technology. It actually sounded like the people I was working with. And so it really changed my world. And because I was known for voice banking, I was getting a lot of inquiries. And I ended up spending so much time on answering questions about ElevenLabs. What really, really excited me was that many people reported that their social networks was being strengthened again, that more people were coming to talk to them and to reflect on times past and times current. And that to me is gold. That’s what voice banking is or should be, to maintain social and work networks.
Sanders: I’d say that we’ve now gone well beyond almost human, as Stephen Hawking put it. Today’s voice clones are fluid, individual, and in many ways, seamless. My synthetic voice says certain words the exact same way I would say them.
[Sanders voice clone] My real voice says certain words the exact same way I would say them. It’s spooky.
Sanders: The stories I’ve heard of how people are using these voices back this up. I emailed with Peggy Merrill, who lost her voice from ALS. She uses her voice clone to read to her grandkids, teach yoga, and lead guided meditation.
Peggy: My name is Peggy and it is my honor to share this guided yoga nidra meditation with you.
Sanders: Peggy wrote to me, “It renews my sense of self to use and hear my own voice.” And there’s Patrick Darling, who you heard from in an earlier episode. He’s a composer and musician, and he’s been writing new songs and performing them with his voice clone.
Patrick Darling: The first time I performed with the AI voice, I was surprised how physically connected I felt with it, even though I was not actually singing.
[music]
Patrick: It’s hard to explain. There’s just this strange energy about it and you feel like a performer again.
Sanders: Patrick’s description of feeling physically connected to his voice, I think, might be a big hint about why this voice cloning technology is so powerful. These realistic voices are able to create connections both within people and between people. These connections were lost, and then they’re remade. And they matter because of not just the big heavy conversations, but the little everyday stuff too.
Jules still has to use eye-tracking technology to type out what he wants to say. He has a device that senses where he’s looking on a screen and uses his gaze to write. Only then does his voice clone say what he wants it to say. That takes a lot of time and focus. Scientists are working on ways to get the words out faster by using signals earlier in the speech process.
On the next episode…researchers want to pull words straight from the brain.
Casey Harrell: And the more I read and I thought, why not me? And why not now?
Sanders: What does it feel like to have your thoughts turned into words?
Leigh Hochberg: The first thing that he said was something along the lines of, you know, is this thing on? And then the second thing that we decoded was, I’m looking for a cheetah. And it’s the second day. You know, there’s no way that the second sentence on the second day of using an implanted brain computer interface is going to say anything about a cheetah. You know, that’s obviously a decoding error, and that’s fine. Except for the fact that his four-year-old daughter had just walked into the room dressed as a cheetah.
Sanders: This is The Deep End. Talk to me. I’m Laura Sanders. If you like this podcast, tell your friends and leave us a review. It helps the show a lot.
Send us your questions and your comments at podcasts@sciencenews.org.
The Deep End is a production of Science News. It’s based on original reporting by me, Laura Sanders. This episode was produced and mixed by Ella Rowen. Our project manager is Ashley Yeager. Nancy Schute is our editor in chief. Our music is by Blue Dot Sessions. The podcast is made possible in part by the Burroughs Wellcome Fund with support from PRX.
This podcast was produced with support from the Burroughs Wellcome Fund and in partnership with PRX. The podcast logo was created by Neil Webb.