内容来源：https://techcrunch.com/2026/09/07/artificial-intelligence-definition-glossary-hallucinations-guide-to-common-ai-terms/

内容总结：

AI新术语速查手册：从AGI到RAMageddon，一文读懂人工智能“黑话”

随着人工智能以前所未有的速度重塑世界，它也在同步创造一套全新的语言体系来描述自身的发展。如今，无论是产品会议、融资路演还是行业论坛，你都会频繁听到诸如LLM（大语言模型）、RAG（检索增强生成）、RLHF（基于人类反馈的强化学习）等术语。上周，随着OpenAI发布其Astra模型，诸如“不透明循环”（一种引发AI安全研究人员担忧的推理技术）等新词汇又引发了新一轮热议。技术词汇更新之快，甚至让科技圈内不少顶尖聪明人也感到一丝焦虑。

为此，本刊特整理这份AI术语速查手册，旨在用通俗易懂的英文为读者解释最可能遇到的AI专业词汇。无论你是开发者、投资人，还是仅通过阅读科技新闻或收听播客来紧跟潮流，这份手册都适用。随着技术领域不断演进，本手册也将定期更新，宛如它所描述的AI系统本身，是一份“活文档”。

AGI（通用人工智能）

这是一个定义模糊的术语，通常指在大多数任务上（甚至几乎所有任务上）能力超越普通人类的AI。OpenAI CEO萨姆·奥尔特曼曾将其描述为“可以雇佣为同事的普通人类水平”。而OpenAI章程则将其定义为“在最具经济价值的工作中表现优于人类的高度自主系统”。Google DeepMind的理解又略有不同，认为是“在大多数认知任务上至少与人类能力相当的AI”。概念混淆？不必担心，即便是AI研究前沿的专家们也对此莫衷一是。

AI Agent（智能体）

指使用AI技术代表用户执行一系列任务的工具，其能力远超基础聊天机器人，例如处理报销、预订机票或餐厅、甚至编写和维护代码。不过，正如我们此前所述，这一新兴领域变量众多，“AI Agent”对不同人可能意味着不同事物。其基础设施仍在建设中，但基本概念是指一个自主系统，可调用多个AI系统来完成多步骤任务。

API端点

可以把API端点想象成软件“背面”的“按钮”，其他程序可以“按下”这些按钮来触发功能。开发者利用这些接口构建集成，例如让一个应用拉取另一个应用的数据，或让AI Agent无需人工手动操作即可直接控制第三方服务。大多数智能家居设备和联网平台都有这些隐藏按钮，普通用户通常看不到也用不到。随着AI Agent能力增强，它们正越来越擅长自主发现和使用这些端点，释放出强大且有时出人意料的自动化可能。

思维链

面对简单问题，人脑可以不假思索地回答。但在很多情况下，需要纸笔辅助，因为存在中间步骤。在AI语境下，大语言模型的思维链推理意味着将一个难题分解为更小、更中间的步骤，以提高最终结果质量。这通常需要更长的回答时间，但答案正确率更高，尤其在逻辑或编程领域。推理模型由传统大语言模型发展而来，并通过强化学习针对思维链进行优化。

编程Agent

此为“AI Agent”的更具体概念，指能自主分步行动实现目标的程序。编程Agent是其应用于软件开发的专业版本。它不仅仅是给人类审查粘贴的代码建议，而是能自主编写、测试和调试代码，处理通常占据开发者日常工作的迭代试错任务。这些Agent能操作整个代码库，发现问题、运行测试并在最少人工监督下推送修复。可以想象成雇佣了一位永不疲倦、永不涣散的极速实习生——当然，任何实习生的工作都需要人工复核。

算力

虽然是个多义词，但算力通常指支撑AI模型运行的关键计算能力。这种处理能力驱动着AI产业，使其得以训练和部署强大的模型。该词常作为提供算力的硬件类型的简称，如GPU、CPU、TPU以及构成现代AI产业基石的其他基础设施。

深度学习

机器学习中一种自我改进的子集，其AI算法采用多层人工神经网络结构。相比线性模型或决策树等更简单的机器学习系统，深度学习能做出更复杂的相关性分析。其算法结构灵感源自人脑神经元相互连接的路径。深度学习模型能自行识别数据中的重要特征，无需人工工程师定义。该结构还支持算法从错误中学习，通过重复和调整过程改进自身输出。然而，深度学习系统需要海量数据（数百万甚至更多）才能产出好结果，训练时间通常也比简单算法长，因此开发成本更高。

扩散

扩散是许多生成艺术、音乐和文本AI模型的核心技术。受物理学启发，扩散系统通过添加噪声“破坏”数据结构，直至其完全消失。在物理学中，扩散是自发且不可逆的——溶于咖啡的糖无法恢复成方糖。但AI中的扩散系统旨在学习一种“逆扩散”过程以恢复被破坏的数据，从而获得从噪声中还原数据的能力。

蒸馏

蒸馏是一种使用“师生模型”从大型AI模型中提取知识的技术。开发者向教师模型发送请求并记录输出，有时会将答案与数据集比较以检验准确性。这些输出随后用于训练学生模型，使其学会模拟教师行为。蒸馏可用于在最小精度损失下，基于大模型创建更小、更高效的模型。这可能是OpenAI开发GPT-4 Turbo（GPT-4的更快版本）的方式。虽然所有AI公司都在内部使用蒸馏，但也可能被某些公司用来追赶前沿模型。从竞争对手处蒸馏通常违反AI API和聊天助手的服务条款。

微调

指对AI模型进行进一步训练，以优化其在更特定任务或领域的性能，通常方式是输入新的、专门的（即面向任务的）数据。许多AI初创公司以大语言模型为起点构建商业产品，通过补充早期训练周期，利用自身领域知识进行微调，力求提升在目标领域或任务中的实用性。

GAN（生成对抗网络）

一种机器学习框架，支撑着生成式AI在生成逼真数据（包括但不限于深度伪造工具）方面的若干重要发展。GAN使用一对神经网络，一个利用其训练数据生成输出，另一个对其进行评估。两个模型本质上被设定为相互竞争：生成器试图让其输出骗过判别器，而判别器则努力识别人工生成的数据。这种结构化对抗可以在无需额外人工干预的情况下，优化AI输出使其更逼真。不过GAN最适合较窄的应用（如生成逼真照片或视频），而非通用AI。

幻觉

幻觉是AI行业对模型编造内容——即字面意义上生成错误信息——的行业术语。显然，这是影响AI质量的巨大问题。幻觉会导致生成式AI输出具有误导性，甚至可能带来现实风险，后果堪忧。AI编造信息的问题被认为源于训练数据的缺口。幻觉问题正在推动AI向更专业化或垂直化方向发展，即通过缩小领域减少知识空白，降低虚假信息风险。

推理

推理是运行AI模型的过程，即让模型基于已有数据进行预测或得出结论。需要明确的是，没有训练就没有推理；模型必须先学习数据集中的模式，才能有效从训练数据中进行推断。许多硬件都能执行推理，从智能手机处理器到强力GPU再到定制的AI加速器。但不同硬件运行模型的能力不同。超大模型在笔记本电脑上预测可能耗时极久，而在配备高端AI芯片的云服务器上则快得多。

大语言模型（LLM）

大语言模型是ChatGPT、Claude、Google Gemini、Meta Llama、Microsoft Copilot或Mistral Le Chat等流行AI助手使用的AI模型。当你与AI助手聊天时，实际上是在与大语言模型交互，它直接处理你的请求，或借助网络搜索、代码解释器等不同工具。

LLM是由数十亿数值参数构成深度神经网络，学习词语间关系并创建语言表征——一张多维词语地图。这些模型通过编码数十亿册书籍、文章和抄本中的模式创建。当你给LLM提示时，模型会生成最适合该提示的模式。

高速缓存

缓存指提升推理效率的重要过程。本质上，缓存是一种优化技术，旨在让推理更高效。AI由高强度的数学计算驱动，每次计算都消耗更多电力。缓存通过保存特定计算结果以应对未来用户查询和操作，减少模型必须运行的计算次数。有不同类型的缓存，其中较知名的是KV（键值）缓存。KV缓存适用于基于Transformer的模型，通过减少生成答案所需的时间（和算法劳动）来提高效率、加速结果输出。

模型上下文协议（MCP）

MCP是一个开放标准，允许AI模型连接外部工具和数据——如你的文件、数据库或Slack、Google Drive等应用——而无需开发者为每次配对构建自定义连接器。可将其视为AI的USB-C接口。Anthropic于2024年推出MCP，后移交Linux基金会，现已被OpenAI、Google和Microsoft采用，成为近期AI历史上传播最快的标准之一。

混合专家模型（MoE）

这是一种将神经网络拆分为多个更小的专业子网络（即“专家”）的模型架构，只针对特定任务激活其中少数几个。MoE模型并非像为每个问题召集整个办公室那样，而是通过内置的“路由器”只挑选合适的专家处理任务。这使得构建庞大模型的同时保持相对快速和低运行成本成为可能，因为任何时刻只有一小部分网络在工作。Mistral AI的Mixtral模型是知名例子；OpenAI较新的GPT模型也被广泛认为使用了类似方法，但官方从未确认。

神经网络

神经网络指支撑深度学习、并更广泛地支撑大语言模型出现以来整个生成式AI热潮的多层算法结构。虽然从人脑密集连接的路径中汲取灵感的算法设计思想可追溯至上世纪40年代，但真正释放这一理论力量的是近年来图形处理硬件（GPU）在视频游戏行业的兴起。这些芯片被证明非常适合训练此前无法想象的深层算法，使基于神经网络的AI系统在语音识别、自动驾驶、药物发现等多个领域都取得了更优表现。

神经语（Neuralese）

一种假设的最坏情况，即模型完全在其内部数字表征而非人类可读语言中进行推理，使其思维成为完全的黑匣子。目前尚无已发布的模型会这样做——OpenAI表示其Astra模型保持了思维链的清晰可读，并反驳了与“神经语”的类比。但安全研究人员指出，Astra使用“不透明循环”技术（定义见下文）是朝该方向迈出的真实第一步，这也是该词在相关报道后迅速引发关注的原因。

不透明循环

“不透明循环”指AI模型将同一查询反复循环通过其内部层，而非用平实的语言逐步推理。这更高效——较小的模型可以用较少的算力发挥超常水平——但留下的可读痕迹远少于正常思维链。这让安全研究人员担忧，因为这些日志是捕捉模型不当行为的关键工具，而该技术可能使监督变得困难得多。

开源

开源指软件——或越来越多的AI模型——的底层代码向公众开放，任何人都可以使用、检查或修改。在AI领域，Meta的Llama系列是突出例子；Linux是操作系统领域的著名历史类比。开源方式让世界各地的研究人员、开发者和公司能在他人的工作基础上构建，加速进步，并实现封闭系统难以提供的独立安全审计。闭源则意味着代码私有，你可以使用产品但无法了解其运作方式，这正是OpenAI GPT模型的情况。

并行化

并行化指同时做多件事，而非一件接一件——好比让10名员工同时负责项目的不同部分，而不是一人顺序完成所有工作。在AI中，并行化对训练和推理都至关重要：现代GPU专为并行执行数千次计算而设计，这正是它们成为行业硬件支柱的重要原因。随着AI系统复杂度提升和模型规模扩大，跨多芯片、多机器并行工作的能力已成为决定模型构建与部署速度和成本效益的最关键因素之一。更好的并行化策略研究现已成为独立的研究领域。

内存危机（RAMageddon）

这是一个有趣的新词，用来描述席卷科技行业的不那么有趣的趋势：随机存取存储器（RAM）芯片持续短缺。RAM支撑着我们日常生活中几乎所有科技产品。随着AI产业蓬勃，大型科技公司和AI实验室都在争夺最强大高效的AI，导致它们为数据中心大量采购RAM芯片，留给其他行业的不多了。供应瓶颈意味着剩余芯片价格越来越高。受影响行业包括游戏业（主要厂商因难以为设备找到存储芯片而被迫提高主机价格）、消费电子（存储短缺可能导致智能手机出货量出现十多年来最大跌幅）以及一般企业计算（企业无法为数据中心获得足够内存）。价格飙升需待短缺结束后才会停止，但遗憾的是目前尚无迹象表明这很快就会发生。

循环深度

这是对“不透明循环”背后底层方法的更“技术性”名称。媒体报道几乎可以互换使用这两个词，这也是我们将其收录的原因。“循环深度”是工程术语，“不透明循环”则是强调安全担忧的表述。

递归自我改进

与AGI类似，递归自我改进是AI智慧程度和减少依赖人类程度的一个门槛。在这个设想场景中，AI模型开始在没有人类干预的情况下自我改进，导致能力和自主性大幅加速。在某些叙事中，这将是堪比奇点（AI模型变得不受外部干预）的变革性时刻。但递归自我改进也描述了一种基础能力——AI模型能否设计出自己的继任者？——这使得工程师更容易尝试构建。近期多家AI初创公司已着手构建递归自我改进的模型，但多数淡化了世界末日式影响，仅将其描述为下一个研究前沿。

强化学习

一种训练AI的方式：系统通过尝试并从正确答案中获得奖励来学习，就像用零食训练宠物一样——只是这里“宠物”是神经网络，“零食”是表示成功的数学信号。与在固定标签数据集上训练的监督学习不同，强化学习让模型探索环境、采取行动，并根据收到的反馈持续更新行为。这种方法在训练AI玩游戏、控制机器人方面尤为强大，最近还被用于增强大语言模型的推理能力。基于人类反馈的强化学习（RLHF）等技巧，现已成为领先AI实验室微调模型以使其更友善、准确和安全的核心方法。

Token（词元）

人机交流存在明显挑战——人类用自然语言，AI程序则通过复杂算法和数据驱动流程执行任务。Token弥合了这种鸿沟：它们是人与AI交流的基本构件，代表LLM处理或产生的离散数据段。Token通过“分词”（tokenization）过程创建，将原始文本分解为语言模型可消化的小单位，类似于编译器将人类语言翻译成计算机能理解的二进制代码。在企业环境中，Token还决定成本——大多数AI公司按Token计费LLM使用，用得多花得多。

Token吞吐量

Token是AI语言模型处理前将语言分解成的小文本块，理解AI工作负载时可粗略类比为“词汇”。吞吐量指给定时间内可处理的数量，因此Token吞吐量本质上衡量系统能同时处理的AI工作总量。高Token吞吐量是AI基础设施团队的关键目标，因为它决定模型能同时服务的用户数及每位用户的响应速度。AI研究员Andrej Karpathy曾形容空闲的AI订阅让他感到焦虑，就像读研时昂贵的硬件未被充分利用一样，这反映出最大化Token吞吐量成为业界执念的原因。

训练

开发机器学习AI涉及一个名为“训练”的过程。简单说，就是输入数据让模型能从中学习模式并生成有用的输出。本质上是系统对数据特征做出反应，使其输出适应目标要求的过程。训练费用高昂，因为需要大量输入且需求量持续上升，这也是采用混合方法（如用目标数据微调基于规则的AI）有助于控制成本的原因——不必从头开始。

迁移学习

一种将先前训练好的AI模型作为新模型开发起点、用于不同但通常相关任务的技术，从而重用先前训练周期获得的知识。迁移学习可通过缩短模型开发流程节省成本。当目标任务数据有限时也很有用。但需注意其局限性：依赖迁移学习获得通用能力的模型，很可能需额外训练数据才能在专精领域表现良好。

验证损失

验证损失是一个告诉你AI模型训练期间学习效果的数字，越低越好。研究人员密切关注该指标，视其为实时成绩单，用于决定何时停止训练、何时调整超参数或是否调查潜在问题。它帮助发现的关键问题之一是过拟合，即模型死记硬背训练数据而非真正学习可泛化到新情况的模式。可以理解为真正理解知识的学生与死记硬背去年考题的学生的区别——验证损失有助于判断你的模型正在变成哪一种。

权重

权重是AI训练核心，决定了不同特征在训练数据中受到的重视程度，从而塑造AI模型的输出。换言之，权重是定义数据集中哪些因素对训练任务最重要的数值参数，通过将输入相乘实现功能。模型训练通常从随机赋权开始，但过程中权重会调整，因为模型试图产生更接近目标的输出。例如，一个基于历史数据预测某地房价的AI，可能包含卧室数量、卫浴数量、是否独栋或联排、是否有停车位、车库等特征的权重。最终，模型对这些输入的权重反映了基于给定数据集，它们对房产价值的影响力。

中文翻译：

人工智能正在重塑世界，与此同时，它也在发明一套全新的语言来描述自身的发展方式。如今，随便参加一场产品会议、融资路演或专家座谈，你都会听到人们谈论LLM、RAG、RLHF——以及，就在上周，还出现了像“不透明递归”这样的术语，这是OpenAI新款Astra模型中采用的推理技术，让AI安全研究人员感到不安。这套词汇更新速度之快，连科技界最聪明的人都不免感到有些底气不足。

这份术语表就是我们为解决这一问题所做的尝试：用通俗易懂的英文定义你最可能遇到的AI术语——无论你是在用这些技术开发产品、投资相关项目，还是只是通过阅读TechCrunch或收听相关播客来努力跟上节奏。我们会随着领域的发展定期更新这份术语表，因此请将它视为一份活文档，就像它所描述的AI系统一样。

AGI（通用人工智能）

通用人工智能，简称AGI，是一个含义模糊的术语。但它通常指的是在众多（如果不是大多数）任务上比普通人更有能力的AI。OpenAI首席执行官山姆·奥特曼曾将AGI描述为“相当于一个可以被你雇来当同事的普通人”。与此同时，OpenAI的公司章程将AGI定义为“在大多数具有经济价值的工作上表现优于人类的高度自主系统”。谷歌DeepMind的理解与这两种定义略有不同；该实验室认为AGI是“在大多数认知任务上至少与人类能力相当的AI”。觉得困惑？别担心——就连处于AI研究前沿的专家们也一样困惑。

AI智能体（AI agent）

AI智能体指的是一种利用AI技术代表你执行一系列任务的工具——其能力远超基本的AI聊天机器人——例如处理费用报销、预订机票或餐厅座位，甚至编写和维护代码。然而，正如我们之前解释过的，这个新兴领域有很多不断变化的组成部分，所以“AI智能体”对不同的人可能有不同的含义。支撑其预期能力的基础设施也仍在建设之中。但基本概念意味着一个自主系统，它可能调用多个AI系统来执行多步骤任务。

API端点（API endpoints）

可以把API端点想象成软件背面的“按钮”，其他程序可以按这些按钮来让该软件执行操作。开发者使用这些接口来构建集成——例如，允许一个应用程序从另一个应用程序拉取数据，或者使AI智能体能够直接控制第三方服务，而无需人工逐个操作每个接口。大多数智能家居设备和联网平台都有这些隐藏按钮，即使普通用户从未见过或与之交互过。随着AI智能体能力的增强，它们越来越能够自行找到并使用这些端点，为自动化开辟了强大——有时甚至是意想不到的——可能性。

思维链（Chain of thought）

面对一个简单的问题，人脑可以不加思索地给出答案——比如“哪种动物更高，长颈鹿还是猫？”但在很多情况下，你需要纸笔才能得出正确答案，因为其中涉及中间步骤。例如，如果一个农夫养了鸡和牛，它们总共有40个头和120条腿，你可能需要写下简单的方程式才能得出答案（20只鸡和20头牛）。

在AI语境下，大型语言模型的思维链推理意味着将问题分解为更小的中间步骤，以提高最终结果的质量。这通常需要更长时间才能得到答案，但答案更可能是正确的，尤其是在逻辑或编程场景中。推理模型是在传统大型语言模型基础上开发出来的，并通过强化学习针对思维链思考进行了优化。

（参见：大型语言模型）

编程智能体（Coding agents）

这是一个比“AI智能体”更具体的概念，“AI智能体”是指能够自主地、一步步采取行动来完成目标的程序。编程智能体是应用于软件开发的专门版本。编程智能体不仅仅是建议代码供人工审核和粘贴，它还能自主编写、测试和调试代码，处理那种通常占据开发者一整天的迭代式试错工作。这些智能体可以跨整个代码库运作，发现错误、运行测试并推送修复，只需最少的人工监督。可以把它想象成雇佣了一个非常快的实习生，他从不睡觉，也从不分心——不过，和任何实习生一样，仍然需要人工来审查其工作。

算力（Compute）

虽然这是一个多义词，但算力通常指让AI模型得以运行的至关重要的计算能力。这种处理能力驱动着整个AI行业，使其能够训练和部署强大的模型。这个词经常被用作提供计算能力的硬件类型的简称——比如GPU、CPU、TPU以及其他构成现代AI行业基石的基础设施。

深度学习（Deep learning）

机器学习中自我完善的一个子集，其中AI算法被设计为具有多层人工神经网络结构。这使得它们能够比基于简单机器学习的系统（如线性模型或决策树）建立更复杂的相关性。深度学习算法的结构灵感来自人脑中神经元相互连接的路径。

深度学习AI模型能够自行识别数据中的重要特征，而无需人类工程师定义这些特征。这种结构还支持算法从错误中学习，并通过重复和调整的过程改进自身输出。然而，深度学习系统需要大量数据点才能产生良好效果（数百万个或更多）。与更简单的机器学习算法相比，它们的训练时间通常也更长——因此开发成本往往更高。

（参见：神经网络）

扩散模型（Diffusion）

扩散模型是许多生成艺术、音乐和文本的AI模型的核心技术。受物理学启发，扩散系统通过添加噪声来缓慢“破坏”数据的结构——例如照片、歌曲等——直到什么都不剩。在物理学中，扩散是自发且不可逆的——扩散到咖啡中的糖无法恢复成方块形状。但AI中的扩散系统旨在学习一种“反向扩散”过程来恢复被破坏的数据，从而获得从噪声中恢复数据的能力。

蒸馏（Distillation）

蒸馏是一种通过“师生模型”从大型AI模型中提取知识的技术。开发者向教师模型发送请求并记录输出结果。有时会将答案与数据集进行比较以检查准确性。然后这些输出被用来训练学生模型，使其学会近似教师模型的行为。

蒸馏可以用来基于更大的模型创建一个更小、更高效的模型，且蒸馏损失最小。OpenAI很可能就是利用这种方法开发了GPT-4 Turbo，即GPT-4的更快版本。

虽然所有AI公司都在内部使用蒸馏技术，但一些AI公司也可能利用它来追赶前沿模型。从竞争对手那里进行蒸馏通常违反AI API和聊天助手的服务条款。

微调（Fine-tuning）

这是指对AI模型进行进一步训练，以优化其在比之前训练重点更具体的任务或领域上的性能——通常是通过输入新的、专门化的（即面向任务的）数据。

许多AI初创公司以大型语言模型为起点来构建商业产品，但都竞相通过基于自身领域知识和专业经验的微调来补充早期训练周期，从而为目标行业或任务增强实用性。

（参见：大型语言模型）

GAN（生成对抗网络）

GAN，即生成对抗网络，是一种机器学习框架，为生成式AI在产生逼真数据方面的一些重要发展提供了支撑——包括（但不限于）深度伪造工具。GAN涉及使用一对神经网络，其中一个网络利用其训练数据生成输出，传递给另一个网络进行评估。

这两个模型本质上被编程为试图超越对方。生成器试图让其输出通过判别器的检验，而判别器则致力于识别人工生成的数据。这种有组织的竞争可以在无需额外人工干预的情况下优化AI输出，使其更加逼真。不过，GAN最适合较窄的应用场景（如生成逼真的照片或视频），而非通用型AI。

幻觉（Hallucination）

幻觉是AI行业用来描述AI模型编造信息的首选术语——即字面意义上生成不正确的信息。显然，这对AI质量来说是一个巨大的问题。

幻觉产生的生成式AI输出可能具有误导性，甚至可能带来现实生活中的风险——后果可能很危险（想想健康查询返回有害医疗建议的情况）。

AI编造信息的问题被认为源于训练数据的空白。幻觉正在推动AI朝着日益专业化和/或垂直化的方向发展——即需要较窄专业知识的领域专用AI——以此作为减少知识空白可能性和降低虚假信息风险的一种方式。

推理（Inference）

推理是运行AI模型的过程。就是让模型根据已见数据进行预测或得出结论。需要明确的是，没有训练就无法进行推理；模型必须先学习数据集中的模式，然后才能有效地从训练数据中进行推断。

许多类型的硬件都可以执行推理，从智能手机处理器到强大的GPU，再到定制设计的AI加速器。但并非所有硬件都能同等地运行各种模型。非常大的模型在笔记本电脑上做预测可能需要很长时间，而在配备高端AI芯片的云服务器上则快得多。

（参见：训练）

大型语言模型（LLM）

大型语言模型，简称LLM，是流行AI助手所使用的AI模型，例如ChatGPT、Claude、谷歌Gemini、Meta的AI Llama、微软Copilot或Mistral的Le Chat。当你与AI助手聊天时，你就是在与一个大型语言模型交互，它直接处理你的请求，或借助不同的可用工具（如网页浏览或代码解释器）来处理。

LLM是由数十亿个数值参数（或权重，见下文）组成的深度神经网络，它们学习单词和短语之间的关系，并创建语言的表征——一种多维的单词映射。

这些模型是通过编码它们从数十亿本书籍、文章和记录中发现模式而创建的。当你向LLM发出提示时，模型会生成与提示最匹配的模式。

（参见：神经网络）

内存缓存（Memory cache）

内存缓存是指一种提升推理（即AI生成用户查询响应的过程）效率的重要技术。本质上，缓存是一种优化技术，旨在使推理更高效。AI显然由高强度的数学计算驱动，每次进行计算都会消耗更多电力。缓存通过保存特定计算结果以供未来用户查询和操作使用，来减少模型可能需要运行的计算次数。内存缓存有不同的类型，其中较为知名的一种是KV（键值）缓存。KV缓存适用于基于Transformer的模型，通过减少生成用户问题答案所需的时间（和算法工作量）来提高效率，带来更快的响应速度。

（参见：推理）

模型上下文协议（MCP）

模型上下文协议，简称MCP，是一种开放标准，允许AI模型连接到外部工具和数据——如你的文件、数据库或Slack、Google Drive等应用——而无需开发者针对每一对组合构建自定义连接器。可以把它想象成AI的USB-C接口。Anthropic于2024年推出了MCP，后来将其移交给Linux基金会管理，此后OpenAI、谷歌和微软均采用了该标准，使其成为近年以来传播最快的AI标准之一。

专家混合模型（MoE）

专家混合模型是一种将神经网络拆分为许多较小的专用子网络（即“专家”）的模型架构，并且对于任何给定任务只激活其中少数几个。与让每个请求都经过整个模型不同——就像每个问题都召集整个办公室的人一样——MoE模型内置了一个“路由器”，只为任务挑选合适的专家。这使得构建规模庞大的模型成为可能，同时运行起来相对快速且成本较低，因为任一时刻只有一小部分网络在工作。Mistral AI的Mixtral模型就是一个著名例子；OpenAI较新的GPT模型也被广泛认为使用了某种形式的这种方法，尽管该公司从未官方确认。

（参见：神经网络、深度学习）

神经网络（Neural network）

神经网络指的是支撑深度学习——以及更广泛地说，大型语言模型出现后整个生成式AI工具热潮——的多层算法结构。

虽然以人脑密集互连的路径为灵感作为数据处理算法的设计结构的想法可以追溯到20世纪40年代，但真正释放这一理论力量的，是近年来通过电子游戏行业兴起的图形处理硬件。事实证明，这些芯片非常擅长训练比早期时代具有更多层数的算法——使基于神经网络的AI系统在语音识别、自主导航和药物发现等众多领域取得了远为更好的表现。

（参见：大型语言模型）

神经语（Neuralese）

一种假设的最坏情况：模型完全以其内部数值表征而非人类可读语言进行推理，使其思维过程成为一个完全的黑匣子。目前还没有已发布的模型做到这一点——OpenAI表示其Astra模型（2026年9月发布，因早期使用“不透明递归”推理技术而备受关注）保持了思维链的可读性，并反驳了将其与神经语相提并论的比较。但安全研究人员指出，Astra使用不透明递归——其定义在下面——是朝着那个方向迈出的真实第一步，这正是该术语在Astra发布的报道之后迅速升温的原因。

不透明递归（Opaque recurrence）

不透明递归指的是AI模型反复将同一查询循环通过其内部层，而不是以通俗语言逐步推理。它更高效——较小的模型可以用更少的算力实现超水平表现——但留下的可读痕迹比正常思维链（即你向聊天机器人提问后看到的运行注释）要少得多。这让安全研究人员感到担忧，因为这些日志是捕捉不当行为的关键工具——而这项技术可能使监督变得更加困难。

（参见：思维链）

开源（Open source）

开源指的是软件——或者越来越常见的AI模型——其底层代码公之于众，任何人都可以使用、检查或修改。在AI领域，Meta的Llama系列模型就是一个突出的例子；Linux是操作系统中著名的历史类比。开源方式使世界各地的研究人员、开发者和公司能够在彼此工作的基础上继续构建，加速进展并实现封闭系统难以提供的独立安全审计。闭源意味着代码是私有的——你可以使用产品但看不到它的工作原理——OpenAI的GPT模型就是这种情况——这一区别已成为AI行业中最具标志性的争论之一。

并行化（Parallelization）

并行化意味着同时做许多事情，而不是一件接一件——就像让10名员工同时处理项目的不同部分，而不是由一名员工依次完成所有工作。在AI中，并行化对训练和推理都至关重要：现代GPU专门设计用于并行执行数千次计算，这是它们成为该行业硬件骨干的重要原因。随着AI系统日益复杂和模型规模越来越大，跨多芯片和多机器并行工作的能力已成为决定模型构建和部署速度与成本效益的最重要因素之一。对更好并行化策略的研究现在已自成一门学科。

内存危机（RAMageddon）

RAMageddon是一个有趣的新闻，用来形容席卷科技行业的一个不太好玩的趋势：随机存取存储器即RAM芯片持续不断的短缺——这些芯片为我们日常生活中几乎所有科技产品提供动力。随着AI行业蓬勃发展，最大的科技公司和AI实验室——都在竞相追求最强大、最高效的AI——大量购买RAM来支撑其数据中心，导致留给其他人的所剩无几。而供应瓶颈意味着剩余的产品越来越贵。

这波及到游戏（大公司因难以找到设备所需的内存芯片而不得不提高主机价格）、消费电子（内存短缺可能导致十多年来智能手机出货量最大跌幅）和一般企业计算（因为这些公司无法为其自家数据中心获得足够的RAM）等行业。价格飙升预计只有在令人担忧的短缺结束之后才会停止，但不幸的是，目前几乎没有迹象表明这会在短期内发生。

递归深度（Recurrent depth）

这是与不透明递归相同底层方法的更“技术性”的名称（将查询循环通过模型层多次，而非用语言顺序推理）。媒体几乎可以互换地使用这两个术语，这就是我们在这里收录它的原因，尽管“递归深度”是工程术语，而“不透明递归”是强调安全问题的表述。

（参见：不透明递归、思维链）

递归自我改进（Recursive self-improvement）

与AGI一样，递归自我改进是衡量AI能变得多聪明、对人类的依赖能降低到多低的一个门槛。在RSI的情境下，AI模型开始在无需人工干预的情况下自我改进，导致能力和自主性大幅加速。在某些说法中，这将是类似奇点的大事件——AI模型对外部干预免疫的时刻。但RSI也描述了一种基本能力——AI模型能否设计自己的继任者？——这使得工程师更容易尝试去构建它。最近有几家AI初创公司致力于构建递归自我改进的模型，但大多数都淡化了末世论的含义，将RSI仅仅呈现为研究的下一前沿。

强化学习（Reinforcement learning）

强化学习是一种训练AI的方式，系统通过尝试并因正确答案获得奖励来学习——就像用零食训练你心爱的宠物一样，只是这个场景中的“宠物”是神经网络，“零食”是表示成功的数学信号。与监督学习不同（监督学习中模型在固定标签数据集上训练），强化学习让模型探索环境、采取行动，并根据收到的反馈持续更新行为。事实证明，这种方法在训练AI玩游戏、控制机器人以及最近提升大型语言模型推理能力方面尤为强大。像基于人类反馈的强化学习（RLHF）这样的技术，如今已成为领先AI实验室微调其模型以使其更有帮助、更准确和更安全的核心手段。

词元（Token）

在人与机器的通信中，存在一些明显的挑战——人用人类语言交流，而AI程序通过基于数据的复杂算法流程执行任务。词元弥合了这一鸿沟：它们是人与AI通信的基本构建块，代表已被LLM处理或生成的离散数据片段。它们通过称为“词元化”的过程创建，该过程将原始文本分解为语言模型可以消化的微小单元，类似于编译器将人类语言翻译成计算机可以理解的二进制代码。在企业环境中，词元还决定成本——大多数AI公司按词元计费LLM使用量，企业使用越多，支付越多。

词元吞吐量（Token throughput）

再次说明，词元是AI语言模型在将语言拆解后进行处理的小块文本——通常是单词的一部分而非完整单词；为了理解AI工作负载的目的，它们大致相当于“单词”。吞吐量指的是在给定时间内可以处理多少内容，因此词元吞吐量本质上是衡量系统一次可以处理多少AI工作的指标。高词元吞吐量是AI基础设施团队的关键目标，因为它决定了模型可以同时服务多少用户，以及每个用户收到回应的速度。AI研究员安德烈·卡帕西曾描述过当他的AI订阅闲置时感到焦虑——呼应了他读研究生时昂贵计算机硬件未被充分利用的感觉——这种情绪说明了为什么最大化词元吞吐量已成为该领域的一种痴迷。

训练（Training）

开发现机器学习AI涉及一个称为“训练”的过程。简单地说，这指的是输入数据，以便模型能从模式中学习并生成有用输出。本质上，这是系统响应数据特征、使其能调整输出以趋向目标的过程——无论是识别猫的图片还是按需生成俳句。

训练可能很昂贵，因为它需要大量输入，且所需数量一直呈上升趋势——这就是为什么混合方法（例如使用定向数据微调基于规则的AI）可以帮助控制成本而不必完全从零开始。

（参见：推理）

迁移学习（Transfer learning）

一种技术，其中先前训练好的AI模型被用作开发新模型的起点，该新模型服务于不同但通常相关的任务——使得先前训练周期中获得的知识能够被重新应用。

迁移学习可以通过缩短模型开发周期来提高效率。当为模型开发的任务可用的数据有限时，它也很有用。但重要的是要注意这种方法有局限性。依赖迁移学习来获得通用能力的模型可能需要额外数据的训练，才能在其重点领域表现良好。

（参见：微调）

验证损失（Validation loss）

验证损失是一个告诉你AI模型在训练期间学习效果如何的数字——越低越好。研究人员将其作为实时成绩单密切关注，用它来决定何时停止训练、何时调整超参数，或者是否调查潜在问题。它帮助发现的关键问题之一是过拟合——模型记忆训练数据而非真正学习可以泛化到新情况的模式。可以把它想象成真正理解材料的学生与只是背下了去年试题的学生之间的区别——验证损失有助于揭示你的模型正在成为哪种。

权重（Weights）

权重是AI训练的核心，因为它们决定了在用于训练系统的数据中不同特征（或输入变量）被赋予多少重要性（或权重）——从而塑造AI模型的输出。

换句话说，权重是定义数据集中什么对给定训练任务最重要的数值参数。它们通过对输入进行乘法运算来实现其功能。模型训练通常以随机赋值的权重开始，但随着过程的推进，权重的值会随模型试图得出更接近目标的输出而调整。

例如，一个预测房价的AI模型在目标地区的历史房地产数据上训练，其权重可以包括卧室和浴室数量、房产是独栋还是半独栋、是否有停车位、车库等特征。

最终，模型赋予每个输入的权重反映了这些输入对房产价值的影响程度——基于给定的数据集。

本文会定期更新新信息。

英文来源：

AI is rewriting the world and, at the same time, inventing a whole new language to describe how it’s doing it. Sit in on any product meeting, pitch, or panel these days, and you’ll hear people toss around LLMs, RAG, RLHF — and, as of last week, terms like “opaque recurrence,” the reasoning technique in OpenAI’s new Astra model that’s got AI safety researchers rattled. The vocabulary moves fast enough to make even very smart people in the tech world feel a little insecure.
This glossary is our attempt to fix that: plain-English definitions of the AI terms you’re most likely to see, whether you’re building with this stuff, investing in it, or just trying to keep up by reading TechCrunch or listening to related podcasts. We update it regularly as the field evolves, so consider it a living document, much like the AI systems it describes.
AGI
Artificial general intelligence, or AGI, is a nebulous term. But it generally refers to AI that’s more capable than the average human at many, if not most, tasks. OpenAI CEO Sam Altman once described AGI as the “equivalent of a median human that you could hire as a co-worker.” Meanwhile, OpenAI’s charter defines AGI as “highly autonomous systems that outperform humans at most economically valuable work.” Google DeepMind’s understanding differs slightly from these two definitions; the lab views AGI as “AI that’s at least as capable as humans at most cognitive tasks.” Confused? Not to worry — so are experts at the forefront of AI research.
AI agent
An AI agent refers to a tool that uses AI technologies to perform a series of tasks on your behalf — beyond what a more basic AI chatbot could do — such as filing expenses, booking tickets or a table at a restaurant, or even writing and maintaining code. However, as we’ve explained before, there are lots of moving pieces in this emergent space, so “AI agent” might mean different things to different people. Infrastructure is also still being built out to deliver on its envisaged capabilities. But the basic concept implies an autonomous system that may draw on multiple AI systems to carry out multistep tasks.
API endpoints
Think of API endpoints as “buttons” on the back of a piece of software that other programs can press to make it do things. Developers use these interfaces to build integrations — for example, allowing one application to pull data from another, or enabling an AI agent to control third-party services directly without a human manually operating each interface. Most smart home devices and connected platforms have these hidden buttons available, even if ordinary users never see or interact with them. As AI agents grow more capable, they are increasingly able to find and use these endpoints on their own, opening up powerful — and sometimes unexpected — possibilities for automation.
Chain of thought
Given a simple question, a human brain can answer without even thinking too much about it — things like “which animal is taller, a giraffe or a cat?” But in many cases, you often need a pen and paper to come up with the right answer because there are intermediary steps. For instance, if a farmer has chickens and cows, and together they have 40 heads and 120 legs, you might need to write down a simple equation to come up with the answer (20 chickens and 20 cows).
In an AI context, chain-of-thought reasoning for large language models means breaking down a problem into smaller, intermediate steps to improve the quality of the end result. It usually takes longer to get an answer, but the answer is more likely to be correct, especially in a logic or coding context. Reasoning models are developed from traditional large language models and optimized for chain-of-thought thinking thanks to reinforcement learning.
(See: Large language model)
Coding agents
This is a more specific concept that an “AI agent,” which means a program that can take actions on its own, step by step, to complete a goal. A coding agent is a specialized version applied to software development. Rather than simply suggesting code for a human to review and paste in, a coding agent can write, test, and debug code autonomously, handling the kind of iterative, trial-and-error work that typically consumes a developer’s day. These agents can operate across entire codebases, spotting bugs, running tests, and pushing fixes with minimal human oversight. Think of it like hiring a very fast intern who never sleeps and never loses focus — though, as with any intern, a human still needs to review the work.
Compute
Although somewhat of a multivalent term, compute generally refers to the vital computational power that allows AI models to operate. This type of processing fuels the AI industry, giving it the ability to train and deploy its powerful models. The term is often a shorthand for the kinds of hardware that provides the computational power — things like GPUs, CPUs, TPUs, and other forms of infrastructure that form the bedrock of the modern AI industry.
Deep learning
A subset of self-improving machine learning in which AI algorithms are designed with a multi-layered, artificial neural network (ANN) structure. This allows them to make more complex correlations compared to simpler machine learning-based systems, such as linear models or decision trees. The structure of deep learning algorithms draws inspiration from the interconnected pathways of neurons in the human brain.
Deep learning AI models are able to identify important characteristics in data themselves, rather than requiring human engineers to define these features. The structure also supports algorithms that can learn from errors and, through a process of repetition and adjustment, improve their own outputs. However, deep learning systems require a lot of data points to yield good results (millions or more). They also typically take longer to train compared to simpler machine learning algorithms — so development costs tend to be higher.
(See: Neural network)
Diffusion
Diffusion is the tech at the heart of many art-, music-, and text-generating AI models. Inspired by physics, diffusion systems slowly “destroy” the structure of data — for example, photos, songs, and so on — by adding noise until there’s nothing left. In physics, diffusion is spontaneous and irreversible — sugar diffused in coffee can’t be restored to cube form. But diffusion systems in AI aim to learn a sort of “reverse diffusion” process to restore the destroyed data, gaining the ability to recover the data from noise.
Distillation
Distillation is a technique used to extract knowledge from a large AI model with a ‘teacher-student’ model. Developers send requests to a teacher model and record the outputs. Answers are sometimes compared with a dataset to see how accurate they are. These outputs are then used to train the student model, which is trained to approximate the teacher’s behavior.
Distillation can be used to create a smaller, more efficient model based on a larger model with a minimal distillation loss. This is likely how OpenAI developed GPT-4 Turbo, a faster version of GPT-4.
While all AI companies use distillation internally, it may have also been used by some AI companies to catch up with frontier models. Distillation from a competitor usually violates the terms of service of AI API and chat assistants.
Fine-tuning
This refers to the further training of an AI model to optimize performance for a more specific task or area than was previously a focal point of its training — typically by feeding in new, specialized (i.e., task-oriented) data.
Many AI startups are taking large language models as a starting point to build a commercial product but are vying to amp up utility for a target sector or task by supplementing earlier training cycles with fine-tuning based on their own domain-specific knowledge and expertise.
(See: Large language model [LLM])
GAN
A GAN, or Generative Adversarial Network, is a type of machine learning framework that underpins some important developments in generative AI when it comes to producing realistic data — including (but not only) deepfake tools. GANs involve the use of a pair of neural networks, one of which draws on its training data to generate an output that is passed to the other model to evaluate.
The two models are essentially programmed to try to outdo each other. The generator is trying to get its output past the discriminator, while the discriminator is working to spot artificially generated data. This structured contest can optimize AI outputs to be more realistic without the need for additional human intervention. Though GANs work best for narrower applications (such as producing realistic photos or videos), rather than general purpose AI.
Hallucination
Hallucination is the AI industry’s preferred term for AI models making stuff up — literally generating information that is incorrect. Obviously, it’s a huge problem for AI quality.
Hallucinations produce GenAI outputs that can be misleading and could even lead to real-life risks — with potentially dangerous consequences (think of a health query that returns harmful medical advice).
The problem of AIs fabricating information is thought to arise as a consequence of gaps in training data. Hallucinations are contributing to a push toward increasingly specialized and/or vertical AI models — i.e. domain-specific AIs that require narrower expertise — as a way to reduce the likelihood of knowledge gaps and shrink disinformation risks.
Inference
Inference is the process of running an AI model. It’s setting a model loose to make predictions or draw conclusions from previously seen data. To be clear, inference can’t happen without training; a model must learn patterns in a set of data before it can effectively extrapolate from this training data.
Many types of hardware can perform inference, ranging from smartphone processors to beefy GPUs to custom-designed AI accelerators. But not all of them can run models equally well. Very large models would take ages to make predictions on, say, a laptop versus a cloud server with high-end AI chips.
[See: Training]
Large language model (LLM)
Large language models, or LLMs, are the AI models used by popular AI assistants, such as ChatGPT, Claude, Google’s Gemini, Meta’s AI Llama, Microsoft Copilot, or Mistral’s Le Chat. When you chat with an AI assistant, you interact with a large language model that processes your request directly or with the help of different available tools, such as web browsing or code interpreters.
LLMs are deep neural networks made of billions of numerical parameters (or weights, see below) that learn the relationships between words and phrases and create a representation of language, a sort of multidimensional map of words.
These models are created from encoding the patterns they find in billions of books, articles, and transcripts. When you prompt an LLM, the model generates the most likely pattern that fits the prompt.
(See: Neural network)
Memory cache
Memory cache refers to an important process that boosts inference (which is the process by which AI works to generate a response to a user’s query). In essence, caching is an optimization technique, designed to make inference more efficient. AI is obviously driven by high-octane mathematical calculations and every time those calculations are made, they use up more power. Caching is designed to cut down on the number of calculations a model might have to run by saving particular calculations for future user queries and operations. There are different kinds of memory caching, although one of the more well-known is KV (or key value) caching. KV caching works in transformer-based models, and increases efficiency, driving faster results by reducing the amount of time (and algorithmic labor) it takes to generate answers to user questions.
(See: Inference)
Model Context Protocol (MCP)
Model Context Protocol, or MCP, is an open standard that lets AI models connect to outside tools and data — your files, databases, or apps like Slack and Google Drive — without a developer building a custom connector for every single pairing. Think of it as a USB-C port for AI. Anthropic introduced MCP in 2024 and later handed it over to the Linux Foundation, and it’s since been adopted by OpenAI, Google, and Microsoft, making it one of the fastest-spreading standards in recent AI history.
Mixture of Experts (MoE)
Mixture of Experts is a model architecture that splits a neural network into many smaller specialized sub-networks, or “experts,” and only activates a handful of them for any given task. Rather than routing every request through the entire model — like calling in your whole office for every question — an MoE model has a built-in “router” that picks just the right specialists for the job. This makes it possible to build enormous models that stay relatively fast and cheap to run, since only a fraction of the network is doing work at any one time. Mistral AI’s Mixtral model is a well-known example; OpenAI’s newer GPT models are also widely believed to use some version of this approach, though the company has never officially confirmed it.
(See: Neural network, deep learning)
Neural network
A neural network refers to the multi-layered algorithmic structure that underpins deep learning — and, more broadly, the whole boom in generative AI tools following the emergence of large language models.
Although the idea of taking inspiration from the densely interconnected pathways of the human brain as a design structure for data processing algorithms dates all the way back to the 1940s, it was the much more recent rise of graphical processing hardware (GPUs) — via the video game industry — that really unlocked the power of this theory. These chips proved well suited to training algorithms with many more layers than was possible in earlier epochs — enabling neural network-based AI systems to achieve far better performance across many domains, including voice recognition, autonomous navigation, and drug discovery.
(See: Large language model [LLM])
Neuralese
A hypothetical worst-case scenario where a model reasons entirely in its internal numeric representations rather than human-readable language, making its thinking a total black box. No shipped model does this today — OpenAI has said its Astra model (released in September 2026 and notable for its early use of the “opaque recurrence” reasoning technique) keeps its chain of thought legible, and has pushed back on comparisons to neuralese. But safety researchers point to Astra’s use of opaque recurrence — its definition is directly below — as a real first step in that direction, which is why the term has surged since the reporting around Astra’s launch.
Opaque recurrence
Opaque recurrence is when an AI model loops the same query through its internal layers repeatedly, instead of reasoning step-by-step in plain language. It’s more efficient — smaller models can punch above their weight while using less compute — but leaves far fewer readable traces than a normal chain of thought (that running commentary you see after asking a chatbot for help). That worries safety researchers, since those logs are a key tool for catching misbehavior — and this technique could make that oversight much harder.
(See: Chain of thought)
Open source
Open source refers to software — or, increasingly, AI models — where the underlying code is made publicly available for anyone to use, inspect, or modify. In the AI world, Meta’s Llama family of models is a prominent example; Linux is the famous historical parallel in operating systems. Open source approaches allow researchers, developers, and companies around the world to build on top of one another’s work, accelerating progress and enabling independent safety audits that closed systems cannot easily provide. Closed source means the code is private — you can use the product but not see how it works, as is the case with OpenAI’s GPT models — a distinction that has become one of the defining debates in the AI industry.
Parallelization
Parallelization means doing many things at the same time instead of one after another — like having 10 employees working on different parts of a project at the same time instead of one employee doing everything sequentially. In AI, parallelization is fundamental to both training and inference: modern GPUs are specifically designed to perform thousands of calculations in parallel, which is a big reason why they became the hardware backbone of the industry. As AI systems grow more complex and models grow larger, the ability to parallelize work across many chips and many machines has become one of the most important factors in determining how quickly and cost-effectively models can be built and deployed. Research into better parallelization strategies is now a field of study in its own right.
RAMageddon
RAMageddon is the fun new term for a not-so-fun trend that is sweeping the tech industry: an ever-increasing shortage of random access memory, or RAM chips, which power pretty much all the tech products we use in our daily lives. As the AI industry has blossomed, the biggest tech companies and AI labs — all vying to have the most powerful and efficient AI — are buying so much RAM to power their data centers that there’s not much left for the rest of us. And that supply bottleneck means that what’s left is getting more and more expensive.
That includes industries like gaming (where major companies have had to raise prices on consoles because it’s harder to find memory chips for their devices), consumer electronics (where memory shortage could cause the biggest dip in smartphone shipments in more than a decade), and general enterprise computing (because those companies can’t get enough RAM for their own data centers). The surge in prices is only expected to stop after the dreaded shortage ends but, unfortunately, there’s not really much of a sign that’s going to happen anytime soon.
Recurrent depth
This is a more “technical” name for the same underlying method as opaque recurrence (looping a query through a model’s layers numerous times rather than reasoning sequentially in language). Media outlets use the two terms almost interchangeably, which is why we’re including it here, though “recurrent depth” is the engineering term and “opaque recurrence” is the framing that emphasizes the safety concern.
(See Opaque recurrence, chain of thought.)
Recursive self-improvement
Like AGI, recursive self-improvement is a threshhold for how smart AI can get, and how little it may rely on humans. In the RSI scenario, AI models start improving themselves without human intervention, leading to a huge acceleration in capabilities and autonomy. In some tellings, this would be a cataclysmic moment akin to the singularity, a moment when AI models become immune to outside intervention. But RSI also describes a basic capability — can an AI model design its own successor? — which makes it much easier for engineers to try to build it. A number of recent AI startups have set out to build recursively self-improving models, but most of them dismiss the apocalyptic implications, presenting RSI as simply the next frontier for research.
Reinforcement learning
Reinforcement learning is a way of training AI where a system learns by trying things and receiving rewards for correct answers — like training your beloved pet with treats, except the “pet” in this scenario is a neural network and the “treat” is a mathematical signal indicating success. Unlike supervised learning, where a model is trained on a fixed dataset of labeled examples, reinforcement learning lets a model explore its environment, take actions, and continuously update its behavior based on the feedback it receives. This approach has proven especially powerful for training AI to play games, control robots, and, more recently, sharpen the reasoning ability of large language models. Techniques like reinforcement learning from human feedback, or RLHF, are now central to how leading AI labs fine-tune their models to be more helpful, accurate, and safe.
Token
When it comes to human-machine communication, there are some obvious challenges — people communicate using human language, while AI programs execute tasks through complex algorithmic processes informed by data. Tokens bridge that gap: they are the basic building blocks of human-AI communication, representing discrete segments of data that have been processed or produced by an LLM. They are created through a process called tokenization, which breaks down raw text into bite-sized units a language model can digest, similar to how a compiler translates human language into binary code a computer can understand. In enterprise settings, tokens also determine cost — most AI companies charge for LLM usage on a per-token basis, meaning the more a business uses, the more it pays.
Token throughput
So again, tokens are the small chunks of text — often parts of words rather than whole ones — that AI language models break language into before processing it; they are roughly analogous to “words” for the purposes of understanding AI workloads. Throughput refers to how much can be processed in a given period of time, so token throughput is essentially a measure of how much AI work a system can handle at once. High token throughput is a key goal for AI infrastructure teams, since it determines how many users a model can serve simultaneously and how quickly each of them receives a response. AI researcher Andrej Karpathy has described feeling anxious when his AI subscriptions sit idle — echoing the feeling he had as a grad student when expensive computer hardware wasn’t being fully utilized — a sentiment that captures why maximizing token throughput has become something of an obsession in the field.
Training
Developing machine learning AIs involves a process known as training. In simple terms, this refers to data being fed in in order that the model can learn from patterns and generate useful outputs. Essentially, it’s the process of the system responding to characteristics in the data that enables it to adapt outputs toward a sought-for goal — whether that’s identifying images of cats or producing a haiku on demand.
Training can be expensive because it requires lots of inputs, and the volumes required have been trending upwards — which is why hybrid approaches, such as fine-tuning a rules-based AI with targeted data, can help manage costs without starting entirely from scratch.
[See: Inference]
Transfer learning
A technique where a previously trained AI model is used as the starting point for developing a new model for a different but typically related task — allowing knowledge gained in previous training cycles to be reapplied.
Transfer learning can drive efficiency savings by shortcutting model development. It can also be useful when data for the task that the model is being developed for is somewhat limited. But it’s important to note that the approach has limitations. Models that rely on transfer learning to gain generalized capabilities will likely require training on additional data in order to perform well in their domain of focus
(See: Fine tuning)
Validation loss
Validation loss is a number that tells you how well an AI model is learning during training — and lower is better. Researchers track it closely as a kind of real-time report card, using it to decide when to stop training, when to adjust hyperparameters, or whether to investigate a potential problem. One of the key concerns it helps flag is overfitting, a condition in which a model memorizes its training data rather than truly learning patterns it can generalize to new situations. Think of it as the difference between a student who genuinely understands the material and one who simply memorized last year’s exam — validation loss helps reveal which one your model is becoming.
Weights
Weights are core to AI training, as they determine how much importance (or weight) is given to different features (or input variables) in the data used for training the system — thereby shaping the AI model’s output.
Put another way, weights are numerical parameters that define what’s most salient in a dataset for the given training task. They achieve their function by applying multiplication to inputs. Model training typically begins with weights that are randomly assigned, but as the process unfolds, the weights adjust as the model seeks to arrive at an output that more closely matches the target.
For example, an AI model for predicting housing prices that’s trained on historical real estate data for a target location could include weights for features such as the number of bedrooms and bathrooms, whether a property is detached or semi-detached, whether it has parking, a garage, and so on.
Ultimately, the weights the model attaches to each of these inputs reflect how much they influence the value of a property, based on the given dataset.
This article is updated regularly with new information.