内容来源：https://research.google/blog/designing-synthetic-datasets-for-the-real-world-mechanism-design-and-reasoning-from-first-principles/

内容总结：

谷歌发布Simula框架：以“机制设计”革新合成数据生成，赋能专业AI发展

2026年4月16日，谷歌学生研究员Tim R. Davidson与高级研究员Hamza Harkous共同发布了一项名为Simula的创新框架。该研究旨在解决专业人工智能领域面临的数据稀缺难题，通过将合成数据生成重新定义为“数据集层面的机制设计”，为隐私敏感或数据匮乏的领域提供了一种可扩展的高质量数据解决方案。

背景：专业AI面临数据瓶颈
当前通用人工智能的快速发展得益于互联网海量数据的滋养。然而，当AI需要深入医疗、法律、网络安全等专业或隐私敏感领域时，真实数据往往难以获取、成本高昂且开发周期漫长。传统依赖真实数据的方式存在明显局限：采集标注耗时费力、静态数据拖慢迭代速度、难以主动针对罕见或危险场景进行模型加固。

突破：从“样本生成”到“机制设计”
现有的合成数据生成方法多依赖于手动提示、进化算法或大量种子数据，存在可扩展性差、过程不透明、控制粒度粗等问题。Simula框架的核心突破在于，将数据生成视为一个系统性的“机制设计”问题，而非简单的样本堆砌。它采用“推理优先”的方法，从第一性原理出发，像架构软件一样架构整个数据集，实现了对数据覆盖度、复杂度和质量的精细化、独立控制。

核心：四步构建可控数据生成
Simula将生成过程分解为四个清晰可控的步骤：

1. 全局多样化：利用推理模型为目标领域构建深层次的概念分类体系，作为数据采样的“脚手架”，确保数据能覆盖领域的长尾分布，而非仅集中于常见模式。
2. 局部多样化：在特定概念下，生成多样化的具体场景实例，避免模式坍塌，确保同一概念（如“SQL注入攻击”）能以不同形式呈现。
3. 复杂度调控：将复杂度作为一个独立维度进行调节，可对部分数据场景进行精细化或复杂化处理，从而在不改变语义覆盖范围的前提下，灵活调整数据集的难度分布。
4. 质量校验：采用“双重评判”循环，自动、独立地验证生成答案的正确性，有效减少模型附和倾向，确保数据标签的高质量。

评估与洞察：没有“放之四海而皆准”的方案
研究团队在网络安全、法律推理、数学、多语言知识等五个不同领域进行了大规模评估（每个领域生成多达51.2万个数据点）。结果揭示了一个关键现实：不存在单一的“最优”数据生成方法，数据与下游模型性能的关系高度依赖于具体情境。

• 机制设计至关重要：整合了全局覆盖、局部多样化和质量评判的完整Simula系统，在所有领域均稳定优于简单基线方法。
• 上下文为王：数据必须与使用模型的特性相匹配。例如，提高复杂度在数学推理（GSM8k）上带来10%的准确率提升，但在教师模型较弱的法律推理（LEXam）中却损害了性能。
• 质量优于数量：Simula生成的高质量数据能以更少的样本量实现更优的下游性能，证明驱动模型性能提升的是数据属性，而不仅仅是数据规模。

从研究到现实应用
Simula不仅是学术研究，更已成为谷歌内部关键业务应用的基石数据引擎。它支撑了Gemma开源模型生态中ShieldGemma、FunctionGemma、MedGemma等专业模型的开发，并为端侧及服务器端的Gemini安全分类器提供了核心合成数据支持。此外，该框架已应用于安卓AI诈骗电话检测、谷歌信息垃圾过滤等用户保护功能，并推动着企业安全攻防模拟、教AI阅读地图等前沿应用研究。

展望：合成数据将成专业AI突破关键
人工智能的发展正处在十字路口。科学、安全、法律等下一波突破所必需的专业化数据，难以依靠人工达到所需规模。合成数据注定将在其中扮演核心角色，但前提是必须采用严谨、可控的科学方法。Simula框架的价值正在于，它通过机制设计为生成下一代AI所需的高保真数据集提供了一条清晰、可控的技术路径。

中文翻译：

为现实世界设计合成数据集：基于第一性原理的机制设计与推理
2026年4月16日
Tim R. Davidson（学生研究员）与 Hamza Harkous（谷歌高级研究科学家）

为解决专业人工智能所需数据稀缺的问题，我们推出Simula框架，将合成数据生成重新定义为数据集层面的机制设计。该框架通过第一性原理的推理构建数据集，实现对覆盖范围、复杂性与质量的精细化控制，为隐私敏感或数据稀缺领域提供可扩展的生成方案。

快速链接
通用人工智能模型的快速发展得益于互联网数据的丰富性。然而，人工智能的广泛应用要求模型能够专注于新颖、小众及隐私敏感的场景，而这些场景的数据天然稀缺或难以获取。

为弥补这一缺口，依赖现实世界数据存在显著局限：

• 成本与可获取性：人工创建专业数据集成本极高、耗时漫长且易出错。
• 开发效率制约：现实数据的静态特性拖慢开发周期。相比之下，“合成优先”方法支持“可编程工作流”——数据可像代码一样进行版本管理、复现与审查。
• 前瞻性能力：在安全性等关键领域，被动应对已不足够。合成数据使我们能主动生成边缘案例，对尚未发生的场景进行系统压力测试。

尽管合成数据是前景广阔的替代方案，但当前生成方法往往缺乏生产级部署所需的严谨性。许多现有方法依赖人工提示、进化算法或大量来自目标分布的种子数据。这些方法受限于可扩展性（依赖种子或人力）、可解释性（黑箱进化步骤）和可控性（参数相互耦合）。最关键的是，它们通常在样本层面运作——逐点优化数据——而非将数据集作为整体进行设计。

为此，我们需要将合成数据生成重构为机制设计问题。生产用例不仅需要“更多数据”，更需精细化的资源分配，使覆盖范围、复杂性与质量成为独立可控的变量。

Simula：推理优先的框架
在发表于《机器学习研究汇刊》的论文《推理驱动的合成数据生成与评估》中，我们提出Simula框架。与依赖不透明过程的方法不同，Simula采用“推理优先”方法论，从第一性原理构建完整数据集。该方法无需种子数据且具备自主性，其生成能力可随底层模型推理能力的提升自然进化。

控制数据生成的维度
Simula将生成过程分解为四个独立可控的步骤：

1. 全局多样化：Simula使用推理模型将目标领域的概念空间映射为深层层次化分类体系，形成“采样脚手架”。通过定义分类体系的采样策略，可控制全局多样性，确保数据集覆盖领域的长尾分布而非仅聚集于常见模式。

在建立深层分类体系后，我们可进一步优化：

2. 局部多样化：为保障特定概念内部的差异性，系统生成源自分类节点的“元提示”场景，并产出该场景的多个不同实例，避免模式坍缩。例如“SQL注入”概念可通过多样化表述呈现，而非简单重复。
3. 复杂化调控：将复杂性作为独立维度，通过可配置比例对元提示进行精细化处理，提升其精细度或难度。这使得实践者能在不改变语义覆盖的前提下调整数据集的难度分布。
4. 质量校验：采用“双重校验”循环，独立评估答案的正确性。这种双重验证机制缓解了模型附和性倾向，确保生成高质量标签。

应对评估挑战
合成数据评估的核心挑战在于目标模糊性以及标准指标与实际效用的脱节。基于嵌入向量的余弦距离等标准指标虽能提供宏观信号，却难以转化为具体洞见。

为提升评估稳健性，我们同样采用推理优先方法，引入基于推理的指标：

• 分类体系覆盖度
• 校准复杂度评分（使用LLM驱动的批量比较为数据点分配国际象棋式的“Elo评级”）
  这些指标能更精准捕捉多样性与难度的细微差异。

不存在通用解决方案
我们使用Gemini 2.5 Flash作为教师模型、Gemma-3 4B作为学生模型，在五个领域评估Simula：网络安全（CTIBench的CTI-MCQ、CTI-RCM）、法律推理（LEXam）、小学数学（GSM8k）及多语言学术知识（Global MMLU）。为每个领域生成高达51.2万个数据点的结果显示：

• 不存在单一的“最优”数据生成方式，“优质数据”与下游性能的关系具有高度特异性。
• 机制设计不可或缺：完整Simula系统（整合全局覆盖、局部多样性与校验机制）在所有领域均稳定优于简易基线。
• 场景决定策略：在数学推理（GSM8k）中提升复杂度带来10%准确率增益，但在教师模型较弱的法律推理（LEXam）中反而损害性能。数据必须适配使用模型的能力特点。
• 质量优于数量：Simula以更少样本实现更高下游性能，证实扩展规律由数据属性驱动而非单纯数量。

虽然本次采用知识蒸馏架构以便系统化评估，但核心结论适用于更广泛配置。

从研究到现实影响
Simula不仅为优化基准测试而构建，更作为谷歌关键业务应用的基础数据引擎。在前沿AI领域，它已成为Gemma生态（包括ShieldGemma、FunctionGemma、MedGemma等专业模型）的核心推动力，同时为设备端与服务器端的Gemini安全分类器提供主要合成数据支持。在基础模型之外，Simula助力推出用户保护功能，包括Android通话的AI诈骗检测与Google Messages的垃圾信息过滤。该框架还驱动着新兴应用研究，通过合成真实攻击场景为企业安全领域降低机器学习门槛，并通过结构化、推理驱动的数据集生成实现突破（如教授AI模型解读地图）。

合成数据在专业AI中的核心作用
人工智能发展正处在十字路口。科学、安全、法律等下一波突破所需的专业数据，已难以通过人工方式大规模产生。合成数据注定将在这些飞跃中扮演核心角色，但前提是采用严谨的方法。Simula的价值在于证明机制设计如何使数据生成成为可控的科学。这份蓝图为构建下一代AI所需的高保真数据集指明了清晰路径——无论是向边缘设备蒸馏知识、通过强化学习训练智能体，还是系统化探索复杂边缘案例。

致谢
本研究由Tim R. Davidson、Benoit Seguin、Enrico Bacis、Cesar Ilharco与Hamza Harkous共同完成。Simula框架由Hamza与Benoit创立并领导。特别感谢Tim在学生研究员期间的卓越贡献。同时感谢Jan Keller的项目管理支持，以及Coran Corbett和Ninny Wan的关键技术与产品协作。最后感谢Nina Taft、Amanda Walker与Pankaj Rohatgi的赞助与支持。

英文来源：

Designing synthetic datasets for the real world: Mechanism design and reasoning from first principles
April 16, 2026
Tim R. Davidson, Student Researcher, and Hamza Harkous, Senior Staff Research Scientist, Google
To address the scarcity of data required for specialized AI, we introduce Simula, a framework that reframes synthetic data generation as dataset-level mechanism design. By using reasoning to architect datasets from first principles, Simula enables fine-grained control over coverage, complexity, and quality, providing scalable generation for privacy-sensitive or data-scarce domains.
Quick links
The rapid advance of generalist AI models has been fueled by the abundance of internet data. However, widespread integration of AI will require models to specialize in novel, uncommon, and privacy-sensitive applications where data is inherently scarce or inaccessible.
To bridge this gap, reliance on real-world data imposes significant limitations:

• Cost and accessibility: Creating specialized datasets manually is prohibitively expensive, time-consuming, and error-prone.
• Operational drag: The static nature of real-world data slows development cycles. In contrast, a synthetic-first approach enables "programmable workflows" where data is treated like code — versioned, reproducible, and inspectable.
• Preparedness: We cannot afford a reactive approach to topics like safety, where models can be hardened only after failures occur. Synthetic data allows us to proactively generate edge cases and stress-test systems against scenarios that have not yet happened in the wild.
  While synthetic data is a promising alternative, current generation methods often lack the rigor required for production-scale deployment. Many existing approaches rely on manual prompts, evolutionary algorithms, or extensive seed data from the target distribution.
  These methods limit scalability (due to reliance on seeds or human effort), explainability (due to black-box evolutionary steps), and control (due to entangled generation parameters). Most critically, they typically operate at the sample level — optimizing one data point at a time — rather than designing the dataset as a whole.
  To solve this, we need to reframe synthetic data generation as a problem of mechanism design. Production use cases require a focus beyond just "more data"; they require fine-grained resource allocation where coverage, complexity, and quality are independently controllable variables.
  Simula: A reasoning-first framework
  In our paper, “Reasoning-Driven Synthetic Data Generation and Evaluation”, published in Transactions on Machine Learning Research, we introduce Simula. Unlike methods that rely on opaque processes, Simula employs a "reasoning-first" methodology, constructing entire datasets from first principles. This approach is seedless and agentic, allowing the generation capabilities to improve naturally as the reasoning capabilities of the underlying models advance.
  Controlling the axes of data generation
  Simula decomposes the generation process into distinct, controllable axes, using four steps:
• Global Diversification: Instead of random sampling, Simula uses reasoning models to map the conceptual space of a target domain into deep, hierarchical taxonomies. This acts as a "sampling scaffold". By defining sampling strategies over these taxonomies, we can control global diversity — ensuring the dataset covers the long tail of a domain rather than clustering around common modes.
  Equipped with a set of deep taxonomies, we can now start mapping out our coverage space of interest and optimize (2) local diversity, (3) complexity, and (4) quality:
  2. Local Diversification: To ensure variation within specific concepts, we employ local diversity mechanisms. The system generates "meta-prompts" — scenarios derived from taxonomy nodes — and then produces multiple distinct instantiations of that scenario. This prevents mode collapse, ensuring that a concept like "SQL injection" is represented through diverse framings rather than identical repetitions.
  3. Complexification: Complexity is treated as an orthogonal axis. We use a "complexification" step where a configurable fraction of meta-prompts is refined to be more elaborate or difficult. This allows practitioners to shift the difficulty distribution of a dataset without changing its semantic coverage.
  4. Quality Checks: To ensure correctness without human intervention, we employ a "dual-critic" loop that independently assesses if an answer is correct or incorrect. This dual-verification helps mitigate sycophancy (where models tend to agree with plausible-sounding outputs) and ensures high-quality labels.
     Addressing challenges in evaluation
     The evaluation of synthetic data is fundamentally challenging due to the ambiguity of its core objectives and the disconnect between standard metrics and practical utility. Standard metrics like embedding-based cosine distance provide a high-level signal but offer limited actionable insights.
     To make evaluations more robust, we apply our reasoning-first approach here as well. Specifically, we introduce reasoning-based metrics — Taxonomic Coverage and Calibrated Complexity Scoring (which uses LLM-driven batch comparisons to assign chess-style "Elo ratings" to individual data points) — to better capture the nuances of diversity and difficulty.
     No universal solution
     We used Gemini 2.5 Flash as a teacher model and Gemma-3 4B as a student to evaluate Simula across five diverse domains — from cybersecurity (CTI-MCQ, CTI-RCM from CTIBench) and legal reasoning (LEXam), to standard AI model evaluations such as grade-school math (GSM8k) and multilingual academic knowledge (Global MMLU). Generating datasets of up to 512K data points for each domain, our results highlight a critical reality: there is no single "optimal" way to generate data, and the relationship between "good" data and downstream performance is deeply idiosyncratic.
• Mechanism design is non-negotiable: Across all domains, the full Simula system — which combines global coverage, local diversity, and critiquing — consistently outperformed simpler baselines.
• Context is king: There are no fixed recipes. While high complexity yielded a 10% accuracy gain in math reasoning (GSM8k), it actually hurt performance in legal reasoning (LEXam) where the teacher model was weaker. Data must be tailored to the capabilities of the model consuming it.
• Quality is the new quantity: Better data scales better. Simula achieved higher downstream performance with fewer samples compared to baseline approaches, confirming that scaling laws are driven by data properties, not just volume.
  While this was a distillation setup, chosen for replicable, systemic evaluation, the core lessons learned extend beyond this specific configuration.
  From research to real-world impact
  Simula was not just built to optimize benchmarks, it serves as a foundational data engine for real-world, business-critical applications across Google. Within the frontier AI space, it has been a key enabler for the Gemma ecosystem — including specialized models like ShieldGemma, FunctionGemma, and MedGemma — while providing the primary synthetic data backbone for both on-device and server-side Gemini safety classifiers. Beyond foundation models, Simula has been instrumental in shipping user protection features, including AI-powered scam detection for Android calls and spam filtering in Google Messages. Furthermore, Simula is actively driving new applied research, facilitating frameworks that democratize ML for enterprise security by synthesizing realistic attack scenarios, and enabling breakthroughs like teaching AI models to read maps through structured, reasoning-driven dataset generation.
  Synthetic data's central role in specialized AI
  AI progress is at a junction. The specialized data required for the next wave of breakthroughs — in science, security, and law — is unlikely to be generated by humans at the necessary scale. Synthetic data is primed to play a central role in these leaps, but only if approached with rigor. Ultimately, Simula's value lies in demonstrating how mechanism design can make data generation a controllable science. This blueprint provides a clear path to building the high-fidelity datasets the next era of AI demands — whether we are distilling knowledge into edge devices, training agents via reinforcement learning, or systematically exploring complex edge-cases.
  Acknowledgements
  This research was authored by Tim R. Davidson, Benoit Seguin, Enrico Bacis, Cesar Ilharco, and Hamza Harkous. The Simula framework was founded and led by Hamza and Benoit. Special thanks go to Tim for his significant contributions during his student researcher tenure. We also thank Jan Keller for his TPM support and Coran Corbett and Ninny Wan for their vital technical and product partnerships. Finally, we thank Nina Taft, Amanda Walker, and Pankaj Rohatgi for their sponsorship and support.