内容来源：https://www.theverge.com/ai-artificial-intelligence/943187/ai-content-creators

内容总结：

深度伪造入侵社交媒体：AI“内容创作者”正变得难以分辨

《The Stepback》科技周报近期刊发深度调查指出，人工智能生成的虚拟网红正以惊人速度渗透主流社交平台，其逼真程度已让普通用户和平台自身都难以辨别真伪。

从“一眼假”到“难分辨”

早期AI虚拟偶像如Lil Miquela、Imma等，因制作粗糙、造型夸张，很容易被识别为数字产物。然而，如今以Emily Pellegrini、Aitana Lopez为代表的新一代AI形象已更贴近真人——她们发布精致旅行照、打卡热门活动，与真实网红的日常内容别无二致。更关键的是，制作门槛大幅降低：普通用户无需专业设备或团队，借助Google、OpenAI等公司的主流AI工具，即可低成本批量创建“虚拟人”。

规模化泛滥：从诈骗到政治宣传

调查发现，这些AI假人已形成庞大生态。它们被用于推广廉价商品、实施照片诈骗、传播虚假信息甚至种族主义言论，部分账号还聚焦色情擦边内容。由于平台不公开AI用户数量统计，且绝大多数虚拟账号未达到受关注的规模，其真实影响力难以评估。据市场研究机构预测，虚拟网红市场规模将从今年的约120亿美元飙升至2030年的600亿美元以上。

平台态度暧昧：既当裁判又当运动员

面对这一乱象，Meta旗下Instagram、TikTok、YouTube等主流平台态度矛盾。表面上，各平台已出台合成媒体标注政策，要求AI生成内容需明确标识；但私下里，它们同时大力推广自家AI创作工具，包括能克隆或模拟用户的功能。这种“既希望利用AI提升内容量，又担心失控”的立场，导致AI虚拟人长期处于监管灰色地带——它们不直接违反现有诈骗、冒充或色情规定，只要主动标注为AI生成，平台便难以处罚。

未来隐患：社交网络或面临“虚假危机”

分析指出，若放任AI虚拟人泛滥，社交网络将面临根基动摇。一方面，持续涌入的劣质AI内容正在加速用户流失；另一方面，当这些虚拟人主要服务于“从真人身上变现”这一目的时，一旦真人用户被耗尽，整个生态将难以为继。目前，已有用户自发寻求“无AI空间”，并呼吁平台开放过滤功能。正如媒体评论所言：“让我们过滤AI垃圾内容吧，你们这些懦夫。”

值得警惕的是，部分高调AI网红已呈现明显政治化倾向。例如英国极右翼政党资助的白人民族主义说唱歌手Danny Bones，以及融合军旅美学与特朗普主义的“MAGA幻想女孩”Jessica Foster。这些案例可能加速立法机构介入。

目前，欧洲《人工智能法案》已要求对AI生成内容进行强制透明披露，违规者将面临巨额罚款。但在更明确的法规出台前，识别AI假人的责任仍主要落在用户自己肩上。

中文翻译：

这是《The Stepback》周刊，每周为您解析科技界的一则核心要闻。想了解更多关于AI带来的困惑，请关注罗伯特·哈特。《The Stepback》于美国东部时间早上8点送达订阅用户邮箱。点击此处订阅《The Stepback》。

AI“内容创作者”越来越难以分辨
社交媒体平台正感到困惑。

起初
最初，AI网红相对容易识别——也容易被忽略。除了偶尔的炒作热潮，它们似乎并未改变社交媒体的运作方式。最早的虚拟网红——留着齐刘海、脸上有雀斑的Lil Miquela，顶着一头泡泡糖粉色波波头的Imma，以及拥有无瑕肌肤的Shudu Gram——显然是数字制作的产物。合作消息会大张旗鼓地宣布。每条动态都需要工作室、资金、协调和大量打磨。

随着时间的推移，我注意到，我信息流中的“假人”开始看起来越来越像其他真实用户。像Emily Pellegrini和Aitana Lopez这样的角色更接近现实——至少更接近那位你早已断了联系、却总在发高档餐厅、美丽景点或科切拉音乐节、温网照片的、见多识广且富有的大学同学的生活状态。不见得能引起共鸣，话说回来，大多数职业网红也做不到。

即便如此，这些账号也绝非普通账号。Lopez是西班牙创意机构“The Clueless”的产品，该机构旗下管理着一批AI网红。Pellegrini的创作者化名“EP教授”，他告诉我他曾管理过OnlyFans的创作者。现在他出售课程，教人们如何打造自己的AI网红。

而这正是人们开始在做的事情。而且是很多人。

现状如何
新奇感已经消失。早期的AI网红之所以突出，是因为数量极少。现在，它们已成为淹没社交媒体的海量AI生成内容中更混乱的一部分：从聊天机器人懒散复制而来的低质废话、劣质图片和视频，以及在TikTok上霸屏一个月的、那首朗朗上口的《指环王》迪斯科歌曲。

假人如今无处不在。它们推销低劣的直销垃圾，用假照片诈骗男性钱财，散布虚假信息和种族主义言论，并迎合日益怪异、往往带有色情意味的小众需求。当然，其中有很多诱导性内容。也有很多平淡无奇的内容，虚拟形象只是简单复制人类创作者当下流行的一切，通常只是把自己的假脸贴上去。

这使得AI内容创作者的影响力规模难以估量。平台不公布有多少用户是假人，而且大多数AI虚拟形象并未变得足够流行或具有影响力，以配得上早期那波网红所获得的媒体关注。像“虚拟人类”（Virtual Humans）这样的数据库追踪着数百个受欢迎的虚拟形象，但这仅限于那些足够奇怪、怪异或庞大到能被注意到的账号。在它们之下，是大量完全不为人知的账号。

这些账号能够不被发现的部分原因，在于制作它们的技术已大幅改进。现在，一张假人的静态图片足以在扫一眼时被误认为真，尤其是在一个充满真人网红、且他们大量使用布景、滤镜和编辑效果的动态信息流中。视频和音频也在迅速跟上，赋予虚拟人物足以欺骗粗心滚动者的声音和动作。这些工具也不再是小众或贵得离谱。来自谷歌、OpenAI等公司的主流产品，与来自Higgsfield、HeyGen和ElevenLabs等公司的专业服务并存。只需稍加努力，几乎任何人都可以制作一个AI网红——或者一批——无需工作室、专业设备或（大量）资金。

这一切都给社交媒体平台带来了一个它们似乎并不特别想正面解决的问题。在应对AI生成的图像、视频和音频数年后，大多数主流平台现在都有了某种涵盖合成媒体的政策。但除了要求给AI生成内容打上标签外，这些规则往往不过是把这类内容硬塞进现有的类别中，比如诈骗、垃圾信息、冒充他人和色情内容。AI人，尤其是那些刻意模仿真人行为的AI人，并不完全符合这些类别中的任何一个。它们未必在实施诈骗、发布色情内容或冒充他人——它们甚至要去冒充谁？而且，如果它们披露其内容是AI生成的，它们显然没有违反任何规则。

目前，平台似乎满足于这种模棱两可的状态，既不完全欢迎也不排斥AI创作者。它们培养了一种自相矛盾的立场：一方面将AI作为创意工具进行推广，另一方面又试图阻止汹涌的劣质内容淹没其服务。YouTube、TikTok、Instagram和其他平台已制定了关于标注合成媒体的规则，尤其是逼真的那类，同时也在推广自己的一套AI工具，其中一些可以克隆或模拟用户。但这些规则往往侧重于单条内容，而非背后的账号和角色，这使得AI网红处于灰色地带。

在这种不确定性中，AI网红生态系统正在蓬勃发展。一些市场研究公司估计，虚拟网红市场到2030年可能价值超过600亿美元，高于今年的约120亿美元。文化影响力也在增长。有AI网红奖项、选美比赛，有专门代表合成创作者的经纪公司，还有一个蓬勃发展的市场，出售课程和工具，承诺帮助人们制作和运营自己的假创作者，通常承诺“无面孔的被动收入”。其中一些带有网上淘金热那种微妙的金字塔式气息：少数可见的成功故事，以及大量出售铲子的人。

接下来会发生什么
我的猜测是，清算即将到来。AI劣质内容已经令人恼火，一个平台能承载的这类内容也是有限的，直至其几乎无法使用，尤其是在平台一贯拒绝让用户过滤AI劣质内容的情况下。假人冒充真人则是同一问题更私密化的版本。但除了标签和执行现有规则，平台似乎大多满足于静观其变。对于平台而言，参与度仍然是参与度，无论它来自假创作者还是真人创作者。只要合成创作者继续发布内容且不越界，似乎就没什么动力去打击。

还有一个问题是，让AI虚拟形象在网上到处游荡的想法本身能持续多久。如果这么多AI形象只是为了从人类用户身上赚钱，那么当人类用户池干涸时会发生什么？例如，愿意购买课程和工具来打造自己网红的数量是有限的。这还假设了社交媒体能够经受住AI网红的涌入。从定义上讲，它需要一定数量的人类来维持“社交性”。如果不加控制，网络将在这些假人的重压下崩溃，因为人类用户必然会流失。

如果公众愤怒持续累积，这种情况可能会改变。对深度伪造、冒充他人和合成垃圾信息的反弹，已经迫使立法者和监管机构更加关注，尤其是在使用Grok等工具生成的非自愿性深度伪造事件之后。欧洲的《人工智能法案》可能是一个推动力，至少在其对AI生成内容的透明度义务生效后是这样。该法规将要求生成式AI系统的部署者明确披露AI生成或篡改的内容，这可能会迫使公司加强标记AI内容，否则将面临可能高达的巨额罚款。但即便如此，焦点仍然主要放在内容上，而非发布该内容的账号是否代表真人。

与社交媒体上许多事情一样，负担又落回到了用户身上。许多平台已有效地将审核AI内容的任务下放给用户，依赖他们来发现和举报可疑账户。但针对旨在规避注意的事物，自我审核是一种糟糕且不可持续的答案。人们对无AI空间的渴望已经日益增长。如果平台拒绝自己划清真实与虚假的界限，我预计用户会自己划清界线。

顺便提一下

• 我最近遇到的许多更知名的AI网红都带有明显的政治倾向，我觉得这可能会加速监管的清算。Danny Bones，一个由英国极右翼政党资助的假白人至上主义说唱歌手，大概是我迄今见过的最好例子。
• 像人类网红一样，许多AI虚拟形象是围绕特定身份和社区构建的，例如种族、残疾、政治立场和国籍，比如MAGA幻想女孩Jessica Foster，她深度融合了性感的军队美学和特朗普主义。并非所有虚拟形象都与创作者一致：例如，黑人AI模特Shudu Gram是由一名白人男性创造的。Emily Pellegrini也是一个男性——“EP教授”——的产物，他告诉我这个角色是利用他从一位匿名OnlyFans创作者那里获得授权的内容构建的。

推荐阅读

• 杰斯·韦瑟比德近期为《The Verge》撰写的报道标题说明了一切：“让我们过滤AI劣质内容吧，你们这群懦夫。”
• 《The Verge》近期报道称，骗子们正在使用假黑人的AI虚拟形象，通过社交媒体直销高价兜售批量生产的产品。
• 《连线》杂志报道了蓬勃发展的“AI拉皮条”行业，在该行业，人类创作者的内容被AI虚拟形象未经许可地窃取并货币化。
• 查理·沃泽尔的播客审视了AI网红泛滥背后的动机，以及许多人在已不再关心我们所消费的内容是真是假时所感到的疲惫。

英文来源：

This is The Stepback, a weekly newsletter breaking down one essential story from the tech world. For more on AI confusion, follow Robert Hart. The Stepback arrives in our subscribers’ inboxes at 8AM ET. Opt in for The Stepback here.
AI ‘content creators’ are getting harder to spot
Social media platforms are baffled.
AI ‘content creators’ are getting harder to spot
Social media platforms are baffled.
How it started
At first, AI influencers were relatively easy to identify — and to ignore. Aside from the occasional bursts of hype, they didn’t seem to change much about the way social media worked. The earliest virtual influencers — Lil Miquela with her blunt fringe and freckles, Imma with her bubblegum pink bob, and Shudu Gram with her flawless complexion — were obviously digital productions. Collaborations were announced with fanfare. Posts required studios, money, coordination, and a lot of polish.
Over time, I’ve noticed that the fake people on my timeline have started looking more and more like everyone else on it. Characters like Emily Pellegrini and Aitana Lopez moved a bit closer to reality — or at least to the reality of that well-traveled, well-off friend from college you didn’t keep in touch with, forever posting from nice restaurants and beautiful places, or from Coachella and Wimbledon. Not exactly relatable, but, then again, most professional influencers aren’t either.
Even then, many of these accounts aren’t standard ones by any means. Lopez is the product of a Spanish creative agency called The Clueless, which manages a stable of AI influencers. Pellegrini’s creator, who goes by the pseudonym Professor EP, told me he used to manage OnlyFans creators. Now he sells courses teaching people how to make AI influencers of their own.
Which is exactly what people are starting to do. A lot of people.
How it’s going
The novelty has worn off. Early AI influencers stood out because there were so few of them. Now they are part of a much larger mess of AI-generated content inundating social media: low-quality drivel lazily copied from chatbots, slop images and videos, and that catchy Lord of the Rings disco song that took over my TikTok for a month.
The fake people are now everywhere. They’re upselling drop-ship junk, scamming men out of money with fake photos, pushing disinformation and racist talking points, and catering to an increasingly weird, often sexual niche. Of course, there are a lot of thirst traps. There’s also a lot of mundane content, with avatars simply copying whatever’s popular among human creators, often just putting their fake faces on it.
That makes the scale of AI content creator influence hard to gauge. Platforms do not publish figures on how many of their users are fake people, and most AI avatars don’t become popular or influential enough to justify the kind of media attention the earlier wave received. Databases like Virtual Humans track hundreds of popular avatars, but those are only the accounts strange, weird, or big enough to get noticed. Below them is an ocean of accounts flying totally under the radar.
Part of the reason these accounts are able to avoid detection is that the technology used to make them has improved massively. A still image of a fake person can now be good enough to pass as genuine at a glance, especially in a feed filled with real influencers making generous use of staging, filters, and editing effects. Video and audio are quickly catching up, giving virtual people voices and movements that could fool undiscerning scrollers. The tools are no longer niche or prohibitively expensive, either. Mainstream products from companies like Google and OpenAI sit alongside specialized services from firms like Higgsfield, HeyGen, and ElevenLabs. With a little effort, almost anyone can make an AI influencer — or stable of them — without needing a studio, specialized equipment, or (much) money.
All this leaves social media platforms with a problem they do not seem especially interested in solving head-on. After several years of grappling with AI-generated images, videos, and audio, most major platforms now have some kind of policy covering synthetic media. But beyond requiring labels for AI-generated content, such rules often amount to little more than shoehorning the material into existing categories covering things like scams, spam, impersonation, and graphic material. AI people, especially those designed to behave like real people, don’t fit neatly into any of these buckets. They are not necessarily running a scam, posting graphic content, or impersonating someone — who would they even impersonate? And if they disclose that their posts are AI-generated, it’s not obvious what rules they’d be breaking.
For now, platforms seem content to live in ambiguity, neither fully welcoming nor shunning AI creators. They have cultivated a contradictory position, promoting AI as a creative tool while also trying to stop a tidal wave of slop from overwhelming their services. YouTube, TikTok, Instagram, and other platforms have developed rules for labeling synthetic media, particularly the realistic kind, while also promoting their own suites of AI tools, including some that can clone or simulate users. But those rules tend to focus on individual posts rather than the accounts and personas behind them, leaving AI influencers in a gray area.
In that uncertainty, the AI influencer ecosystem is thriving. Some market research firms estimate the virtual influencer market could be worth more than $60 billion by 2030, up from around $12 billion this year. Cultural clout is growing too. There are AI influencer awards, beauty pageants, dedicated talent agencies representing synthetic creators, and a booming market of synthetic creators selling courses and tools promising to help people make and run fake creators of their own, often with the promise of faceless passive income. Some of it has the faintly pyramidal smell of an online gold rush, a few visible success stories and an awful lot of people selling shovels.
What happens next
My guess is that a reckoning is on the way. AI slop is already irritating, and there’s only so much of it a platform can carry until it is rendered practically unusable, especially given their persistent refusal to let users filter AI slop. Fake people pretending to be real are an even more intimate version of the same problem. But beyond labels and enforcement of existing rules, platforms mostly seem content to see what happens. To platforms, engagement is still engagement, whether it comes from a fake creator or a real one. So long as synthetic creators keep posting and don’t stray outside of existing rules, there seems to be little incentive to crack down.
There’s also a question of how sustainable the whole idea of having AI avatars running around online is. If so many are built just to make money from human users, what happens when the pool of human users dries up? There’s only so many people who will be willing to buy courses and tools to build influencers of their own, for example. That’s presuming social media can survive the influx of AI influencers. By definition, it requires some critical mass of humanity to keep things social. If left unchecked, networks will collapse under the weight of these fake people, as human users are inevitably driven away.
That could change if public anger keeps building. Backlash over deepfakes, impersonation, and synthetic spam is already forcing lawmakers and regulators to pay closer attention, particularly after incidents involving nonconsensual sexual deepfakes generated with tools like Grok. Europe’s AI Act could be a driver, at least as its transparency obligations for AI-generated content come into force. The regulations will require deployers of generative AI systems to clearly disclose AI-generated or manipulated content, which could pressure companies to step up flagging AI content or face potentially hefty fines. But even then, the focus is still largely on content, not whether the account posting it represents an actual person.
As with so much on social media, the burden falls back on users. Many platforms have effectively delegated the task of moderating AI content to users, relying on them to spot and report suspicious profiles. But self-moderation is a poor and unsustainable answer to something designed to evade notice. There is already a growing appetite for AI-free spaces. If platforms refuse to draw boundaries between real and unreal themselves, I expect users will draw them instead.
By the way

• A lot of the more high-profile AI influencers I’ve encountered recently have had an overtly political bent, which I feel could hasten the regulatory reckoning. Danny Bones, a fake white nationalist rapper funded by a far-right political party in the UK, is perhaps the best example of this I’ve seen so far.
• Like human influencers, many AI avatars are built around specific identities and communities, such as race, disability, politics, and nationality, like MAGA fantasy girl Jessica Foster, who leans heavily into sexualized Army aesthetics and Trumpism. Not all avatars align with their creators: Black AI model Shudu Gram, for example, was made by a white man. Emily Pellegrini is also the product of a man, Professor EP, who told me the character is built using content he licensed from an anonymous OnlyFans creator.
  Read this
• The headline of Jess Weatherbed’s recent story for The Verge says it all: “Let us filter AI slop, you cowards.”
• The Verge recently reported that grifters are using AI avatars of fake Black people to hawk mass-produced products via drop shipping at inflated prices on social media.
• Wired reported on the booming “AI Pimping” industry, where human creators are having their content stolen and monetized by AI avatars without their permission.
• Charlie Warzel’s podcast examined the incentives behind the proliferation of AI influencers and the exhaustion many feel when it comes to caring whether what we consume is real or not anymore.