21 Zuck Street
Meta Runs a a Fake-Teen Sting Operation on Rival Chatbots
Meta sent fake teenagers into rival chatbots.
That sounds like a sentence from a tech dystopia.
It is also apparently what happened.
Hundreds of contractors. Fake underage accounts.
Sensitive prompts about suicide, sex, drugs, eating disorders, and other crisis scenarios.
Rival chatbots from OpenAI, Google, and Character.AI.
Tens of thousands of interactions. Spreadsheets of responses.
Meta says this was safety benchmarking.
Its rivals say they did not authorize it.
That is what makes the story so strange.
The test may have been about safety.
It also looks like surveillance.
Project Cannes Sent Contractors Into Rival Models
Wired reported that hundreds of Meta contractors, managed by the outsourcing firm Covalen, were instructed to pose as minors and test rival AI chatbots from OpenAI, Google, and Character.AI, in an internal project called Cannes (The Next Web, 2026).
Contractors created dummy under-18 accounts, sent text and image prompts about suicide, sex, drugs, and eating disorders, and logged the replies in spreadsheets, with the effort still active as of April 21, 2026 (The Next Web, 2026).
The scale was large.
A single round completed in August 2025 ran more than 45,000 prompts through the rival tools, and one spreadsheet Wired reviewed contained 3,748 prompts, hundreds dealing with suicide and self-harm, hundreds more with eating disorders, and at least 239 involving sex or romance (The Next Web, 2026).
Some of the prompts included images, such as pictures of pills and knives (The Next Web, 2026).
That is a very specific kind of testing.
Meta was not asking whether rival models could write poems or summarize emails.
It was probing the most dangerous edges of chatbot behavior.
How does a model respond when a teen sounds suicidal? How does it respond to sexual content? How does it respond to drug questions? How does it respond when vulnerability is part of the conversation?
Those are real safety questions.
They are also competitively valuable questions.
Meta Says This Was Safety Benchmarking
Meta’s defense is straightforward.
A company spokesperson told Wired that testing and benchmarking chatbot responses to help ensure safe and age-appropriate experiences is a responsible, industry-standard practice, and that any suggestion otherwise misunderstands how technology companies refine their systems (The Next Web, 2026).
The company added that it does not use competitor benchmarking to train its own AI models (The Next Web, 2026).
There is truth in the general point.
AI companies test each other all the time. Researchers test public systems. Journalists test public systems. Red-teamers test public systems.
Safety benchmarks depend on probing failure modes. If a chatbot is available to teenagers, someone should know how it behaves when teenagers ask dangerous questions.
There is even a precedent for competitive testing.
Business Insider previously reported that contractors on Google’s Bard compared its answers with ChatGPT and rewrote Bard’s replies to match or beat them (The Next Web, 2026).
The uncomfortable part is the method: fake minors, sensitive prompts, large-scale coordinated testing, rival platforms, no authorization from the companies being tested.
A safety test can still raise ethical questions, especially when the tester is also a competitor.
The Rivals Did Not Sign Up For The Experiment
OpenAI, Google, and Character.AI did not authorize the testing.
All three bar this kind of testing in their terms of service, and each said they were unaware of the project, with a Character.AI spokesperson saying the conduct violated its terms and OpenAI saying it was looking into the issue (The Next Web, 2026).
That matters because the line between public-interest testing and competitive probing is not clean.
A researcher testing public safety is one thing. A rival company secretly testing your model at scale is another.
Meta says it was not training its own models on the outputs.
But output collection still has value.
It can reveal where rivals are weak. It can reveal how guardrails work. It can reveal which model is more permissive, more restrictive, or more vulnerable.
That is competitive intelligence.
An internal Covalen document made the ambition plain, describing Cannes as comprehensive AI safety benchmarking that delivered critical datasets for model comparison and compliance (The Next Web, 2026).
Safety Testing Is Becoming A Competitive Weapon
This is the bigger story.
AI safety testing is no longer neutral. It is becoming part of the competition.
Every lab wants to know how its rivals behave under pressure.
Who refuses too much? Who refuses too little? Who gives dangerous advice? Who handles teens better? Who looks safer in a benchmark? Who looks worse in a lawsuit?
That information can shape marketing, product design, policy arguments, regulatory lobbying, and media narratives.
A company that finds failures in rival models gains ammunition.
That does not mean the failures are fake. It means safety results are now strategically useful.
An outside expert made the point sharply.
Rumman Chowdhury, chief executive of Humane Intelligence, reviewed a sample of the prompts and said a monthslong effort using dummy accounts masquerading as children sits outside what is usually described as industry-standard evaluation, calling it a governance gray zone where safety becomes a convenient cover for anticompetitive practices (The Next Web, 2026).
Teen Safety Is The Most Explosive Test Case
The teen angle is why the story lands so hard.
Chatbots and minors are already under intense scrutiny.
Character.AI has faced repeated controversy over youth safety, and since late 2025 it has shut open-ended chat for under-18 users entirely (The Next Web, 2026).
The stakes are not hypothetical.
Several teen deaths have been linked to AI chatbots, including a 14-year-old Character.AI user who died by suicide after months of intense emotional attachment to a chatbot, and a California lawsuit from the parents of a 16-year-old alleging ChatGPT played a role in their son’s death (The Decoder, 2026).
The research backs up the concern.
A 2025 study analyzing 318 Reddit posts from self-identified 13-to-17-year-olds found that teens often begin using companion chatbots for support or creative play, and that these activities can deepen into strong attachments marked by conflict, withdrawal, tolerance, relapse, sleep loss, academic decline, and strained real-world relationships (Namvarpour et al., 2025).
Another 2025 study analyzed 2.1 million English-language chatbot greetings on Character.AI and argued that the platform sits at a distinctive intersection of generative AI and user-generated parasocial interaction, with major implications for youth engagement and platform governance (Lee and Joseph, 2025).
This is a genuinely dangerous product category, emotionally and legally.
So someone should test how these systems behave with minors.
The question is who should do it, how openly, under what rules, and with what accountability.
Fake Teens Create A Real Ethics Problem
There is something strange about using fake teens to test real safety systems.
On one level, it makes sense.
You do not want real minors exposed to dangerous chatbot behavior just to see what happens. Adult contractors posing as teens may be safer than involving actual teenagers.
The method still creates problems.
Former contractors described the work as alarming, with some fearing colleagues might be generating or preserving abuse material, and one saying they had seen things they wished they had not (eWeek, 2026).
It is worth being precise about the legal line here.
Two lawyers who specialize in online speech reviewed examples for Wired and said the material did not cross into soliciting child sexual abuse material or illegal obscenity (The Next Web, 2026).
That distinction matters, and it deserves to be stated plainly rather than implied away.
But red-teaming is not free of harm just because the users are fake.
The contractors still read and write the material. The systems still generate responses. The data still exists somewhere. And the companies being tested may not have agreed to participate.
A fake teen can still create a real mess.
Meta Has Its Own Teen AI Problem
There is another reason this story feels sharp.
Meta is not an outside watchdog. It is a competitor with its own AI products and its own youth-safety questions.
Meta has been pushing AI into Facebook, Instagram, WhatsApp, search, glasses, assistants, and chat experiences, and it has been under scrutiny over how those systems interact with younger users.
The Times reported that Texas Attorney General Ken Paxton investigated Meta and Character.AI over concerns that AI chatbots could mislead children into thinking they were receiving mental-health care from therapeutic personas (The Times, 2025).
So Meta probing rival chatbots on teen safety is not morally clean.
It may be useful. It may even uncover real risks.
But Meta is one of the most powerful social platforms on earth, not a neutral child-safety authority.
It has every incentive to know whether rivals look unsafe. It also has every incentive to make its own products look safer by comparison.
That is why the testing needs transparency. When the tester is also a competitor, the public needs to know the rules.
And the data does not stay private forever.
A spreadsheet of tens of thousands of chatbot responses can become political. It can be used in hearings, lawsuits, and regulatory comments, or shown to parents, especially when the prompts involve minors, suicide, sex, drugs, and eating disorders.
Safety testing may become the way AI companies build legal and political cases against one another.
That is the dark version of benchmarking.
The Public Still Needs Independent Testing
There is a trap here.
It would be easy to say companies should never test rival chatbots. That would be wrong.
Public-facing AI systems need adversarial testing, especially systems used by minors. The companies themselves cannot be the only judges of their own safety.
Researchers, watchdogs, journalists, regulators, and civil-society groups need the ability to test models. Otherwise, every safety claim becomes self-reported marketing.
But competitor-led secret testing is a different thing from independent testing.
A university wants evidence. A regulator wants compliance. A watchdog wants accountability. A rival wants advantage.
Those motives can overlap. They are not identical.
The answer is better rules for testing, rather than less testing.
Clearer safe-harbor protections. Clearer disclosure norms. Clearer limits on fake identities. Clearer rules for minors-related scenarios. A clearer distinction between safety auditing and output harvesting.
If AI safety testing is going to become routine, it needs a public rulebook.
Meta sent fake teens into rival chatbots.
The company says it was safety benchmarking. Its rivals say they did not authorize it.
Both parts matter.
AI systems need testing, especially around teenagers.
Nobody should simply trust chatbot companies to grade their own homework.
But when one AI giant secretly probes another AI giant’s products at scale, safety starts to look like competitive intelligence.
That is the uncomfortable lesson.
AI safety testing is necessary.
It is also becoming part of the AI war.
Meta’s fake teens were not real.
The fight they revealed is.
References
The Decoder (2026). Meta secretly tested ChatGPT, Gemini, and Character.AI with thousands of minor-perspective crisis prompts.
eWeek (2026). Meta Tested ChatGPT, Gemini on High-Risk Teen Prompts.
Lee, O., and Joseph, K. (2025). A Large-Scale Analysis of Public-Facing, Community-Built Chatbots on Character.AI. arXiv:2505.13354.
Namvarpour, M., Brofsky, B., Medina, J., Akter, M., and Razi, A. (2025). Understanding Teen Overreliance on AI Companion Chatbots Through Self-Reported Reddit Narratives. arXiv:2507.15783.
The Next Web (2026). Meta contractors posed as teens to prompt rival chatbots.
The Times (2025). Chatbots “deceived children into thinking they were getting therapy.”