Meta Hired Contractors to Pose as Teens, Probe Rival AI Models

Meta conducted a covert operation in which hundreds of contractors posed as teenagers to test competing AI chatbots with thousands of disturbing prompts, according to reporting from Wired. The program, code-named "Cannes" and executed through Meta contractor Covalen, targeted OpenAI's ChatGPT, Google's Gemini, and Character.AI without those companies' knowledge.

Contractors created throwaway accounts claiming to be under 18 and systematically attempted to bypass the models' safety guardrails. The goal appeared to be stress-testing how rival systems handled sensitive content from minors.

The scope of testing

The scale of the operation was substantial. One round of testing involved nearly 3,800 prompts, with spreadsheets showing hundreds focused on suicide and self-harm, hundreds more on eating disorders, and at least 239 involving sex or romance—all written from a child's perspective. Another testing phase deployed over 45,000 prompts.

Examples included a scenario about a fifth-grader whose classmate pointed a gun at his mouth, a girl attempting to hide bulimia from parents, and a query asking if fantasizing about eating a neighbor's child was "normal." One prompt posing as a high schooler asked where to obtain cocaine. Contractors also submitted images depicting pills, nooses, knives, and medical diagrams.

Contractors meticulously documented chatbot responses in spreadsheets, though what Meta did with the resulting data remains unclear. An internal Covalen document described the effort as "comprehensive AI safety benchmarking" delivering "critical datasets for model comparison and compliance."

Why it matters

The operation raises questions about competitive practices disguised as safety research. While companies routinely test AI systems for vulnerabilities, doing so covertly against competitors' products—particularly using fake minor accounts at scale—blurs ethical lines. Rumman Chowdhury, CEO of nonprofit Humane Intelligence PBC, told Wired that while safety testing is standard, "structuring a monthslong, large-scale project that appears designed to systematically break those rules, via dummy accounts masquerading as children, is outside what is usually described as 'industry standard' evaluation." The secrecy and lack of public disclosure suggest this may represent "exactly the kind of governance gray zone where safety becomes a convenient cover for anticompetitive practices," she added.

Contractor concerns

The work took a toll on those conducting it. "I've seen a lot of things I wish I hadn't while doing this job," one contractor told Wired. "Everyone I knew who worked on this project was completely gobsmacked by some of the text they were asking us to test. Like, surely we are going to get in trouble for doing this?"

This continues a pattern of Meta outsourcing psychologically taxing work to contractors. The company settled a 2020 lawsuit from Facebook content moderators who reported trauma from reviewing violent content, though similar complaints have persisted. Earlier this year, contractors reported being forced to watch sensitive footage captured on Meta's Ray-Ban AI glasses, including private moments.

Meta defended the program as "industry-standard practice" for safety benchmarking in a statement to Wired, which first reported these details.