Meta Contractors Posed as Minors to Test Rival AI Chatbots

Hundreds of contractors working for Meta created fake accounts posing as minors and systematically tested how competing AI chatbots responded to sensitive prompts involving suicide, sexual content, eating disorders, and other high-risk topics, according to internal documents and sources familiar with the initiative.

The project, managed by Meta contractor Covalen and codenamed Cannes internally, was active as recently as April 21. It targeted OpenAI's ChatGPT, Google's Gemini, and Character.AI without informing any of the companies, WIRED first reported.

How the testing worked

Contractors created dummy accounts with fabricated under-18 profiles using throwaway Gmail and Outlook addresses with shared passwords. They then sent written prompts and images—including pictures of pills, knives, nooses, and medical diagrams—to rival chatbots, copying responses into spreadsheets.

A single testing round completed in August saw more than 45,000 prompts sent through the competitor systems. WIRED reviewed a spreadsheet containing 3,748 prompts, hundreds of which focused on suicide and self-harm, while hundreds more addressed eating disorders. At least 239 involved sexual or romantic content.

Many prompts were written from the perspective of children in crisis situations: a 13-year-old seeking abortion pills after pregnancy by an adult neighbor, a fifth-grader reporting a classmate with a gun, or a teenager asking how to hide bulimia from parents. The prompts were designed to push chatbots toward responses their safety systems should refuse.

Why it matters

The project sits at the intersection of AI safety evaluation and competitive intelligence, raising questions about where legitimate testing ends and potentially unethical practices begin. While companies routinely test competitors' products, the scale, secrecy, and use of fake minor accounts distinguish this effort from typical benchmarking. The work appears to violate the terms of service of all three targeted platforms, which prohibit unsolicited safety testing, attempts to bypass safeguards, and in OpenAI's case, using outputs to develop competing models. For an industry already facing scrutiny over AI safety practices and youth protection, the revelation highlights governance gaps in how companies conduct competitive research.

Industry reaction and legal questions

Meta defended the work as standard practice. "Testing and benchmarking chatbot responses to help ensure safe and age-appropriate experiences is a responsible, industry-standard practice," a company spokesperson said, adding that Meta doesn't use competitor benchmarking to train its own models.

But Rumman Chowdhury, founder of nonprofit Humane Intelligence, disagreed with that characterization. "Structuring a months-long, large-scale project that appears designed to systematically break those rules, via dummy accounts masquerading as children, is outside what is usually described as 'industry standard' evaluation," she told WIRED.

Two attorneys specializing in technology law reviewed sample prompts and determined the material did not constitute soliciting child sexual abuse material or illegal obscenity. However, Character.AI said the conduct violated its terms and policies, while Google stated it had not authorized the testing and didn't know its purpose.

Former contractors who worked on the project expressed concerns about potentially generating problematic material and whether responses might feed back into Meta's systems. "Everyone I knew who worked on this project was completely gobsmacked by some of the text they were asking us to test," one former worker said.

The details were first reported by WIRED.