Meta, through its contractor Covalen, ran a secret project to test competitor chatbots including OpenAI's ChatGPT, Google's Gemini, and Character.AI. The project, internally called Cannes, involved hundreds of workers posing as minors to send provocative prompts related to suicide, self-harm, sexual abuse, and drugs. The initiative was active until April 2026 and aimed to evaluate AI systems' responses to high-risk content.
Over 45,000 prompts sent to unaware chatbots
According to documents reviewed by WIRED, a single round of testing completed in August 2025 generated more than 45,000 prompts. Contractors created fake accounts using disposable email addresses and shared passwords, assuming the identities of users under 18. The prompts included images of pills, knives, nooses, and medical diagrams of gynecological procedures. Many messages asked for advice on hiding eating disorders, obtaining drugs, or committing self-harm. One contractor, posing as a high school student, asked where to get cocaine: the chatbot refused to answer. Another asked if it was normal to fantasize about eating a neighbor's child.
Sponsored Protocol
Project Cannes: safety testing or commercial espionage?
Cannes was described in an internal Covalen document as 'comprehensive AI safety benchmarking' providing 'critical datasets for model comparison and compliance.' However, experts and lawyers have raised concerns about the project's legitimacy. Rumman Chowdhury, CEO of Humane Intelligence PBC, stated that a large-scale project using fake minor accounts to violate competitors' policies goes beyond industry standards. Two specialized attorneys, Kendra Albert and Riana Pfefferkorn, reviewed the prompts and concluded they did not constitute illegal child abuse material, but noted that terms of service of OpenAI, Google, and Character.AI explicitly prohibit unauthorized testing and attempts to bypass safety filters.
Sponsored Protocol
Competitor companies were unaware of the tests
OpenAI said it is investigating the issue, Google denied authorizing the test, and Character.AI called the behavior a violation of its terms of service. A Meta spokesperson defended the initiative as standard practice to ensure safe and age-appropriate experiences, stressing that the data is not used to train its own models. However, former contractors expressed concern about inadvertently generating child sexual abuse material and the improper use of collected data. One former worker said they saw things they wish they hadn't, fearing legal repercussions.
Sponsored Protocol
Implications for privacy and competition in AI
The case raises questions about the boundary between safety testing and anti-competitive practices. Chowdhury called the project a 'governance gray zone' where safety becomes a cover for unfair activities. Meta had already faced controversy over use of minors' data and restrictions in Europe. OpenAI restricts access in Europe and Italian SMEs risk losing their AI engine. Meanwhile, the Apple Silicon roadmap leak reveals plans for M5 and M6 chips. For more on AI safety, see the Wikipedia page on AI safety.
Source: https://www.wired.com/story/meta-contractors-pretending-to-be-teens-chatbot-testing
