Policy
AI-Generated Child Abuse Material Surged 1,325% in One Year
UN officials warn that predators are weaponizing artificial intelligence faster than protection efforts can respond, with deepfakes and chatbots creating unprecedented risks.
Key takeaways
- Reports of child sexual abuse material linked to generative AI increased 1,325% between 2023 and 2024, according to US authorities.
- At least 1.2 million children in 11 countries had images manipulated into sexually explicit deepfakes in the past year alone.
- Predators use chatbots to groom children, deepfake apps to extort them, and AI profiling to target vulnerable victims with precision.
- Children are turning to AI agents for intimate conversations and answers adults won't provide, often without any safety oversight.
- UN officials argue voluntary tech industry measures have failed and call for mandatory child safety standards in AI governance.
Artificial intelligence is accelerating child exploitation at a pace that far outstrips current protective measures, senior United Nations officials warned at a high-level General Assembly event on Thursday.
Reports of child sexual abuse material connected to generative AI jumped 1,325 percent between 2023 and 2024, according to the US National Center for Missing and Exploited Children. Yet most AI-facilitated crimes against children remain unreported and under-prosecuted, said Dr. Najat Maalla M'jid, the UN Special Representative of the Secretary-General on Violence against Children.
"We lost the battle on social media because we were lagging behind, and we are now losing the battle on artificial intelligence because we are still lagging behind," Dr. M'jid told the gathering at UN Headquarters in New York.
The scale of AI-enabled exploitation
In the past year alone, at least 1.2 million children across 11 countries had their images manipulated into sexually explicit deepfakes, UN Deputy Secretary-General Amina Mohammed said, citing recent UNICEF research.
Predators are deploying AI tools across multiple vectors. Chatbots gain children's trust, extract personal information, and coerce them into sharing explicit images. So-called "nudification" apps create deepfakes used for humiliation, threats, and extortion. Profiling and predictive targeting enable traffickers and criminal groups to identify and reach vulnerable children with greater precision.
John Trenary, who has investigated child exploitation for more than 25 years, described the challenge as "self-perpetuating." Because AI can automate and generate wholly synthetic content at scale, investigators struggle to keep pace. "As soon as we get a handle on one way predators misuse it, something else comes out," he said.
Kay Chau of the Tech Coalition noted that perpetrators may use chatbots to facilitate abusive role-play or generate grooming scripts, while agentic AI systems—those capable of carrying out tasks independently—introduce entirely new risk categories.
Children turn to AI agents for answers adults won't provide
Mariana Rozo-Paz of the Datasphere Initiative recounted working with teenage girls in rural Colombia on digital literacy. When asked to draw what the internet looked like, the girls drew a girl named Lucia—their AI agent.
"We talk to Lucia every day," they explained. "Whenever adults don't want to answer specific questions, we ask Lucia. She's always there for us. She has intimate mode, best friend mode."
The anecdote underscores how deeply AI systems have embedded themselves in children's lives, often filling gaps left by adults—and doing so without meaningful safety oversight.
Why it matters
The explosive growth in AI-generated abuse material represents a fundamental shift in how predators operate. Unlike traditional methods, AI enables mass production of synthetic content and automated targeting at virtually no marginal cost. This isn't a problem that will resolve through incremental improvements—it requires immediate regulatory intervention and mandatory safety standards before AI systems reach children. The gap between technological capability and protective governance is widening, not closing.
Calls for mandatory standards
"We would never accept a children's toy being put on the market without proper safety checks, but AI systems are reaching children without the same scrutiny," Mohammed said. "We cannot leave their safety to the goodwill of technology companies."
In July, Secretary-General António Guterres launched the AI Child Safety Pledge, which aims to ensure AI systems are certifiably safe before reaching children and establishes zero tolerance for AI-generated child sexual abuse material.
Dr. M'jid argued that voluntary approaches by the tech sector have proven insufficient, calling for global standards that include mandatory child safety and privacy by design at the core of AI governance.
While AI presents grave risks, it can also aid detection of abuse material, victim identification, and early grooming indicators, Mohammed noted.
A young person in Sweden, quoted in a World Childhood Foundation study, captured the urgency: "The technology is really cool. But you can tell that the people in charge of controlling it cannot keep up."
"Children should never have to tell us that the adults responsible for their safety cannot keep up," said the foundation's Secretary General, Paula Guillet de Monthoux. "It is our responsibility to prove that we can."
These details were first reported by UN News.