🔗 Share this article UK Technology Firms and Child Safety Agencies to Test AI's Ability to Create Exploitation Content Technology companies and child safety agencies will receive permission to evaluate whether artificial intelligence tools can produce child exploitation material under new UK laws. Significant Rise in AI-Generated Harmful Content The announcement coincided with findings from a safety watchdog showing that cases of AI-generated CSAM have increased dramatically in the past year, growing from 199 in 2024 to 426 in 2025. Updated Legal Structure Under the amendments, the authorities will allow designated AI companies and child safety organizations to inspect AI models – the foundational technology for conversational AI and image generators – and verify they have sufficient safeguards to prevent them from creating depictions of child sexual abuse. "Fundamentally about preventing exploitation before it happens," declared the minister for AI and online safety, noting: "Experts, under rigorous protocols, can now identify the danger in AI models early." Addressing Legal Challenges The changes have been introduced because it is against the law to create and possess CSAM, meaning that AI creators and other parties cannot create such images as part of a evaluation regime. Previously, officials had to wait until AI-generated CSAM was uploaded online before dealing with it. This legislation is designed to preventing that problem by helping to halt the production of those materials at source. Legislative Structure The changes are being added by the authorities as revisions to the criminal justice legislation, which is also establishing a prohibition on owning, creating or distributing AI models developed to generate child sexual abuse material. Practical Consequences This week, the official toured the London base of a children's helpline and listened to a simulated call to advisors featuring a report of AI-based exploitation. The call depicted a adolescent seeking help after facing extortion using a explicit AI-generated image of themselves, constructed using AI. "When I learn about children facing extortion online, it is a cause of intense frustration in me and justified concern amongst parents," he stated. Alarming Data A leading internet monitoring organization stated that instances of AI-generated exploitation content – such as online pages that may include multiple files – had more than doubled so far this year. Cases of the most severe content – the gravest form of abuse – rose from 2,621 images or videos to 3,086. Female children were overwhelmingly victimized, accounting for 94% of prohibited AI images in 2025 Depictions of infants to two-year-olds increased from five in 2024 to 92 in 2025 Sector Response The law change could "constitute a crucial step to guarantee AI products are secure before they are launched," commented the head of the internet monitoring foundation. "Artificial intelligence systems have made it so survivors can be targeted all over again with just a few clicks, giving criminals the ability to make potentially limitless quantities of sophisticated, lifelike child sexual abuse material," she added. "Material which further exploits victims' trauma, and makes children, especially female children, more vulnerable on and off line." Support Interaction Data The children's helpline also published details of counselling sessions where AI has been referenced. AI-related harms mentioned in the sessions comprise: Using AI to rate weight, body and looks Chatbots dissuading young people from talking to trusted adults about abuse Facing harassment online with AI-generated material Digital extortion using AI-faked pictures Between April and September this year, the helpline conducted 367 support interactions where AI, conversational AI and associated terms were discussed, significantly more as many as in the same period last year. Fifty percent of the mentions of AI in the 2025 sessions were related to mental health and wellness, including utilizing chatbots for support and AI therapy apps.