OctoML says it has ended its business relationship with Civitai days after an investigation by 404 Media revealed the text-to-image platform was being used to generate images that “could be categorized as child pornography.” Civitai previously relied on OctoML’s OctoAI for image generation. While OctoML initially indicated it would continue working with Civitai and introduced new measures to curb the creation of harmful images, 404 Media reported on Saturday that it has now decided to cut ties with the platform altogether.
According to 404 Media’s December 5 report, internal communications showed that OctoML was aware some Civitai users were creating sexually explicit material that included nonconsensual images of real people and pornographic depictions of children. In a followup report this weekend, the publication noted that OctoML rolled out a filter to block the generation of all NSFW content on Civitai before announcing its decision to pull out. Civitai also added new moderat
