4chan users who created a game leveraging popular AI image generators appear to be at least partly responsible for the flood of fake images sexualizing Taylor Swift that went viral last month.
Graphika researchers, who study how communities are manipulated online, traced the fake Swift images to a 4chan discussion board that is “increasingly” dedicated to posting “offensive” content generated by AI, according to the New York Times. reported. Fans of the chatroom participate in daily challenges, Graphika reported, sharing tips for bypassing the AI's image generator filters and showing no signs of stopping their game anytime soon.
“Some 4chan users have expressed their stated goal of trying to defeat the protections of traditional AI image generators rather than creating realistic sexual content with alternative open source image generators,” Graphika reported. “They also shared several behavioral techniques for creating image prompts, attempting to avoid bans, and successfully creating sexually explicit celebrity images.”
Ars reviewed a thread flagged by Graphika in which users were specifically challenged to use Microsoft tools such as Bing Image Creator and Microsoft Designer, as well as OpenAI's DALL-E.
“Good luck,” the original poster wrote, while encouraging other users to “be creative.”
OpenAI denied that any of the Swift images were created using DALL-E, while Microsoft continued to claim that it was investigating whether any of its AI tools had been used.
Cristina López G., senior analyst at Graphika, noted that Swift is not the only celebrity targeted in the 4chan thread.
“Although Taylor Swift's viral pornographic images have brought mainstream attention to the issue of AI-generated, non-consensual intimate images, she is far from the only victim,” López G. said. In the 4chan community where these images originated, she is not even the most frequently targeted public figure. This shows that anyone can be targeted in this way, from global celebrities to schoolchildren.”
Originally 404 media reported that the harmful Swift images appeared to originate from 4chan and Telegram channels before spreading to X (formerly Twitter) and other social media. In an attempt to stop the spread, X took the drastic step of blocking all searches for “Taylor Swift” for two days.
But López G. said Graphika's findings suggest that platforms will continue to risk being inundated with offensive content as long as 4chan users are determined to continue challenging each other to overturn the generators' filters. pictures. Rather than expecting platforms to track down harmful content, López G. recommended that AI companies get ahead of the problem, taking responsibility for the results by paying attention to the evolving tactics of toxic online communities which report precisely how they circumvent the safeguards.
“These images come from a community of people motivated by the 'challenge' of circumventing the safeguards of generative AI products, and the new restrictions are seen as just another obstacle to 'defeat,'” said López G .. “It is important to understand the playful nature of this malicious activity in order to prevent further abuse at the source.”
Experts told the Times that 4chan users were likely motivated to participate in these challenges for bragging rights and to “feel connected to a larger community.”