An open wound of the digital age: disenfranchisement of victims of AI pornography

Father

Professional
Messages
2,605
Reputation
4
Reaction score
585
Points
113
We investigate the dark corners of the Internet where personal data becomes a commodity.

In cyberspace, a new scandal is breaking out related to the use of generative AI tools to create pornographic images without the consent of the persons depicted on them.

One Telegram user who advertises his services on the X social network offers to create pornographic images of any person for just $ 10, using just a few photos. All interaction between clients and the service provider is conducted in a specially created Telegram channel.

In one of the chats of this channel, users post real photos of people, apparently taken from their social networks, and there are also pornographic images already created with the help of AI. All this is done both to demonstrate the capabilities of the service to potential customers, and, possibly, to humiliate the honor and dignity of female victims who have fallen under the hot hand of cyber villains.

At the time of its study by security researchers, the Telegram channel described above had 35 rooms, each of which contains generated pornographic materials of famous women, including youtubers, Twitch streamers and Instagram influencers. The targets also included minors, but in these cases, not fully nude images were distributed, but only photos in swimsuits.

0hco6c72qck6iu923tt1cfqn10e6c8db.png


As noted above, this Telegram channel openly advertises its services on the social network X (formerly Twitter*), which raises questions about the ability and readiness of Elon Musk's platform to moderate malicious content. A few months ago, viral pornographic images of Taylor Swift created with the help of AI also spread at a breakneck pace on the popular social network, collecting millions of views before X deleted them and blocked the accounts of all involved.

The researchers also found a similar Discord community, presumably created by the same author. Therefore, with a certain degree of probability, we can assume that even if the channel in the "blue messenger" is blocked, all its malicious activity will simply "move" to Discord.

Telegram and Discord regularly declare their commitment to strict moderation of malicious content on their platforms, but the problem of obscene images created using AI remains relevant and causes concern to people around the world.

Generative AI technologies, such as Stable Diffusion, make it easy to create realistic images using only text hints and a reference photo. This opens up a wide range of opportunities for abuse, including the creation of pornographic content without the consent of the persons depicted.

The use of mobile applications with a similar principle of operation, such as "Undress", "nudify" and "ClothOff", has previously led to a loud scandal caused by active school harassment of girls affected by porn deals in the Spanish town of Almendralejo.
 
Top