Major K-pop agencies such as JYP Entertainment, YG Entertainment and Cube Entertainment have declared war on deepfake visual content created using images of their K-pop artists.

This comes at a time when deepfake crimes have attracted renewed attention in South Korea, after the revelation of online rooms in which users shared AI-generated pornographic videos of local women and acquaintances.

It is becoming more and more common for K-pop singers to fall victim to deepfake pornography, according to industry sources.

YG Entertainment announced on Sept. 2 that it was taking legal action against these activities.

“We are actively monitoring this widespread and malicious illegal activity, and making efforts to delete such illegal videos. We are also taking all possible legal actions. We will continue to respond vigorously and strictly to all illegal activities that seriously harm the reputation of our artists,” said YG Entertainment in a public statement.

(G)I-dle, NewJeans, Twice, Kwon Eun-bi and Blackpink are just a few of the artists who are victims of deepfake pornography makers.

“We can’t reveal the details of these deepfake pornography cases because it could ... spread the damage even further. These cases are identified through internal and external monitoring systems, and reports from fans. We have manuals for responding to such crimes,” an official at a K-pop agency told The Korea Herald.

Celebrities are particularly vulnerable to deepfake pornography because their level of media exposure means they have a high profile and a lot of images of them are readily available.

The use of deepfake technology to manipulate and distribute images of K-pop idols without authorization constitutes a violation of personal information and defamation, according to Jeong Jin-kwon, a lawyer who specializes in AI and start-up fraud cases.

"Public figures such as idols can suffer significant harm to their image and reputation. Even if no direct physical harm is inflicted, videos edited to sexualize or stimulate sexual desires without consent can be subject to penalties for sexual rights violations," Jeong said.

If these actions result in financial gain for the perpetrator, more severe penalties may apply, Jeong added.

Deepfake pornography trends

A report titled “2023 State of Deepfakes” released by the US-based cybersecurity company Security Hero showed that K-pop stars were particularly at risk.

It found a total of 95,820 videos posted on the top 10 deepfake pornography websites and 85 deepfake channels on platforms such as YouTube, Vimeo, and Dailymotion in July and August of last year.

"South Korean singers and actresses constitute 53 percent of the individuals featured in deepfake pornography and are the most commonly targeted group," said Security Hero in its report.

The report shows that the number of deepfake pornography cases increased 4.6-fold, surging to 21,019 in 2023 from 3,725 cases in 2022. It also noted that 9 out of 10 deepfake pornography videos were posted on sites specializing in deepfake porn.

The spread of deepfake pornography created with altered videos of K-pop artists performing on stage has raised criticism that there are not enough legal countermeasures to address the issue.

Under the Act on the Special Cases Concerning the Punishment of Sexual Crimes, individuals involved in editing, synthesizing, or processing deepfake pornography face penalties of up to five years in prison or a fine of up to 50 million won ($37,400). Distributors of such content for profit can be sentenced to up to seven years in prison.

Challenges in enforcing the law

However, among the 71 rulings on deepfake crimes made from 2020 to 2023 in Korea, only four cases resulted in imprisonment solely for the deepfake offense. Worse still, there are currently no laws addressing the act of downloading or viewing deepfake pornography.

“There are no penalties for activities such as acquiring, storing or using deepfake videos, meaning that punishment only applies to those who create or distribute these videos. This is a clear difference from laws regarding illegal recordings or copies, which penalize not only the production and distribution but also the possession, purchase, storage or viewing of such materials,” said lawyer Kim Ye-eun. “Even watching deepfake videos is morally wrong. There is a need to develop a system that can technologically identify deepfake content.”

“Addressing the deepfake issue also requires a shift in societal awareness. Educating people about the risks and legal responsibilities associated with deepfakes and emphasizing the importance of societal efforts to prevent the abuse of technology are crucial,” said Jeong.