Fake goods, fake spirit: China is using Deepfake anchors for propaganda masquerading as news

The “news broadcasters” appear stunningly real, but they are AI-generated Deepfakes in first-of-their-kind propaganda videos that a research report published Tuesday attributed to Chinese state-aligned actors.

The fake anchors — for a fictitious news outlet called Wolf News — were created by artificial intelligence software and appeared in footage on social media that seemed to promote the interests of the Chinese Communist Party, US-based research firm Graphika said in its report.

The ludicrousness of it all…
“This is the first time we’ve seen a state-aligned operation use AI-generated video footage of a fictitious person to create deceptive political content,” Jack Stubbs, vice president of intelligence at Graphika, told AFP.

In one video analysed by Graphika, a fictitious male anchor who calls himself Alex critiques US inaction over gun violence plaguing the country. In the second, a female anchor stresses the importance of “great power cooperation” between China and the United States.

Last year, Facebook owner Meta said it took down a Deepfake video of Ukrainian President Volodymyr Zelensky urging citizens to lay down their weapons and surrender to Russia.

What is Deepfake?
Advancements in AI have stoked global alarm over the technology’s potential for disinformation and misuse, with Deepfake images created out of thin air and people shown mouthing things they never said.

To put it very basically, a Deepfake is an AI-generated video of a person in which their face or body has been digitally altered so that they appear to be someone else. Deepfakes were first used in the adult film industry to meet the evergrowing demand of celebrity porn. However, the industry soon started curtailing its usage after certain users started uploading Deepfaked revenge porn, on websites. Nowadays Deepfakes are typically used maliciously or to spread false information.

China weaponising AI
There was no immediate comment from China on Graphika’s report, which comes just weeks after Beijing adopted expansive rules to regulate Deepfakes.

China enforced new rules last month that will require businesses offering Deepfake services to obtain the real identities of their users. They also require Deepfake content to be appropriately tagged to avoid “any confusion.”

The Chinese government has warned that Deepfakes present a “danger to national security and social stability.”

Graphika’s report said the two Wolf News anchors were almost certainly created using technology provided by the London-based AI startup Synthesia.

The website of Synthesia, which did not immediately respond to AFP’s request for comment, advertises software for creating Deepfake avatars “based on video footage of real actors.”

China and Spamouflage
Graphika said it discovered the Deepfakes on platforms including Twitter, Facebook and YouTube while tracking pro-China disinformation operations known as “spamouflage”.

“Spamouflage is a pro-Chinese influence operation that predominantly amplifies low-quality political spam videos,” said Stubbs.

“Despite using some sophisticated technology, these latest videos are much the same. This shows the limitations of using Deepfakes in influence operations — they are just one tool in an increasingly advanced toolbox.”

Read all the Latest News, Trending News, Cricket News, Bollywood News,
India News and Entertainment News here. Follow us on Facebook, Twitter and Instagram.