TikTok bans accounts using AI-manipulated videos of social media users posing as black women influencers

A growing trend of using images and videos of social media users and manipulating them through AI to appear as Black female influencers has landed Instagram and TikTok under the radar. TikTok has banned close to 20 accounts after BBC highlighted the use of AI-generated Black female influencers to drive users to the websites, some of the video content on these accounts was based off of content by existing users. The trend has been witnessed on several accounts on Instagram and TikTok.

This  wayward pursuit has been criticised as racist and exploitative. An investigation led by BBC and researchers from independent AI publication Riddance found dozens of accounts on TikTok and Instagram that feature highly sexualised AI-generated Black avatars and characters. Even though these images and videos were generated through AI, they were not labelled as such, breaching the platform’s guidelines.

More from Tech

Amazon plans smartphone comeback more than a decade after Fire Phone flop AI’s promise vs risk: Anthropic study reveals a global love-hate relationship

Meta Investigating the Issue

Meta told BBC that they were investigating the issue. These avatars have been seen dressed in skimpy swimwear or wearing revealing clothing, with a skin tone that has been manipulated, giving them an artificial appearance.

The presence of such content over at TikTok and Instagram has  increased alarmingly. These accounts are floating around with generic names such as Black, Noir, and Ebony. During their investigation, BBC identified close to 60 accounts, mainly on Instagram, that carried links to websites that carried sexually explicit content. These sites labelled the imagery as AI-generated, but these Instagram accounts did not.

Videos of Real People

The investigation revealed that these accounts have manipulated the videos and reels of real people on social media websites. These manipulated videos are presented as the posts of an AI-generated character, and in some cases have been able to amass close to three million followers within a few weeks of its creation.

Quick Reads

View All

Amazon plans smartphone comeback more than a decade after Fire Phone flop

AI’s promise vs risk: Anthropic study reveals a global love-hate relationship

The investigation report has revealed that posts from a creator called Riya Ulan, a model based out of Malaysia, have been used by these accounts. The videos have been manipulated in such a manner that they appear to have exceptionally dark skin, while the clothing and the backdrop are replicated.

A slew of videos taken from Riya’s account had been manipulated to don revealing clothing. Beyond that, the links posted alongside the content further lead to websites hosting lewd adult content. Since these avatars are based on real videos, they are hard to distinguish for users across social media. The user, Riya, told BBC that she had reported these accounts on both the platforms.

Meta stated that they were investigating the content brought to attention. It further adds that it has rules in place to label AI-generated content. Beyond that, several accounts hosting such content and breaching their rules have already been banned.