Meta’s policies on non-consensual deepfake images need updating, including wording that’s “not sufficiently clear,” the company’s oversight panel said Thursday in a decision on cases involving AI-generated explicit depictions of two famous women.
The quasi-independent Oversight Board said in one of the cases, the social media giant failed to take down the deepfake intimate image of a famous Indian woman, whom it didn’t identify, until the company’s review board got involved.
Deepake nude images of women and celebrities including Taylor Swift have proliferated on social media because the technology used to make them has become more accessible and easier to use. Online platforms have been facing pressure to do more to tackle the problem.
The board, which Meta set up in 2020 to serve as a referee for content on its platforms including Facebook and Instagram, has spent months reviewing the two cases involving AI-generated images depicting famous women, one Indian and one American. The board did not identify either woman, describing each only as a “female public figure.”
Meta said it welcomed the board’s recommendations and is reviewing them.
One case involved an “AI-manipulated image” posted on Instagram depicting a nude Indian woman shown from the back with her face visible, resembling a “female public figure.” The board said a user reported the image as pornography but the report wasn’t reviewed within a 48 hour deadline so it was automatically closed. The user filed an appeal to Meta, but that was also automatically closed.
It wasn’t until the user appealed to the Oversight Board that Meta decided that its original decision not to take the post down was made in error.
Meta also disabled the account that posted the images and added them to a database used to automatically detect and remove images that violate its rules.
In the second case, an AI-generated image depicting the American women nude and being groped were posted to a Facebook group. They were automatically removed because they were already in the database. A user appealed the takedown to the board, but it upheld Meta’s decision.
The board said both images violated Meta’s ban on “derogatory sexualized photoshop” under its bullying and harassment policy.
However it added that its policy wording wasn’t clear to users and recommended replacing the word “derogatory” with a different term like “non-consensual” and specifying that the rule covers a broad range of editing and media manipulation techniques that go beyond “photoshop.”
Deepfake nude images should also fall under community standards on “adult sexual exploitation” instead of “bullying and harassment,” it said.
When the board questioned Meta about why the Indian woman was not already in its image database, it was alarmed by the company’s response that it relied on media reports.
“This is worrying because many victims of deepfake intimate images are not in the public eye and are forced to either accept the spread of their non-consensual depictions or search for and report every instance,” the board said.
The board also said it was concerned about Meta’s “auto-closing” of appeals image-based sexual abuse after 48 hours, saying it “could have a significant human rights impact.”
Meta, then called Facebook, launched the Oversight Board in 2020 in response to criticism that it wasn’t moving fast enough to remove misinformation, hate speech and influence campaigns from its platforms. The board has 21 members, a multinational group that includes legal scholars, human rights experts and journalists.
