New AI voice cloning tools have the potential to spread false information on social media

In a video from a January 25 report, President Joe Biden talks about tanks. But a doctored version of the video has amassed hundreds of thousands of views this week on social media, making it look like he gave a speech that attacks transgender people.

Digital forensics experts say the video was created using a new generation of artificial intelligence tools, which allow anyone to quickly generate sound simulating a person’s voice with just a few clicks on a button. And while Biden’s social media clip may not have fooled most users this time around, the clip shows how easy it is now for people to generate ‘deepfake’ videos. hateful and filled with misinformation that could do harm in the real world.

“Tools like this are basically going to add more fuel to the fire,” said Hafiz Malik, a professor of electrical and computer engineering at the University of Michigan who focuses on multimedia forensics. “The monster is already on the loose.”

It arrived last month with the beta phase of ElevenLabs’ text-to-speech platform, which allowed users to generate realistic sound from any person’s voice by uploading a few minutes of audio samples and typing any text to say.

The startup says the technology was developed to duplicate audio in different languages ​​for movies, audiobooks, and games to preserve the speaker’s voice and emotions.

Social media users quickly began sharing an AI-generated audio sample of Hillary Clinton reading the same transphobic text featured in Biden’s clip, as well as fake audio clips of Bill Gates supposedly saying the vaccine COVID-19 causes AIDS and actress Emma Watson supposedly reading Hitler’s manifesto ‘Mein Kampf’.

Shortly after, ElevenLabs tweeted that it was seeing “a growing number of instances of voice cloning misuse” and announced that it was now exploring safeguards to combat abuse. One of the first steps was to make the feature available only to those who provide payment information. Initially, anonymous users could access the voice clone tool for free. The company also claims that if something goes wrong, it can trace all generated sound back to the creator.

But even the ability to track creators won’t mitigate the tool’s damage, said Hany Farid, a professor at the University of California, Berkeley, who focuses on digital forensics and disinformation.

“The damage is done,” he said.

As an example, Farid said bad actors could rock the stock market with a fake sound of a top CEO saying earnings are down. And there’s already a clip on YouTube that used the tool to edit a video to make it appear that Biden said the US was launching a nuclear attack on Russia.

Free and open-source software with the same capabilities has also emerged online, meaning paywalls on commercial tools are not an obstacle. Using a free online template, the AP generated audio samples to sound like actors Daniel Craig and Jennifer Lawrence in just minutes.

“The question is where to point and how to put the genie back in the bottle?” Malik said. “We can’t do it.”

When deepfakes first hit the headlines about five years ago, they were fairly easy to detect because the subject wasn’t blinking and the sound sounded robotic. This is no longer the case as the tools become more sophisticated.

The edited video of Biden making derogatory comments about transgender people, for example, combined AI-generated audio with an actual clip of the president, taken from a January 25 CNN live broadcast announcing the US send-off. tanks in Ukraine. Biden’s mouth was manipulated in the video to match the audio. While most Twitter users acknowledged that the content was not something Biden was likely to say, they were nonetheless shocked at how realistic it sounded. Others seemed to believe it was real – or at least didn’t know what to believe.

Hollywood studios have long been able to warp reality, but access to this technology has democratized without considering the implications, Farid said.

“It’s a combination of very, very powerful AI-based technology, ease of use, and the fact that the model looks like it is: let’s put it on the internet and see what happens next” , said Farid.

Audio is just one area where AI-generated misinformation poses a threat.

Free online AI image generators like Midjourney and DALL-E can produce photorealistic images of war and natural disasters in the style of traditional media with a simple text prompt. Last month, some school districts in the United States began blocking ChatGPT, which can produce readable text — like student essays — on demand.

ElevenLabs did not respond to a request for comment.