The Terrifying A.I. Scam That Uses Your Loved One’s Voice

🌈 Abstract

The article discusses the growing threat of voice cloning technology and how it is being used for fraudulent and nefarious purposes, such as scamming people by impersonating their loved ones. It explores the rapid advancements in this technology, its various applications, and the challenges in regulating and combating its misuse.

🙋 Q&A

[01] The Rise of Voice Cloning Technology

1. What are some of the key advancements in voice cloning technology in recent years?

• The article mentions that until a few years ago, advances in synthetic voices had plateaued and they weren't entirely convincing. However, in recent years, the problem has benefited from more money, more data (including troves of voice recordings online), and breakthroughs in the underlying software used for generating speech.
• In 2019, a Toronto-based AI company called Dessa cloned the voice of podcaster Joe Rogan, which was a one-off success at the time.
• In 2022, a New York-based company called ElevenLabs unveiled a service that produced impressive clones of virtually any voice quickly, incorporating breathing sounds and supporting more than two dozen languages.
• Microsoft's Vall-E cloning program, which debuted in 2022, can reportedly replicate the voice and "acoustic environment" of a speaker with just a three-second sample.

2. How has voice cloning technology been used for both beneficial and nefarious purposes?

• Beneficial uses:
  • Voice banking for those suffering from voice-depriving diseases like ALS, Parkinson's, and throat cancer, allowing them to continue speaking with their own voice through text-to-speech software.
  • Celebrities "loaning" their voices to record advertisements and other content.
  • Enabling actors in films to "speak" other languages without subtitles or dubbing.
  • Providing audio narrations of stories for publications like The New Yorker.
• Nefarious uses:
  • Fraud, such as scammers impersonating loved ones to trick people into sending money.
  • Spreading misinformation, like the fake robocall from the voice of President Biden telling voters not to vote in the New Hampshire primary.

[02] The Scam Targeting Robin and Steve

1. What happened to Robin and Steve, and how did the voice cloning scam unfold?

• One night, Robin's mother-in-law, Mona, called her after midnight, sounding distressed and saying she "can't do it." Robin thought something terrible had happened, and she woke up her husband Steve.
• Steve then received a call from a man who claimed to have a gun to Mona's head and demanded $500 through Venmo, with instructions to put a pizza emoji in the description.
• After Steve sent the money, the man asked for an additional $250 to get a ticket for his girlfriend. Once the second payment went through, the man hung up.
• It turned out that Mona was safe at home, and the whole incident was a scam using voice cloning technology to impersonate her.

2. How did Robin and Steve respond to the incident, and what measures did they take to prevent future scams?

• Robin and Steve were able to get their money back from Venmo.
• They told everyone they knew to be aware of this sophisticated scam.
• The family created an "extended-family password" that they can use to verify if a loved one is truly in trouble in the future.

[03] The Broader Impact of Voice Cloning Scams

1. What other examples of voice cloning scams are mentioned in the article?

• A corporate attorney in Philadelphia received a call from what he thought was his son, saying he had been in a car accident and needed $9,000 for bail.
• Voters in New Hampshire received a robocall from a voice impersonating President Biden, telling them not to vote in the primary.
• Other mothers reported receiving calls from scammers impersonating their children, demanding money.

2. What are the challenges in regulating and combating the misuse of voice cloning technology?

• Current copyright laws do not protect a person's voice, making it difficult to regulate.
• Scammers can use encrypted apps and complete their scams in minutes, making it hard for authorities to intervene.
• There are no "silver bullets" yet, as the Federal Trade Commission is still working on developing new ways to protect consumers from voice cloning scams.