5 ms·

I have yet to hear any example of good quality voice deepfakes. Every example I've heard so far was completely unconvincing, and absolute trash compared to huma

I have yet to hear any example of good quality voice deepfakes. Every example I've heard so far was completely unconvincing, and absolute trash compared to human impersonators. Speaking of, human voice impersonation has been possible and quite convincing for forever. Audio deepfakes, when they finally arrive for real, will simply reduce the effort required for voice impersonation. They do not make possible new feats of impersonation that were never possible before. They can still have an effect on society for sure, but I believe the concern is out of proportion with the effect.

chrisco255 6y ago
The effort will be reduced to typing sentences and selecting emotions from a dropdown menu. That opens up a whole new can of worms for a society that already has trust issues.
- exporectomy 6y ago
  Is that your own society? How would it exacerbate your trust issues? Surely making people doubt untrustworthy things is good?
  - garg 6y ago
    Making people doubt untrustworthy things by educating them through a proper process is good. Exposing an unsuspecting/uneducated population to manipulated content designed to deceive/fool them for nefarious purposes is not an ideal process if the goal is to make the general public have better judgement. This is a global issue. For examples, please look into the societal concerns surrounding 'deep fakes', and 'fake news' for answers to your first two questions.
    - exporectomy 6y ago
      Pitting people in a social game of who can be fooled the easiest sounds like a more effective approach than reading a dry textbook at them. There's shame in being seen as gullible. This happens already in casual conversation and people already have opportunities to believe false information from all over the internet or in books or tabloid newspapers or told to them by their peers or parents. I don't believe the concerns about deep fakes and fake news. I think they're no worse than what we've always had. Those fears are exaggerated by arrogant hand-wringers who somehow assume their own beliefs are exempt from manipulation and it's only intellectually inferior "other" people who are susceptible. They're so fixated on the popular political ideas that they forget they should be on a crusade against religion which is the elephant in the room when it comes to people being misled.
BikiniPrince 6y ago
The Jordan Peterson df was quite good. I made a few sound bytes to send to some friends as a gag, but they were already aware it was in the wild. The compression is probably the worst here, but it was fun when it was up. https://www.notjordanpeterson.com/ https://www.notjordanpeterson.com/
spyder 6y ago
It was already successfully used to "Scam A CEO Out Of $243,000": https://www.forbes.com/sites/jessedamiani/2019/09/03/a-voice-deepfake-was-used-to-scam-a-ceo-out-of-243000/?sh=321ae41e2241 https://www.forbes.com/sites/jessedamiani/2019/09/03/a-voice... Probably it' easier because phone calls aren't high quality.
- arnaudsm 6y ago
  How did they prove the voice was an AI and not a human impersonator ?
  - tyingq 6y ago
    They didn't, it was all speculation, mostly speculation from the person that received the call and the company's insurance firm.
ansk 6y ago
If you look at the latest research in neural network based approaches to text-to-speech, you'll find plenty of examples of realistic synthetic voices. However, due to the nature of the available training data (audiobooks), these models 1) are restricted to reading text in a narrative style and 2) aren't generally trained on speech of recognizable public figures. There was a paper a while back that was trained on TED talks and produced fairly convincing voices, some of which you may recognize (Bill gates, Stephen Wolfram and some others - see here https://sjvasquez.github.io/blog/melnet/ https://sjvasquez.github.io/blog/melnet/) Definitely not 100% convincing, but I wouldn't call it absolute trash.
egfx 6y ago
Check out the demos at https://www.descript.com/overdub https://www.descript.com/overdub I wouldn’t call the voices unconvincing. They used to have a better demo though where it would impersonate celebrities like Trump. It was very interesting to demo it.
- dbspin 6y ago
  Obviously it's subjective, but all of these demos have the 'watery' digital sound that's an immediate giveaway. By definition they're unconvincing.
- kenoph 6y ago
  I am using descript for a project of mine. You can definitely still hear that they are AI-generated. But I'm sure it's not gonna take long. Couple this with SSML and it's going to be quite convincing.
canadianfella 6y ago
> Audio deepfakes, when they finally arrive for real, will simply reduce the effort required for voice impersonation. They do not make possible new feats of impersonation that were never possible before. Just pulled this out of your ass or do you have a crystal ball?
bryanrasmussen 6y ago
automating sending out fake emails allowed any idiot to do it. there is a difference in scale and effect between every idiot in the world can fake as many impersonations of people that they want (and they can coordinate this impersonation with 100s of others who have a goal with this impersonation) and Rich Little calling up somebody's granny and claiming to be Jimmy Stewart.
sillysaurusx 6y ago
I made Dr Kleiner sing "I am the very model of a modern major general": https://www.youtube.com/watch?v=koU3L7WBz_s&ab_channel=ShawnPresser https://www.youtube.com/watch?v=koU3L7WBz_s&ab_channel=Shawn... It's pretty trash, but it's not completely horrible. It's also over a year old, so the SOTA is likely better today.
chrisin2d 6y ago
> They can still have an effect on society for sure, but I believe the concern is out of proportion with the effect. I believe the contrary: that high-quality, convincing audio (and video) deepfakes will be cataclysmic for society. Voice audio is one of the last digital artifacts that people will believe with their senses. When Donald Trump’s “grab them by the p—“ audioclip came out, he was at least forced to acknowledge it. Once high-quality audio deepfake tools become available, amateurs will be able to flood the web with fake audioclips of politicians saying things they never did, and truly unscrupulous politicians will be given technological cover for any damning audioclips of theirs that get leaked. Once we cannot trust our senses, then we will be only able to trust our institutions and then our tribes. But trust in institutions is already decaying rapidly, so in the end there only ideological tribes will be left.
SubiculumCode 6y ago
Do not underestimate the power of mediocre at scale. Synthetic is scalable.
gotrythis 6y ago
Did you check out the company linked in the article? https://www.respeecher.com/ https://www.respeecher.com/

chrisco255 6y ago

The effort will be reduced to typing sentences and selecting emotions from a dropdown menu. That opens up a whole new can of worms for a society that already has trust issues.

exporectomy 6y ago
Is that your own society? How would it exacerbate your trust issues? Surely making people doubt untrustworthy things is good?
- garg 6y ago
  Making people doubt untrustworthy things by educating them through a proper process is good. Exposing an unsuspecting/uneducated population to manipulated content designed to deceive/fool them for nefarious purposes is not an ideal process if the goal is to make the general public have better judgement. This is a global issue. For examples, please look into the societal concerns surrounding 'deep fakes', and 'fake news' for answers to your first two questions.
  - exporectomy 6y ago
    Pitting people in a social game of who can be fooled the easiest sounds like a more effective approach than reading a dry textbook at them. There's shame in being seen as gullible. This happens already in casual conversation and people already have opportunities to believe false information from all over the internet or in books or tabloid newspapers or told to them by their peers or parents. I don't believe the concerns about deep fakes and fake news. I think they're no worse than what we've always had. Those fears are exaggerated by arrogant hand-wringers who somehow assume their own beliefs are exempt from manipulation and it's only intellectually inferior "other" people who are susceptible. They're so fixated on the popular political ideas that they forget they should be on a crusade against religion which is the elephant in the room when it comes to people being misled.

BikiniPrince 6y ago

The Jordan Peterson df was quite good. I made a few sound bytes to send to some friends as a gag, but they were already aware it was in the wild. The compression is probably the worst here, but it was fun when it was up. https://www.notjordanpeterson.com/ https://www.notjordanpeterson.com/

spyder 6y ago

It was already successfully used to "Scam A CEO Out Of $243,000": https://www.forbes.com/sites/jessedamiani/2019/09/03/a-voice-deepfake-was-used-to-scam-a-ceo-out-of-243000/?sh=321ae41e2241 https://www.forbes.com/sites/jessedamiani/2019/09/03/a-voice... Probably it' easier because phone calls aren't high quality.

arnaudsm 6y ago
How did they prove the voice was an AI and not a human impersonator ?
- tyingq 6y ago
  They didn't, it was all speculation, mostly speculation from the person that received the call and the company's insurance firm.

ansk 6y ago

If you look at the latest research in neural network based approaches to text-to-speech, you'll find plenty of examples of realistic synthetic voices. However, due to the nature of the available training data (audiobooks), these models 1) are restricted to reading text in a narrative style and 2) aren't generally trained on speech of recognizable public figures. There was a paper a while back that was trained on TED talks and produced fairly convincing voices, some of which you may recognize (Bill gates, Stephen Wolfram and some others - see here https://sjvasquez.github.io/blog/melnet/ https://sjvasquez.github.io/blog/melnet/) Definitely not 100% convincing, but I wouldn't call it absolute trash.

egfx 6y ago

Check out the demos at https://www.descript.com/overdub https://www.descript.com/overdub I wouldn’t call the voices unconvincing. They used to have a better demo though where it would impersonate celebrities like Trump. It was very interesting to demo it.

dbspin 6y ago
Obviously it's subjective, but all of these demos have the 'watery' digital sound that's an immediate giveaway. By definition they're unconvincing.
kenoph 6y ago
I am using descript for a project of mine. You can definitely still hear that they are AI-generated. But I'm sure it's not gonna take long. Couple this with SSML and it's going to be quite convincing.

canadianfella 6y ago

> Audio deepfakes, when they finally arrive for real, will simply reduce the effort required for voice impersonation. They do not make possible new feats of impersonation that were never possible before. Just pulled this out of your ass or do you have a crystal ball?

bryanrasmussen 6y ago

automating sending out fake emails allowed any idiot to do it. there is a difference in scale and effect between every idiot in the world can fake as many impersonations of people that they want (and they can coordinate this impersonation with 100s of others who have a goal with this impersonation) and Rich Little calling up somebody's granny and claiming to be Jimmy Stewart.

sillysaurusx 6y ago

I made Dr Kleiner sing "I am the very model of a modern major general": https://www.youtube.com/watch?v=koU3L7WBz_s&ab_channel=ShawnPresser https://www.youtube.com/watch?v=koU3L7WBz_s&ab_channel=Shawn... It's pretty trash, but it's not completely horrible. It's also over a year old, so the SOTA is likely better today.

chrisin2d 6y ago

> They can still have an effect on society for sure, but I believe the concern is out of proportion with the effect. I believe the contrary: that high-quality, convincing audio (and video) deepfakes will be cataclysmic for society. Voice audio is one of the last digital artifacts that people will believe with their senses. When Donald Trump’s “grab them by the p—“ audioclip came out, he was at least forced to acknowledge it. Once high-quality audio deepfake tools become available, amateurs will be able to flood the web with fake audioclips of politicians saying things they never did, and truly unscrupulous politicians will be given technological cover for any damning audioclips of theirs that get leaked. Once we cannot trust our senses, then we will be only able to trust our institutions and then our tribes. But trust in institutions is already decaying rapidly, so in the end there only ideological tribes will be left.

SubiculumCode 6y ago

Do not underestimate the power of mediocre at scale. Synthetic is scalable.

gotrythis 6y ago

Did you check out the company linked in the article? https://www.respeecher.com/ https://www.respeecher.com/