6 ms·

Welcome to the new era of fakes and scams beyond our wildest imagination !

danielbln 3y ago
Elevenlabs has been around for a while now. Genie has been out of the bottle for a bit, and the sooner the notion that anything digital can be easily faked seeps into the wider consciousness the better. Trust nothing.
- ethanbond 3y ago
  It can both be true that people need to adapt/“trust nothing,” and that this is bad.
- ignu 3y ago
  I've seen some prank calls (a YouTuber cloned Tucker Carlson's voice and called Alex Jones) but he just had a sound bank with a few pre-generated lines and it fell apart pretty quickly. At least for now there's too much lag to do a real time conversation with a cloned voice. Speech to Text > LLM Response > Generate Audio If that time can shrink to subsecond, I think there'll be madness. (Specifically thinking of romance scammers)
  - whywhywhywhy 3y ago
    You don't need an LLM Response
  - shinycode 3y ago
    Awful, bots on their own having real conversations with people with the voice of a loved one. Scamming on steroids
  - ben_w 3y ago
    At last summer's WeAreDevelopers World Congress in Berlin, one of the talks I went to was by someone who did this with their own voice, to better respond to (really long?) WhatsApp messages they kept getting. It worked a bit too well, as it could parse the sound file and generate a complete response faster than real-time, leading people to ask if he'd actually listened to the messages they sent him. Also they had trouble believing him when he told them how he'd done it.
- smt88 3y ago
  > * the notion that anything digital can be easily faked seeps into the wider consciousness the better. Trust nothing.* This is a society-destroying idea. Most of us, especially younger people, only know how to vote, where there are wars, or even what our parents are doing by using digital media. If digital media becomes untrustworthy, everyone will live in a warped and fragile alternate reality that no one can agree on.
  - diggan 3y ago
    > Trust nothing > This is a society-destroying idea. Believe it or not, this is how much of the population saw The Internet when it first came close to being mainstream. Everyone and their mother said "Don't believe anything you read on the cybernet", which ended up ironic as everyone and their mother ended up being the ones to believe anything on the cybernet anyways. > everyone will live in a warped and fragile alternate reality that no one can agree on. How is this any different from today? The various corners of the internet (which is mostly divided by languages: English, Russian, Spanish, Chinese and Portuguese) already have these vastly different realities and ground-truths. I'm sure we could survive another Internet-Winter where people trust everything a bit less than today.
    - smt88 3y ago
      It's vastly different than today because today (or at least a few years ago), I could trust videos and voices delivered digitally. I can't do that anymore.
      - thomashop 3y ago
        How long has society had voice and video delivered digitally? We managed to survive fine before we had it. If it now becomes impossible to trust a voice received through the internet without being connected to a verified telephone number I don't know how that can be classified as society-changing.
      - ilikehurdles 3y ago
        Technology and society will adapt, just as we adapted encryption to verify credentials and secure banking data online, we'll end up with a validation signal for video and audio.
treprinum 3y ago
VALL-E is on Github for over a year already...
underlines 3y ago
This aera is barely new. Look at how old some of the projects are: https://github.com/underlines/awesome-ml/blob/master/audio-ai.md#voice-conversion https://github.com/underlines/awesome-ml/blob/master/audio-a... The thing that changes is the complexity to run it. I was training my wife's voice and my voice for fun and needed 15min of audio and trained on my 3080 for 40 minutes. Now it's 2 Minutes.
- thfuran 3y ago
  Yes, and the more accessible it is, the more widespread it will be.
  - underlines 3y ago
    True. But the better way forward IMHO is to give access to technology in equal ways, instead of keeping it in the hands of a few who promise they are the good ones. Because then an chilling effect can happen, and the playing field is level.
ponector 3y ago
Maybe this will teach people to rise up awareness and take personal security serious. Like not to trust anyone who is calling, especially from legacy line. Phone number and voice could be easily cloned.
- dijksterhuis 3y ago
  The problem is that it's not just personal security, and it requires significant expertise / research to identify. https://www.bbc.co.uk/news/world-africa-66987869 https://www.bbc.co.uk/news/world-africa-66987869
deleted 3y ago
[deleted]

Elevenlabs has been around for a while now. Genie has been out of the bottle for a bit, and the sooner the notion that anything digital can be easily faked seeps into the wider consciousness the better. Trust nothing.

ethanbond 3y ago
It can both be true that people need to adapt/“trust nothing,” and that this is bad.
ignu 3y ago
I've seen some prank calls (a YouTuber cloned Tucker Carlson's voice and called Alex Jones) but he just had a sound bank with a few pre-generated lines and it fell apart pretty quickly. At least for now there's too much lag to do a real time conversation with a cloned voice. Speech to Text > LLM Response > Generate Audio If that time can shrink to subsecond, I think there'll be madness. (Specifically thinking of romance scammers)
- whywhywhywhy 3y ago
  You don't need an LLM Response
- shinycode 3y ago
  Awful, bots on their own having real conversations with people with the voice of a loved one. Scamming on steroids
- ben_w 3y ago
  At last summer's WeAreDevelopers World Congress in Berlin, one of the talks I went to was by someone who did this with their own voice, to better respond to (really long?) WhatsApp messages they kept getting. It worked a bit too well, as it could parse the sound file and generate a complete response faster than real-time, leading people to ask if he'd actually listened to the messages they sent him. Also they had trouble believing him when he told them how he'd done it.
smt88 3y ago
> * the notion that anything digital can be easily faked seeps into the wider consciousness the better. Trust nothing.* This is a society-destroying idea. Most of us, especially younger people, only know how to vote, where there are wars, or even what our parents are doing by using digital media. If digital media becomes untrustworthy, everyone will live in a warped and fragile alternate reality that no one can agree on.
- diggan 3y ago
  > Trust nothing > This is a society-destroying idea. Believe it or not, this is how much of the population saw The Internet when it first came close to being mainstream. Everyone and their mother said "Don't believe anything you read on the cybernet", which ended up ironic as everyone and their mother ended up being the ones to believe anything on the cybernet anyways. > everyone will live in a warped and fragile alternate reality that no one can agree on. How is this any different from today? The various corners of the internet (which is mostly divided by languages: English, Russian, Spanish, Chinese and Portuguese) already have these vastly different realities and ground-truths. I'm sure we could survive another Internet-Winter where people trust everything a bit less than today.
  - smt88 3y ago
    It's vastly different than today because today (or at least a few years ago), I could trust videos and voices delivered digitally. I can't do that anymore.
    - thomashop 3y ago
      How long has society had voice and video delivered digitally? We managed to survive fine before we had it. If it now becomes impossible to trust a voice received through the internet without being connected to a verified telephone number I don't know how that can be classified as society-changing.
    - ilikehurdles 3y ago
      Technology and society will adapt, just as we adapted encryption to verify credentials and secure banking data online, we'll end up with a validation signal for video and audio.

treprinum 3y ago

VALL-E is on Github for over a year already...

underlines 3y ago

This aera is barely new. Look at how old some of the projects are: https://github.com/underlines/awesome-ml/blob/master/audio-ai.md#voice-conversion https://github.com/underlines/awesome-ml/blob/master/audio-a... The thing that changes is the complexity to run it. I was training my wife's voice and my voice for fun and needed 15min of audio and trained on my 3080 for 40 minutes. Now it's 2 Minutes.

thfuran 3y ago
Yes, and the more accessible it is, the more widespread it will be.
- underlines 3y ago
  True. But the better way forward IMHO is to give access to technology in equal ways, instead of keeping it in the hands of a few who promise they are the good ones. Because then an chilling effect can happen, and the playing field is level.

ponector 3y ago

Maybe this will teach people to rise up awareness and take personal security serious. Like not to trust anyone who is calling, especially from legacy line. Phone number and voice could be easily cloned.

dijksterhuis 3y ago
The problem is that it's not just personal security, and it requires significant expertise / research to identify. https://www.bbc.co.uk/news/world-africa-66987869 https://www.bbc.co.uk/news/world-africa-66987869

deleted 3y ago

[deleted]