4 ms·

Forgive my ignorance, but why are we surprised that voice messages aren't being spoofed more often? Doesn't this require a pretty darn decent dataset for traini

Forgive my ignorance, but why are we surprised that voice messages aren't being spoofed more often? Doesn't this require a pretty darn decent dataset for training? Unless they've got a ton of videos of themselves shared on a public social media profile, I don't know that this is going to be a thing.

savanaly 2y ago
The dataset you need to train in the first place is indeed huge, but I think the idea is once them model is trained, new "voices" can be acquired with much less data than was required to train it in the first place. Just like you can instruct ChatGPT to talk about topics never heard of on the internet and in a dialect you customize and invent on the spot and it can comply, despite not consuming an internet's worth of subject matter about it.
- nanidin 2y ago
  Soon the role of the Indian call centers will change from running the scam directly to making spam calls to trusted contacts of the intended mark to collect voice data for TTS model fine tuning.
Zambyte 2y ago
> Unless they've got a ton of videos of themselves shared on a public social media profile, I don't know that this is going to be a thing. That sounds like a great recipe for spear phishing public figures.
dbspin 2y ago
I'm not a data science / AI person at all, but AFAIK while initial model training is enormously processor intensive, customising a trained model is not. This has been my experience playing with custom training stable diffusion on very low memory home GPU. I'd assume it's true for voice generators like Tortoise [1] also. Moreover, while Eleven Labs isn't in my opinion as good, they let you custom train voices for an incredibly low cost and with a tiny amount of sample data [3]. For perfect audio spoofing, lots of audio would be needed. Bear in mind there are literally millions of podcasts available [2], and billions of youtube videos. Should be trivial to grab biographic data and voice samples from a subset of them. [1] https://github.com/neonbjb/tortoise-tts https://github.com/neonbjb/tortoise-tts [2] https://whatsthebigdata.com/podcasts-statistics/ https://whatsthebigdata.com/podcasts-statistics/ [3] https://elevenlabs.io/ https://elevenlabs.io/
nanomonkey 2y ago
I've had this nagging worry that my voice will be harvested during a random call acting like a sales person or survey. Besides getting money from family members, I really can't think of a scenario where my voice would allow a scammer to get into one of my accounts, but I'm sure it would make social engineering such an attach that much easier.
- Acrobatic_Road 2y ago
  Earlier I received a strange call from someone. The voice asked if I could hear him. I said "I can hear you, now who is speaking". Then they hung up. I'm pretty sure they were hoping that I would respond with the word "yes". That is a very dangerous word in the wrong hands!
- rightbyte 2y ago
  There was some "technician" calling my wife just today about cleaning the house's vents and she handed me the phone without much context. He claimed to have personally cleaned them for the former owner, and I told him to send me a mail or letter, and gave him my email address. Afterwards I realized it might have been some sort of scam attempt. He seemed to be in such a rush to get it done tomorrow and book a time now now now. And he sent no mail. I guess the scam would have been to prepay some sort of fee or the whole cleaning. But I've got this feeling he was fishing for something more. Dunno if they have the know-how to actually use my voice to scam e.g. my mother.

savanaly 2y ago

The dataset you need to train in the first place is indeed huge, but I think the idea is once them model is trained, new "voices" can be acquired with much less data than was required to train it in the first place. Just like you can instruct ChatGPT to talk about topics never heard of on the internet and in a dialect you customize and invent on the spot and it can comply, despite not consuming an internet's worth of subject matter about it.

nanidin 2y ago
Soon the role of the Indian call centers will change from running the scam directly to making spam calls to trusted contacts of the intended mark to collect voice data for TTS model fine tuning.

Zambyte 2y ago

> Unless they've got a ton of videos of themselves shared on a public social media profile, I don't know that this is going to be a thing. That sounds like a great recipe for spear phishing public figures.

dbspin 2y ago

I'm not a data science / AI person at all, but AFAIK while initial model training is enormously processor intensive, customising a trained model is not. This has been my experience playing with custom training stable diffusion on very low memory home GPU. I'd assume it's true for voice generators like Tortoise [1] also. Moreover, while Eleven Labs isn't in my opinion as good, they let you custom train voices for an incredibly low cost and with a tiny amount of sample data [3]. For perfect audio spoofing, lots of audio would be needed. Bear in mind there are literally millions of podcasts available [2], and billions of youtube videos. Should be trivial to grab biographic data and voice samples from a subset of them. [1] https://github.com/neonbjb/tortoise-tts https://github.com/neonbjb/tortoise-tts [2] https://whatsthebigdata.com/podcasts-statistics/ https://whatsthebigdata.com/podcasts-statistics/ [3] https://elevenlabs.io/ https://elevenlabs.io/

nanomonkey 2y ago

I've had this nagging worry that my voice will be harvested during a random call acting like a sales person or survey. Besides getting money from family members, I really can't think of a scenario where my voice would allow a scammer to get into one of my accounts, but I'm sure it would make social engineering such an attach that much easier.

Acrobatic_Road 2y ago
Earlier I received a strange call from someone. The voice asked if I could hear him. I said "I can hear you, now who is speaking". Then they hung up. I'm pretty sure they were hoping that I would respond with the word "yes". That is a very dangerous word in the wrong hands!
rightbyte 2y ago
There was some "technician" calling my wife just today about cleaning the house's vents and she handed me the phone without much context. He claimed to have personally cleaned them for the former owner, and I told him to send me a mail or letter, and gave him my email address. Afterwards I realized it might have been some sort of scam attempt. He seemed to be in such a rush to get it done tomorrow and book a time now now now. And he sent no mail. I guess the scam would have been to prepay some sort of fee or the whole cleaning. But I've got this feeling he was fishing for something more. Dunno if they have the know-how to actually use my voice to scam e.g. my mother.