3 ms·

I doubt a single voice recording could generate speech with enough emotional depth that I'd be fooled by a fake call from my better half. Do people really have

by lynx23 3y ago

I doubt a single voice recording could generate speech with enough emotional depth that I'd be fooled by a fake call from my better half. Do people really have such a shallow connections to their loved ones that they wouldn't instinctively feel that something isn't right? C'mon, we humans operate with a lot of extra clues. Its not like we just STT and "read" the resulting output.

idank 3y ago
It doesn't have to work 100% of the time to be a successful scam.
godelski 3y ago
You're right to be suspicious but wrong in your interpretation of reality. The issue is that there's a big difference in how you'll interpret things in different environments and stress levels. Under pressure a lot of your normal checks will go out the window. This is why conmen often put you under time pressures, because they're stressing your system. Under normal circumstances, you're prefectly right, but under duress weird things seem normal. I mean duress is abnormal to begin with. Also enshitification is an important aspect here. We readily overlook things like sharp cuts in video, degraded signals in video and voice, distortion, etc. Because that stuff happens normally. Post hoc it is often easy to see scams and think how dumb someone must have been to fall for something so obvious, but that's actually a tool of the scammers themselves. They love that because the more embarrassed you are, the less likely you will report it due to shame. Be careful with over confidence, because that's what they target.
- lynx23 3y ago
  As someone who works in IT for roughly 30 years, and who never fell for any kind of phishing, I beg to differ. I think my interpretation of reality, as you put it, isn't as bad as you might believe. Also, all the phishing-victims I know have something in common, a certain lack of rigour. You seem to claim that everyone can fall for phishing. I beg to differ, its a certain class of not very thorough (l)users. Your comment regarding enshitifcation has some truth to it I didn't really think about yet, thanks. However, my original comment still stands as it is. I challenge anyone to create a voice scam that fools me.
nickfromseattle 3y ago
> that I'd be fooled by a fake call Can you say the same thing about all of your friends and loved ones? You don't have to personally fall for the scam for it to negatively impact you. Maybe your mom falls for it, and now she can't pay her mortgage. Fake call scams already generate billions of dollars in stolen money. Telecoms let spam phone calls run rampant for a decade before being forced to fix it. AI video/audio deepfakes at scale could 10x this industry.
beej71 3y ago
There was an interesting paper about a year ago that talked about synthesizing voices with minimal training data, iirc on the order of 30 seconds. The results were imperfect but wholly impressive given the input. "Think of what will be possible just two more papers down the line."
Ferret7446 3y ago
From experience, it is the people who are confident they won't be tricked that are most vulnerable to being tricked.
Turing_Machine 3y ago
The spouse of a former boss of mine regularly mistook me for him, if I happened to answer his phone while he was at lunch or in the toilet or something (this was long before people had personal cell phones). Once she even said "Hi! This is your little snuggle bunny!". Mutual embarrassment ensued.

by lynx23 3y ago

idank 3y ago

It doesn't have to work 100% of the time to be a successful scam.

godelski 3y ago

You're right to be suspicious but wrong in your interpretation of reality. The issue is that there's a big difference in how you'll interpret things in different environments and stress levels. Under pressure a lot of your normal checks will go out the window. This is why conmen often put you under time pressures, because they're stressing your system. Under normal circumstances, you're prefectly right, but under duress weird things seem normal. I mean duress is abnormal to begin with. Also enshitification is an important aspect here. We readily overlook things like sharp cuts in video, degraded signals in video and voice, distortion, etc. Because that stuff happens normally. Post hoc it is often easy to see scams and think how dumb someone must have been to fall for something so obvious, but that's actually a tool of the scammers themselves. They love that because the more embarrassed you are, the less likely you will report it due to shame. Be careful with over confidence, because that's what they target.

lynx23 3y ago
As someone who works in IT for roughly 30 years, and who never fell for any kind of phishing, I beg to differ. I think my interpretation of reality, as you put it, isn't as bad as you might believe. Also, all the phishing-victims I know have something in common, a certain lack of rigour. You seem to claim that everyone can fall for phishing. I beg to differ, its a certain class of not very thorough (l)users. Your comment regarding enshitifcation has some truth to it I didn't really think about yet, thanks. However, my original comment still stands as it is. I challenge anyone to create a voice scam that fools me.

nickfromseattle 3y ago

> that I'd be fooled by a fake call Can you say the same thing about all of your friends and loved ones? You don't have to personally fall for the scam for it to negatively impact you. Maybe your mom falls for it, and now she can't pay her mortgage. Fake call scams already generate billions of dollars in stolen money. Telecoms let spam phone calls run rampant for a decade before being forced to fix it. AI video/audio deepfakes at scale could 10x this industry.

beej71 3y ago

There was an interesting paper about a year ago that talked about synthesizing voices with minimal training data, iirc on the order of 30 seconds. The results were imperfect but wholly impressive given the input. "Think of what will be possible just two more papers down the line."

Ferret7446 3y ago

From experience, it is the people who are confident they won't be tricked that are most vulnerable to being tricked.

Turing_Machine 3y ago

The spouse of a former boss of mine regularly mistook me for him, if I happened to answer his phone while he was at lunch or in the toilet or something (this was long before people had personal cell phones). Once she even said "Hi! This is your little snuggle bunny!". Mutual embarrassment ensued.