6 ms·

Those fake AI voices are perfect tools for phishing campaigns. I wonder if ChatGPT combined with a voice generator can mount successful phishing campaigns. It

Those fake AI voices are perfect tools for phishing campaigns. I wonder if ChatGPT combined with a voice generator can mount successful phishing campaigns. It looks like Cyberpunk future we never asked is already here.

i_love_cookies 4y ago
[dead]
Teever 4y ago
Imagine an AI voice of Trump that phones lonely old people to thank them for coming to his rally. After 20 minutes of small talk he asks them to donate to his campaign via bitcoin...
- smitty1e 4y ago
  Lonely old people sophisticated enough to do bitcoin? Color me skeptical.
  - smithcoin 4y ago
    They get conned into sending money from Bitcoin ATMs[0] [0] https://www.atmmarketplace.com/news/elderly-michigan-couple-loses-350k-in-bitcoin-atm-scam/ https://www.atmmarketplace.com/news/elderly-michigan-couple-...
  - hotpotamus 4y ago
    I'd imagine an AI could quite patiently walk them through how to set it up and link their bank accounts. Honestly, it could be really helpful for tech support.
  - andreareina 4y ago
    I know at least a couple. Doesn't take much sophistication with all the hype the last few years and different apps making it easy. It's essentially as easy as using any other mobile payment tech e.g. mpesa or alipay
- shakna 4y ago
  Even simpler. Have the voice give them a short onboarding call, but then have it regularly call them again using reverse charging, and then chat with them. Convince them that they've formed a friendship.
13of40 4y ago
I'm sure it's going to get exponentially better, but right now ChatGPT sounds like one very vanilla guy. The other day I asked it to write me a conversation between David Foster Wallace and Ernest Hemingway, and it came out something like (paraphrasing): HEMINGWAY: I notice you write about political issues and philosophy. WALLACE: Thank you, I noticed you write about raw experience and manliness. HEMINGWAY: Well, whatever our differences I feel like we can get along and work together. We will soon be fast friends.
- chatmasta 4y ago
  I tried to get it to intentionally respond with grammatically incorrect sentences and it had a lot of trouble. I basically couldn't get it to work.
- Nition 4y ago
  Has ChatGPT actually read any books (apart from snippets on the Internet), or has that been avoided for copyright reasons? I wonder how long until someone pirates every ebook on Z-Library or SciHub and trains a language model off that. Imagine a language model that's read every book and every scientific paper.
  - techdragon 4y ago
    I wonder if that would even generate good outcomes given how much of the data in some disciplines is conflicting, and if you don’t have a good way to get it to ignore bad subsets of that data it could be more likely to hallucinate convincing scientific sounding explanations based on a preponderance of bad poorly researched evidence. There’s a lot of papers that discredit older papers or work or entire subfields of study and the current large language models would just all all of it and go “these are words” with no analytical reasoning applied. These are predictive mechanisms not analytical ones and until we get a better handle on how we can make them “stop and think”… I’m just not sure how much benefit each larger dataset will add beyond “sounds more human” while it’s core problems of “still hallucinates” and “can’t do basic reasoning” remain. Also one question I’ve got on this front is how much duplicate data is processed out of the input sets? Using LibGen as an example there’s a lot of books that get reprinted and uploaded multiple times in different formats… does having these remain in the data provide a “valuable bias” or is it something that needs removing? Do the people building large language models pre-process their data to de-duplicate out these kinds of things from their current data sources?
svachalek 4y ago
It's kind of amazing we're not seeing more of this already. It's a trivial application of the tech. We've already achieved a culture where the more batty something is the more likely people are to believe it, and now we have an infinite, personalized quackery generator. Without a massive cultural revolution putting the brakes on this, which I can't imagine coming from anywhere, I fear we're barreling into a future where nothing is true anymore, a hall of mirrors where the capacity to fake anything at all far surpasses anyone's ability to make sense of it.
- hosh 4y ago
  MacLuhan and others have already talked about this, from way back in the radio and TV generation. I don't say this to mean, "oh, and they were wrong because everything turned out OK" If anything, we have had generations that grew up with this dysfunction in a normalized way, and it took another advanced in tech to really see it again. (To take it further back into history, I think I now get what Cervantes was trying to say with the image of Don Quxiote tilting at windmills. It seemed so absurd, amusing, and quaint when I read it as a teen).
  - pixl97 4y ago
    If anything TV and radio presented a larger shared reality, even if it was one filled with falsehoods. What is culture but a shared set of values? What we are growing now is a system that can give each and every one of us our own reality, and if it succeeds, what of society?
    - hosh 4y ago
      There are some fascinating areas to explore those questions from the perspective of shamans, mystics, and yogis.
anaganisk 4y ago
TikTok lady AI voice starts talking, hangs up phone
mmcwilliams 4y ago
Would it really be wise to run a criminal operation while piping all your requests through Microsoft servers, considering the history of the company collaborating with law enforcement?
- DethNinja 4y ago
  It is trivial to get access to some else’s router & credit cards in 2023. Years and years of bad software practices, corporate cost cutting measures resulted in a planet scale cybersecurity mess. Current hacking scene is more similar to 80s than early 2000s, it is extremely hard to catch advanced actors.
kevin_thibedeau 4y ago
We can just make the AI phishing bots talk to a Lenny 2.0 trained to tie them up in endless conversations. AI interacting with AI will be the next economic boom fomenting a wave of new wealth creation.
ethicalsmacker 4y ago
"Hey mom, it's X. Sorry to leave this voice mail, I got in a bit of a car accident and lost my phone. I need to get back home, could you send $$$$ to ####?"
- wilsonnb3 4y ago
  Just have to establish real life security questions. They did this in one of the Harry Potter books (the sixth one?) since it is easy for them to impersonate other people. If you can’t tell me my favorite flavor of jam then you aren’t getting my money.
  - ccvannorman 4y ago
    "Your foster parents are dead"
  - tjfl 4y ago
    What is your favorite flavor of jam?
  - th13row 4y ago
    This wouldn't work for me. I always pretend to like something and then change my mind the next day to piss people off. Maybe just a call to ensure it's legit. Vocal deepfakes will never mimic voices and ways of talking in real time, it will always sound "off". It doesn't make it any less dystopian, though.
    - ethicalsmacker 4y ago
      Are you my wife?
  - pixl97 4y ago
    Eh, if they hack any adtech provider they may be able glean that you like loganberry.
stubish 4y ago
You can already fool real world voice authentication systems. No need to phish for credentials when the voice is the credential: https://www.theguardian.com/technology/2023/mar/16/voice-system-used-to-verify-identity-by-centrelink-can-be-fooled-by-ai https://www.theguardian.com/technology/2023/mar/16/voice-sys...

i_love_cookies 4y ago

[dead]

Teever 4y ago

Imagine an AI voice of Trump that phones lonely old people to thank them for coming to his rally. After 20 minutes of small talk he asks them to donate to his campaign via bitcoin...

smitty1e 4y ago
Lonely old people sophisticated enough to do bitcoin? Color me skeptical.
- smithcoin 4y ago
  They get conned into sending money from Bitcoin ATMs[0] [0] https://www.atmmarketplace.com/news/elderly-michigan-couple-loses-350k-in-bitcoin-atm-scam/ https://www.atmmarketplace.com/news/elderly-michigan-couple-...
- hotpotamus 4y ago
  I'd imagine an AI could quite patiently walk them through how to set it up and link their bank accounts. Honestly, it could be really helpful for tech support.
- andreareina 4y ago
  I know at least a couple. Doesn't take much sophistication with all the hype the last few years and different apps making it easy. It's essentially as easy as using any other mobile payment tech e.g. mpesa or alipay
shakna 4y ago
Even simpler. Have the voice give them a short onboarding call, but then have it regularly call them again using reverse charging, and then chat with them. Convince them that they've formed a friendship.

13of40 4y ago

I'm sure it's going to get exponentially better, but right now ChatGPT sounds like one very vanilla guy. The other day I asked it to write me a conversation between David Foster Wallace and Ernest Hemingway, and it came out something like (paraphrasing): HEMINGWAY: I notice you write about political issues and philosophy. WALLACE: Thank you, I noticed you write about raw experience and manliness. HEMINGWAY: Well, whatever our differences I feel like we can get along and work together. We will soon be fast friends.

chatmasta 4y ago
I tried to get it to intentionally respond with grammatically incorrect sentences and it had a lot of trouble. I basically couldn't get it to work.
Nition 4y ago
Has ChatGPT actually read any books (apart from snippets on the Internet), or has that been avoided for copyright reasons? I wonder how long until someone pirates every ebook on Z-Library or SciHub and trains a language model off that. Imagine a language model that's read every book and every scientific paper.
- techdragon 4y ago
  I wonder if that would even generate good outcomes given how much of the data in some disciplines is conflicting, and if you don’t have a good way to get it to ignore bad subsets of that data it could be more likely to hallucinate convincing scientific sounding explanations based on a preponderance of bad poorly researched evidence. There’s a lot of papers that discredit older papers or work or entire subfields of study and the current large language models would just all all of it and go “these are words” with no analytical reasoning applied. These are predictive mechanisms not analytical ones and until we get a better handle on how we can make them “stop and think”… I’m just not sure how much benefit each larger dataset will add beyond “sounds more human” while it’s core problems of “still hallucinates” and “can’t do basic reasoning” remain. Also one question I’ve got on this front is how much duplicate data is processed out of the input sets? Using LibGen as an example there’s a lot of books that get reprinted and uploaded multiple times in different formats… does having these remain in the data provide a “valuable bias” or is it something that needs removing? Do the people building large language models pre-process their data to de-duplicate out these kinds of things from their current data sources?

svachalek 4y ago

It's kind of amazing we're not seeing more of this already. It's a trivial application of the tech. We've already achieved a culture where the more batty something is the more likely people are to believe it, and now we have an infinite, personalized quackery generator. Without a massive cultural revolution putting the brakes on this, which I can't imagine coming from anywhere, I fear we're barreling into a future where nothing is true anymore, a hall of mirrors where the capacity to fake anything at all far surpasses anyone's ability to make sense of it.

hosh 4y ago
MacLuhan and others have already talked about this, from way back in the radio and TV generation. I don't say this to mean, "oh, and they were wrong because everything turned out OK" If anything, we have had generations that grew up with this dysfunction in a normalized way, and it took another advanced in tech to really see it again. (To take it further back into history, I think I now get what Cervantes was trying to say with the image of Don Quxiote tilting at windmills. It seemed so absurd, amusing, and quaint when I read it as a teen).
- pixl97 4y ago
  If anything TV and radio presented a larger shared reality, even if it was one filled with falsehoods. What is culture but a shared set of values? What we are growing now is a system that can give each and every one of us our own reality, and if it succeeds, what of society?
  - hosh 4y ago
    There are some fascinating areas to explore those questions from the perspective of shamans, mystics, and yogis.

anaganisk 4y ago

TikTok lady AI voice starts talking, hangs up phone

mmcwilliams 4y ago

Would it really be wise to run a criminal operation while piping all your requests through Microsoft servers, considering the history of the company collaborating with law enforcement?

DethNinja 4y ago
It is trivial to get access to some else’s router & credit cards in 2023. Years and years of bad software practices, corporate cost cutting measures resulted in a planet scale cybersecurity mess. Current hacking scene is more similar to 80s than early 2000s, it is extremely hard to catch advanced actors.

kevin_thibedeau 4y ago

We can just make the AI phishing bots talk to a Lenny 2.0 trained to tie them up in endless conversations. AI interacting with AI will be the next economic boom fomenting a wave of new wealth creation.

ethicalsmacker 4y ago

"Hey mom, it's X. Sorry to leave this voice mail, I got in a bit of a car accident and lost my phone. I need to get back home, could you send $$$$ to ####?"

wilsonnb3 4y ago
Just have to establish real life security questions. They did this in one of the Harry Potter books (the sixth one?) since it is easy for them to impersonate other people. If you can’t tell me my favorite flavor of jam then you aren’t getting my money.
- ccvannorman 4y ago
  "Your foster parents are dead"
- tjfl 4y ago
  What is your favorite flavor of jam?
- th13row 4y ago
  This wouldn't work for me. I always pretend to like something and then change my mind the next day to piss people off. Maybe just a call to ensure it's legit. Vocal deepfakes will never mimic voices and ways of talking in real time, it will always sound "off". It doesn't make it any less dystopian, though.
  - ethicalsmacker 4y ago
    Are you my wife?
- pixl97 4y ago
  Eh, if they hack any adtech provider they may be able glean that you like loganberry.

stubish 4y ago

You can already fool real world voice authentication systems. No need to phish for credentials when the voice is the credential: https://www.theguardian.com/technology/2023/mar/16/voice-system-used-to-verify-identity-by-centrelink-can-be-fooled-by-ai https://www.theguardian.com/technology/2023/mar/16/voice-sys...