12 ms·

The golden age of scammers: AI-powered phishing

197 points by pwmtr 2y ago

darefalcon 2y ago
It’s actually worse than that - AI powered phishing sites will also copy your device profile and mouse, gesture and keyboard signature and use this to get past common anti-fraud techniques like device fingerprinting and behavioural biometrics.
- advael 2y ago
  I mean, the mere existence of said biometrics imply that they're inferrable and thus bad security, like basically all biometrics
  - sroussey 2y ago
    And yet, used extensively
    - advael 2y ago
      I mean SSNs are the worst possible authentication mechanism and yet we still have to freak out every time they're leaked. Security practices are so utterly backwards everywhere that it's quite apparent no one powerful is incentivised to care even a little bit
      - jimmaswell 2y ago
        what's the practical alternative?
        immibis 2y ago
        Scanning your digital ID card would be a start, but oHnO that's cOmmUniSM!
        bryanrasmussen 2y ago
        pickpocket's paradise!
        vincnetas 2y ago
        But it's not so scalable as online connected DB.
        reginald78 2y ago
        Wouldn't this just result in my digitalid getting lost in the next equifax breech?
        immibis 2y ago
        Did your credit card get lost in the last one?
        reginald78 2y ago
        How would I know one way or the other if it had? I don't have the same number anymore.
        advael 2y ago
        Not pretending a GUID constitutes a security measure in the first place? It's just not the right tool for the job in any sense
        mnau 2y ago
        Digital id card for every citizen? My id card can prove identity or sign a document and could for ~10 years. Estonia had it for over 20 years. Just give it to everyone. Today, it can likely be embedded in a cell phone instead of separate physical card.
        Kiro 2y ago
        What do you mean? I don't know of any country except the US where SSN is used for authentication. In my country SSN is public information.
    - TechDebtDevin 2y ago
      When I call my bank they verify with my voice. There are further verification for meaningful actions but its still kind of crazy to be using "My Voice is my Password" this day in age.
      - AlotOfReading 2y ago
        Better than my bank, which tries to ask questions about my life from some lookup service that has incorrect information.
      - reginald78 2y ago
        Especially since "My Voice is my Passport" was defeated in that movie with a tape recorder and technology at the time. It was never a good idea and even the movie didn't seem to think so. Yet my bank just turned this on for me as well in 2024. Now I have to figure out how to disable it...and will it really be disabled?
  - bryanrasmussen 2y ago
    basically everything that retains the same structure between two occurrences can be inferred. Only randomness cannot be inferred. But true randomness is not useful for determining if you are who you say you are.
    - advael 2y ago
      Even a password can be changed if there's a compromise. Biometrics are bad because they can be imitated, but not changed. A breach is permanent
      - bryanrasmussen 2y ago
        good point, but that was left out of the earlier statement about inference. I suppose I should have inferred it however.
        advael 2y ago
        Yea, my bad I guess. I tend to think people mostly get that biometrics are, well, mostly immutable and that not being able to switch them up in response to a suspected breach is a huge inherent weakness. So the only defense I really get of them from anyone is that the effort for the user is minimized while the effort for the attacker is still fairly high. The problem with that is why I mention inferrability: The existence of a computer system that can authenticate via a biometric implies the existence of one that can capture and spoof it, and we don't have any reason to believe this involves, say, more of a cost disparity than cracking a password, let alone anything approaching a strong one-way function. If your face is your key, do you start hiding your face on the street so no one can steal it? Same thing for behaviorals
  - chefandy 2y ago
    I think this is one of those "the only thing that's worse is everything else" situations. Surely there are solutions, but I doubt there are solutions banks and payment processors would be interested in paying for, and at least the US government isn't particularly interested in compelling banks to do anything expensive.
- coffeebeqn 2y ago
  What does AI have to do with capturing inputs ?
  - orbital-decay 2y ago
    Not just capturing, but training on captured inputs to replicate the fingerprint.
    - snowfield 2y ago
      laughable
      - ted_bunny 2y ago
        How about an actual rebuttal instead of smug dismissal? Let us in on the joke, genius.
        snowfield 2y ago
        If you want to replicate a fingerprint. Just get one image of one fingerprint. If you want to create an average looking fingerprint, use an AI
WesternWind 2y ago
Hey, just going to say what I've been telling folks IRL, if you are reading this, and your parents and family members aren't tech savvy, you need to set them up with two factor authentication now. Because you know how to do that, and it's so much easier than helping them when they get hacked.
- ethbr1 2y ago
  As evidence of the current state of play: Friend receives an email from ISP, asking her to contact them. She searches, comes across a "customer service number" on a legit looking page, calls them up. (Whoever she called) plays out a 30 minute charade about how she's been flagged by IRS for illegal activity and is about to have her business accounts frozen, including multiple phone transfers to "another party" (played by different people) to boost authenticity. And during this whole time, they not once asked her for any "red flag" information (e.g. account #, SSN). Instead, it seemed to be a shell game of extracting limited information (last 3 of your account #?), then having "unrelated" parties parrot that back as proof of their "working for the government." I expect it would have eventually escalated into an actionable ask, but they were definitely playing the intermediate-term game. If not for the utter moral black hole of the endeavor, I'd be kind of impressed.
  - __MatrixMan__ 2y ago
    I shouldn't, but sometimes I play along just to see what the scam looks like. Last time I did this, it took three days of texting my new friend before it was finally clear that what she really wanted more than anything was to teach me to trade cryptocurrency. Once, I thought I had her, because she spelled D&D like: D&D, but she played it off real cool and just explained that her English isn't that great so she used translation software. In retrospect I think that all of her probing questions about my Svirfneblin cleric were because she later intended call him up and teach him to trade cryptocurrency. I like to think he's in some scammer's database now, causing confusion. He'd like that too. Once I understood what she was after, I explained that my problem with cryptocurrency was that it resembled money too closely and really what I'd like to do with blockchains is to do away with money in favor of something entirely different. Her training dataset had not prepared her for this conversation, so it was quite clear when her human handler took over. They were very rude, unlike their AI pet, and tried to bully me into sharing other people's contact info, which is when I lost interest.
29athrowaway 2y ago
On YouTube, I saw a deepfake of Elon Musk asking people to scan a QR code and buy crypto.
- pixl97 2y ago
  Yep these are super common, especially on days spacex is launching starship. They only live broadcast it on X so it allows these scammers to step in and attempt to trick people.
  - h4kor 2y ago
    I watched the last starship launch on a scam YouTube stream. It was 30 minutes delayed, which a thought was because Musk wanted to promote Twitter. They said multiple times that Elon will announce something big after the launch. Directly after lift off it cut to Elon holding a speech and I only noticed this been a scam channel when he talked about the QR code and crypto.
    - yamumsahoe 2y ago
      same. stopped watching youtube live after this, realize that every live video could be a scam or distorted or modified or outdated in any way. weird.
- 6510 2y ago
  When I typed my phone number in the box on "Musk" "investment" website my landline rang instantly after entering the last digit. It was definitely an onkeydown event. A friendly fast talking man in an extremely busy sounding (fake) call center asked me if I was $name_I_put_in_the_form. The voices in the background were people further down the sign up process. I said yes, then asked how he got my number. He said I just filled out the form on the website. Then the form was replaced by a new page. They did a good job confusing me, it was very impressive. I don't confuse easily.
  - desolved 2y ago
    Why did you do that? Did you think it was real?
lordofmoria 2y ago
The section “Recognizing AI phishing attempts” is mournfully short, but there’s some companies out there like Jericho Security (https://www.jerichosecurity.com https://www.jerichosecurity.com) that are working on countermeasures, at least for enterprises.
- achneerov 2y ago
  Is this a self promo?
  - lordofmoria 2y ago
    no, it’s not…
- asynchronous 2y ago
  I’m skeptical we can develop effective ones, due to the fact that we have been unable to solve non-AI phishing problems but I welcome their attempt.
  - Mistletoe 2y ago
    When Google added Yubikeys it reduced phishing internally to zero. https://krebsonsecurity.com/2018/07/google-security-keys-neutralized-employee-phishing/ https://krebsonsecurity.com/2018/07/google-security-keys-neu... I adore my Yubikey.
    - asynchronous 2y ago
      Good article, but that does not cover or solve B2B email compromise/scams. It’s impossible to take the human out of the box here.
luen 2y ago
I want to know, how are you using AI now.
kagevf 2y ago
I expect that with AI, we'll be less able to rely on the heuristic of bad grammar to easily detect phishing. That one flaw gave the phishers away so often, and made it so obvious ...
- snowfield 2y ago
  It goes the other way now, overly verbose responses and perfect grammar sets off the warning bells.
  - pzo 2y ago
    Scammers can probably easy adapt adding some random misspellings after inference
  - irjustin 2y ago
    They'll adapt. You can easily ask chat4 to make it sound "less formal" or "more teenager"
    - IanCal 2y ago
      Absolutely. I was talking to a teacher and they said the translations were easy to spot because they were so good, far beyond what the kid was normally doing. I then showed them the same kind of prompt plus something like "write this as an X year old French student with only moderate grasp of ..." and it was far more plausible. I noticed how well it understands general parlance after it created marketing style copy for me and I told it to sound "less wanky" and it made it much more to the point.
- salviati 2y ago
  The bad grammar is on purpose. I know of two possible reasons: * Bayesian poisoning https://en.wikipedia.org/wiki/Bayesian_poisoning https://en.wikipedia.org/wiki/Bayesian_poisoning * Weeding out poor mark candidates https://josephsteinberg.com/why-scammers-make-spelling-and-grammar-mistakes/ https://josephsteinberg.com/why-scammers-make-spelling-and-g...
aaron695 2y ago
[dead]
nottorp 2y ago
> Is it your fate now to do due diligence on every email you receive? Always has been. Tbh the browser/email client makers are complicit in these phishing attempts for hiding the URLs and the actual email addresses. Put them back!
- HPsquared 2y ago
  Also the URL "security scanner" things on corporate email systems. The user can't see the URL by hovering over it.
  - reginald78 2y ago
    I was about to complain about this. I take security training to inspect URLs and then Microsoft safe link or whatever it's called gives me a twenty line long URL filled with random characters. I have to trust it works since they took my manual inspection ability away.
    - Grimblewald 2y ago
      i am fairly certain that whole thing is about tracking and not security. If it was about security, then they could still include the real URL in the "safeurl" url. They do not do this though, because it is not about safety, it is about data.
  - switch007 2y ago
    When the security team does that, they give up all right to tell people to analyse URLs, and assume all responsibility for anything bad happening from clicking on links. As they're advertising links as secure
- nonrandomstring 2y ago
  > Tbh the browser/email client makers are complicit in these phishing attempts for hiding the URLs and the actual email addresses. It's worse. Research "Scamicry". Big business now is so fake, such a grift, drenched in PR deception, and lacking integrity and trustworthiness, there isn't much space left between what is "legitimate" and what is a scam. If businesses like Google or Facebook hide URLs and email addresses that's not a casual "mistake". It's because that's to their profitable advantage to do so. And they know it puts you in harms way. So yes, they're complicit in scams. To make themselves a little more competitive businesses are always learning from scammers, meanwhile good scammers keenly learn from businesses to look more legit. Some ransomware "services" even have better customer support than billion dollar companies. And big business is certainly using the same AI tools as cybercriminals. So a problem isn't how clever and scurrilous scammers have gotten, it's how far legitimate services have fallen so that ordinary folk struggle to know the difference. How can we trust our own insticts for selecting what is good and wholesome from what is rotten, when there are few moral differences? The only difference resides in a digital identifier.
feverzsj 2y ago
Artificial Intelligence vs. Actually Indian
- psychoslave 2y ago
  I was called by an Indian like ten years ago, trying to convince me to follow some script obviously made for Windows when at the time I already hadn’t a Windows box for quite some time. Fun moment, in that specific case. Probably the funniest thing here is that this call reached me despite the fact that I am French, living in France. And so I really wonder how they ended up calling me. I mean, chance I would understand some English speaker with an Indian accent (I like how it sounds, but it’s definitely an additional difficulty as a non-native). I read here and there how extortion of old USA citizens by some organized Indian citizens is really a thing. To my mind the main issue at stake is that we have global level communication facilities, extremely high wealth disparities at world scale, and no compelling global social endeavor to reach an harmonization of human quality of life for everyone. I don’t mean the latter is on the official agenda of most countries out there either, but at global scale it’s obviously even worst. With all that in mind, blaming a whole nation for the illegitimate actions of some minority in the country, all the more when the international geopolitical context itself is all but fair, is probably not going to solve any issue.
  - dghughes 2y ago
    It seems like the majority are from India but no surprise with a population that's technical and soon to exceed China's population. CBC the Canadian nations news service has been trying to track scammers in India https://www.cbc.ca/news/world/tech-support-scam-india-marketplace-1.5298336 https://www.cbc.ca/news/world/tech-support-scam-india-market... Even Jim Browning and Kit Boga on YouTube two guys who scam the scammers it seems to be 100% people based in India.
    - kibwen 2y ago
      > soon to exceed China's population Note that India has had a larger population than China for at least a year or two.
greyrouting 2y ago
Why don’t US phone carriers give their users the ability to block foreign calls terminating in the U.S., at the telephony signaling layer? In almost no case do I ever want to receive a phone call from a foreign country with a spoofed number. Nor do I think anyone in my family wants to either.
- smeej 2y ago
  I've assumed these were VOIP calls, not actual telephony calls from other countries.
- mrguyorama 2y ago
  The same exact reason they are so slow to roll out new security features that would prevent things like caller ID spoofing: They get paid every time a scammer makes a phone call.
dbspin 2y ago
I'm kind of amazed how slow 'AI phishing' has been to roll out. The technology for customised text based attacks at scale has been available at least since Llama was open sourced. The tech for custom voice and image based attacks is basically there too with whisper / tortoise and stable diffusion - though clearly more expensive to render. I'm honestly not sure why social networks aren't being leveraged more to target and spoof individuals - especially elderly people. Tailored attacks impersonating text or voice messages from close contacts and family members should be fairly common, and yet they're not. Robo-calls that carry out a two way conversation convincingly impersonating bank or police officials should be everywhere. Yet the only spam-calls I ever receive are from Indian call centres or static messages using decades old synthesised voice tech.
- Der_Einzige 2y ago
  It appears that criminals really are stupid, and thank goodness for it!
  - cess11 2y ago
    Most people are criminals. Speeding, piracy, dubious porn, and so on. In a wider sense consumption of products or other use of criminally exploited labour. At the same time, most people are more clever than one tends to expect.
    - throwup238 2y ago
      ”If the prosecutor is obliged to choose his cases, it follows that he can choose his defendants. Therein is the most dangerous power of the prosecutor: that he will pick people that he thinks he should get, rather than pick cases that need to be prosecuted. With the law books filled with a great assortment of crimes, a prosecutor stands a fair chance of finding at least a technical violation of some act on the part of almost anyone." -Justice Robert H Jackson
  - ToucanLoucan 2y ago
    Why bother operating a much more complex LLM stack when you're already raking in cash from confused boomers trying to pay the IRS off with iTunes gift cards? Their system works. They'll take up machine learning powered tools once the old farts all die off/go broke and they need more complicated scams for more technology-savvy victims. I'm being glib here but also if you're the type of person who gets texts from the IRS from a number you've never seen and take it at face value that you can pay off your overdue tax bill with gift cards... like, you are already the perfect victim for this sort of scam. They don't need to be good, they just need you to self identify and leap right into the trap.
manishsharan 2y ago
This is going to become so much worse https://news.ycombinator.com/item?id=40942307 https://news.ycombinator.com/item?id=40942307 Imagine old people getting phone calls from frantic children. They won't know real from fake. Add tech like this to SIM forgery ..and we will devolve from a high trust society to a no trust society.
- meroes 2y ago
  Happened to my family. Grandfather got a call from a panicky grandchild that sounded like them. The teller at Western Union is the only reason it was stopped. The scary thing is this happened more than 5 years ago so it’s only getting worse.
- ryandrake 2y ago
  I think everyone with elderly parents already needs to have "the talk" with those parents, to help them to understand and deal with common (and less common) scams that prey on old people, what forms of communication to trust, what capabilities scammers have, and so on. All that is changing is the scammers' capabilities.
jbaczuk 2y ago
> link or attachment that when clicked or downloaded, takes you to a spoofed website or installs malicious software on your device. Can someone show me a modern OS that would install software by clicking a link?
- autoexec 2y ago
  Does IOS count? In that case people have been compromised without clicking anything and just getting an invisible text message is enough. Browsers have exploits with sandbox escapes. Any link to a file that is automatically downloaded and opened in an application (office doc or PDF for example) can exploit vulnerabilities in the underlying application and allow for anything including remote code execution.
- mrguyorama 2y ago
  Early on in COVID, Zoom was doing very sketchy shit so they could "one click" install from a web browser on MacOS
MARK1947 2y ago
[flagged]
rldjbpin 2y ago
using ai might be bringing out some low-effort success but at the end of the day, it is skill issue on our front. a common heuristic to look out for is "badly"-written/spoken communication. the "AI vs Actual Indian" comment and nigerian prince emails stand out for most people, but they still ended up working well enough to become this wide-spread. you just need to employ some critical thinking now for most external communication now. it is no different from some highly-motivated scammers doing it the old-fashioned way. at the end of the day, we are trying to replicate the success of some native-speaking teens (https://news.ycombinator.com/item?id=32959001 https://news.ycombinator.com/item?id=32959001).
mmaniac 2y ago
I've already experienced two AI-powered phishing attempts personally in the last few weeks. One was pretty transparent, but the other almost got me. I expect we'll all see a lot more of these soon.
dough101 2y ago
[dead]

197 points by pwmtr 2y ago

It’s actually worse than that - AI powered phishing sites will also copy your device profile and mouse, gesture and keyboard signature and use this to get past common anti-fraud techniques like device fingerprinting and behavioural biometrics.

advael 2y ago
I mean, the mere existence of said biometrics imply that they're inferrable and thus bad security, like basically all biometrics
- sroussey 2y ago
  And yet, used extensively
  - advael 2y ago
    I mean SSNs are the worst possible authentication mechanism and yet we still have to freak out every time they're leaked. Security practices are so utterly backwards everywhere that it's quite apparent no one powerful is incentivised to care even a little bit
    - jimmaswell 2y ago
      what's the practical alternative?
      - immibis 2y ago
        Scanning your digital ID card would be a start, but oHnO that's cOmmUniSM!
        bryanrasmussen 2y ago
        pickpocket's paradise!
        vincnetas 2y ago
        But it's not so scalable as online connected DB.
        reginald78 2y ago
        Wouldn't this just result in my digitalid getting lost in the next equifax breech?
        immibis 2y ago
        Did your credit card get lost in the last one?
        reginald78 2y ago
        How would I know one way or the other if it had? I don't have the same number anymore.
      - advael 2y ago
        Not pretending a GUID constitutes a security measure in the first place? It's just not the right tool for the job in any sense
      - mnau 2y ago
        Digital id card for every citizen? My id card can prove identity or sign a document and could for ~10 years. Estonia had it for over 20 years. Just give it to everyone. Today, it can likely be embedded in a cell phone instead of separate physical card.
      - Kiro 2y ago
        What do you mean? I don't know of any country except the US where SSN is used for authentication. In my country SSN is public information.
  - TechDebtDevin 2y ago
    When I call my bank they verify with my voice. There are further verification for meaningful actions but its still kind of crazy to be using "My Voice is my Password" this day in age.
    - AlotOfReading 2y ago
      Better than my bank, which tries to ask questions about my life from some lookup service that has incorrect information.
    - reginald78 2y ago
      Especially since "My Voice is my Passport" was defeated in that movie with a tape recorder and technology at the time. It was never a good idea and even the movie didn't seem to think so. Yet my bank just turned this on for me as well in 2024. Now I have to figure out how to disable it...and will it really be disabled?
- bryanrasmussen 2y ago
  basically everything that retains the same structure between two occurrences can be inferred. Only randomness cannot be inferred. But true randomness is not useful for determining if you are who you say you are.
  - advael 2y ago
    Even a password can be changed if there's a compromise. Biometrics are bad because they can be imitated, but not changed. A breach is permanent
    - bryanrasmussen 2y ago
      good point, but that was left out of the earlier statement about inference. I suppose I should have inferred it however.
      - advael 2y ago
        Yea, my bad I guess. I tend to think people mostly get that biometrics are, well, mostly immutable and that not being able to switch them up in response to a suspected breach is a huge inherent weakness. So the only defense I really get of them from anyone is that the effort for the user is minimized while the effort for the attacker is still fairly high. The problem with that is why I mention inferrability: The existence of a computer system that can authenticate via a biometric implies the existence of one that can capture and spoof it, and we don't have any reason to believe this involves, say, more of a cost disparity than cracking a password, let alone anything approaching a strong one-way function. If your face is your key, do you start hiding your face on the street so no one can steal it? Same thing for behaviorals
- chefandy 2y ago
  I think this is one of those "the only thing that's worse is everything else" situations. Surely there are solutions, but I doubt there are solutions banks and payment processors would be interested in paying for, and at least the US government isn't particularly interested in compelling banks to do anything expensive.
coffeebeqn 2y ago
What does AI have to do with capturing inputs ?
- orbital-decay 2y ago
  Not just capturing, but training on captured inputs to replicate the fingerprint.
  - snowfield 2y ago
    laughable
    - ted_bunny 2y ago
      How about an actual rebuttal instead of smug dismissal? Let us in on the joke, genius.
      - snowfield 2y ago
        If you want to replicate a fingerprint. Just get one image of one fingerprint. If you want to create an average looking fingerprint, use an AI

WesternWind 2y ago

Hey, just going to say what I've been telling folks IRL, if you are reading this, and your parents and family members aren't tech savvy, you need to set them up with two factor authentication now. Because you know how to do that, and it's so much easier than helping them when they get hacked.

ethbr1 2y ago
As evidence of the current state of play: Friend receives an email from ISP, asking her to contact them. She searches, comes across a "customer service number" on a legit looking page, calls them up. (Whoever she called) plays out a 30 minute charade about how she's been flagged by IRS for illegal activity and is about to have her business accounts frozen, including multiple phone transfers to "another party" (played by different people) to boost authenticity. And during this whole time, they not once asked her for any "red flag" information (e.g. account #, SSN). Instead, it seemed to be a shell game of extracting limited information (last 3 of your account #?), then having "unrelated" parties parrot that back as proof of their "working for the government." I expect it would have eventually escalated into an actionable ask, but they were definitely playing the intermediate-term game. If not for the utter moral black hole of the endeavor, I'd be kind of impressed.
- __MatrixMan__ 2y ago
  I shouldn't, but sometimes I play along just to see what the scam looks like. Last time I did this, it took three days of texting my new friend before it was finally clear that what she really wanted more than anything was to teach me to trade cryptocurrency. Once, I thought I had her, because she spelled D&D like: D&D, but she played it off real cool and just explained that her English isn't that great so she used translation software. In retrospect I think that all of her probing questions about my Svirfneblin cleric were because she later intended call him up and teach him to trade cryptocurrency. I like to think he's in some scammer's database now, causing confusion. He'd like that too. Once I understood what she was after, I explained that my problem with cryptocurrency was that it resembled money too closely and really what I'd like to do with blockchains is to do away with money in favor of something entirely different. Her training dataset had not prepared her for this conversation, so it was quite clear when her human handler took over. They were very rude, unlike their AI pet, and tried to bully me into sharing other people's contact info, which is when I lost interest.

29athrowaway 2y ago

On YouTube, I saw a deepfake of Elon Musk asking people to scan a QR code and buy crypto.

pixl97 2y ago
Yep these are super common, especially on days spacex is launching starship. They only live broadcast it on X so it allows these scammers to step in and attempt to trick people.
- h4kor 2y ago
  I watched the last starship launch on a scam YouTube stream. It was 30 minutes delayed, which a thought was because Musk wanted to promote Twitter. They said multiple times that Elon will announce something big after the launch. Directly after lift off it cut to Elon holding a speech and I only noticed this been a scam channel when he talked about the QR code and crypto.
  - yamumsahoe 2y ago
    same. stopped watching youtube live after this, realize that every live video could be a scam or distorted or modified or outdated in any way. weird.
6510 2y ago
When I typed my phone number in the box on "Musk" "investment" website my landline rang instantly after entering the last digit. It was definitely an onkeydown event. A friendly fast talking man in an extremely busy sounding (fake) call center asked me if I was $name_I_put_in_the_form. The voices in the background were people further down the sign up process. I said yes, then asked how he got my number. He said I just filled out the form on the website. Then the form was replaced by a new page. They did a good job confusing me, it was very impressive. I don't confuse easily.
- desolved 2y ago
  Why did you do that? Did you think it was real?

lordofmoria 2y ago

The section “Recognizing AI phishing attempts” is mournfully short, but there’s some companies out there like Jericho Security (https://www.jerichosecurity.com https://www.jerichosecurity.com) that are working on countermeasures, at least for enterprises.

achneerov 2y ago
Is this a self promo?
- lordofmoria 2y ago
  no, it’s not…
asynchronous 2y ago
I’m skeptical we can develop effective ones, due to the fact that we have been unable to solve non-AI phishing problems but I welcome their attempt.
- Mistletoe 2y ago
  When Google added Yubikeys it reduced phishing internally to zero. https://krebsonsecurity.com/2018/07/google-security-keys-neutralized-employee-phishing/ https://krebsonsecurity.com/2018/07/google-security-keys-neu... I adore my Yubikey.
  - asynchronous 2y ago
    Good article, but that does not cover or solve B2B email compromise/scams. It’s impossible to take the human out of the box here.

luen 2y ago

I want to know, how are you using AI now.

kagevf 2y ago

I expect that with AI, we'll be less able to rely on the heuristic of bad grammar to easily detect phishing. That one flaw gave the phishers away so often, and made it so obvious ...

snowfield 2y ago
It goes the other way now, overly verbose responses and perfect grammar sets off the warning bells.
- pzo 2y ago
  Scammers can probably easy adapt adding some random misspellings after inference
- irjustin 2y ago
  They'll adapt. You can easily ask chat4 to make it sound "less formal" or "more teenager"
  - IanCal 2y ago
    Absolutely. I was talking to a teacher and they said the translations were easy to spot because they were so good, far beyond what the kid was normally doing. I then showed them the same kind of prompt plus something like "write this as an X year old French student with only moderate grasp of ..." and it was far more plausible. I noticed how well it understands general parlance after it created marketing style copy for me and I told it to sound "less wanky" and it made it much more to the point.
salviati 2y ago
The bad grammar is on purpose. I know of two possible reasons: * Bayesian poisoning https://en.wikipedia.org/wiki/Bayesian_poisoning https://en.wikipedia.org/wiki/Bayesian_poisoning * Weeding out poor mark candidates https://josephsteinberg.com/why-scammers-make-spelling-and-grammar-mistakes/ https://josephsteinberg.com/why-scammers-make-spelling-and-g...

aaron695 2y ago

[dead]

nottorp 2y ago

> Is it your fate now to do due diligence on every email you receive? Always has been. Tbh the browser/email client makers are complicit in these phishing attempts for hiding the URLs and the actual email addresses. Put them back!

HPsquared 2y ago
Also the URL "security scanner" things on corporate email systems. The user can't see the URL by hovering over it.
- reginald78 2y ago
  I was about to complain about this. I take security training to inspect URLs and then Microsoft safe link or whatever it's called gives me a twenty line long URL filled with random characters. I have to trust it works since they took my manual inspection ability away.
  - Grimblewald 2y ago
    i am fairly certain that whole thing is about tracking and not security. If it was about security, then they could still include the real URL in the "safeurl" url. They do not do this though, because it is not about safety, it is about data.
- switch007 2y ago
  When the security team does that, they give up all right to tell people to analyse URLs, and assume all responsibility for anything bad happening from clicking on links. As they're advertising links as secure
nonrandomstring 2y ago
> Tbh the browser/email client makers are complicit in these phishing attempts for hiding the URLs and the actual email addresses. It's worse. Research "Scamicry". Big business now is so fake, such a grift, drenched in PR deception, and lacking integrity and trustworthiness, there isn't much space left between what is "legitimate" and what is a scam. If businesses like Google or Facebook hide URLs and email addresses that's not a casual "mistake". It's because that's to their profitable advantage to do so. And they know it puts you in harms way. So yes, they're complicit in scams. To make themselves a little more competitive businesses are always learning from scammers, meanwhile good scammers keenly learn from businesses to look more legit. Some ransomware "services" even have better customer support than billion dollar companies. And big business is certainly using the same AI tools as cybercriminals. So a problem isn't how clever and scurrilous scammers have gotten, it's how far legitimate services have fallen so that ordinary folk struggle to know the difference. How can we trust our own insticts for selecting what is good and wholesome from what is rotten, when there are few moral differences? The only difference resides in a digital identifier.

feverzsj 2y ago

Artificial Intelligence vs. Actually Indian

psychoslave 2y ago
I was called by an Indian like ten years ago, trying to convince me to follow some script obviously made for Windows when at the time I already hadn’t a Windows box for quite some time. Fun moment, in that specific case. Probably the funniest thing here is that this call reached me despite the fact that I am French, living in France. And so I really wonder how they ended up calling me. I mean, chance I would understand some English speaker with an Indian accent (I like how it sounds, but it’s definitely an additional difficulty as a non-native). I read here and there how extortion of old USA citizens by some organized Indian citizens is really a thing. To my mind the main issue at stake is that we have global level communication facilities, extremely high wealth disparities at world scale, and no compelling global social endeavor to reach an harmonization of human quality of life for everyone. I don’t mean the latter is on the official agenda of most countries out there either, but at global scale it’s obviously even worst. With all that in mind, blaming a whole nation for the illegitimate actions of some minority in the country, all the more when the international geopolitical context itself is all but fair, is probably not going to solve any issue.
- dghughes 2y ago
  It seems like the majority are from India but no surprise with a population that's technical and soon to exceed China's population. CBC the Canadian nations news service has been trying to track scammers in India https://www.cbc.ca/news/world/tech-support-scam-india-marketplace-1.5298336 https://www.cbc.ca/news/world/tech-support-scam-india-market... Even Jim Browning and Kit Boga on YouTube two guys who scam the scammers it seems to be 100% people based in India.
  - kibwen 2y ago
    > soon to exceed China's population Note that India has had a larger population than China for at least a year or two.

greyrouting 2y ago

Why don’t US phone carriers give their users the ability to block foreign calls terminating in the U.S., at the telephony signaling layer? In almost no case do I ever want to receive a phone call from a foreign country with a spoofed number. Nor do I think anyone in my family wants to either.

smeej 2y ago
I've assumed these were VOIP calls, not actual telephony calls from other countries.
mrguyorama 2y ago
The same exact reason they are so slow to roll out new security features that would prevent things like caller ID spoofing: They get paid every time a scammer makes a phone call.

dbspin 2y ago

I'm kind of amazed how slow 'AI phishing' has been to roll out. The technology for customised text based attacks at scale has been available at least since Llama was open sourced. The tech for custom voice and image based attacks is basically there too with whisper / tortoise and stable diffusion - though clearly more expensive to render. I'm honestly not sure why social networks aren't being leveraged more to target and spoof individuals - especially elderly people. Tailored attacks impersonating text or voice messages from close contacts and family members should be fairly common, and yet they're not. Robo-calls that carry out a two way conversation convincingly impersonating bank or police officials should be everywhere. Yet the only spam-calls I ever receive are from Indian call centres or static messages using decades old synthesised voice tech.

Der_Einzige 2y ago
It appears that criminals really are stupid, and thank goodness for it!
- cess11 2y ago
  Most people are criminals. Speeding, piracy, dubious porn, and so on. In a wider sense consumption of products or other use of criminally exploited labour. At the same time, most people are more clever than one tends to expect.
  - throwup238 2y ago
    ”If the prosecutor is obliged to choose his cases, it follows that he can choose his defendants. Therein is the most dangerous power of the prosecutor: that he will pick people that he thinks he should get, rather than pick cases that need to be prosecuted. With the law books filled with a great assortment of crimes, a prosecutor stands a fair chance of finding at least a technical violation of some act on the part of almost anyone." -Justice Robert H Jackson
- ToucanLoucan 2y ago
  Why bother operating a much more complex LLM stack when you're already raking in cash from confused boomers trying to pay the IRS off with iTunes gift cards? Their system works. They'll take up machine learning powered tools once the old farts all die off/go broke and they need more complicated scams for more technology-savvy victims. I'm being glib here but also if you're the type of person who gets texts from the IRS from a number you've never seen and take it at face value that you can pay off your overdue tax bill with gift cards... like, you are already the perfect victim for this sort of scam. They don't need to be good, they just need you to self identify and leap right into the trap.

manishsharan 2y ago

This is going to become so much worse https://news.ycombinator.com/item?id=40942307 https://news.ycombinator.com/item?id=40942307 Imagine old people getting phone calls from frantic children. They won't know real from fake. Add tech like this to SIM forgery ..and we will devolve from a high trust society to a no trust society.

meroes 2y ago
Happened to my family. Grandfather got a call from a panicky grandchild that sounded like them. The teller at Western Union is the only reason it was stopped. The scary thing is this happened more than 5 years ago so it’s only getting worse.
ryandrake 2y ago
I think everyone with elderly parents already needs to have "the talk" with those parents, to help them to understand and deal with common (and less common) scams that prey on old people, what forms of communication to trust, what capabilities scammers have, and so on. All that is changing is the scammers' capabilities.

jbaczuk 2y ago

> link or attachment that when clicked or downloaded, takes you to a spoofed website or installs malicious software on your device. Can someone show me a modern OS that would install software by clicking a link?

autoexec 2y ago
Does IOS count? In that case people have been compromised without clicking anything and just getting an invisible text message is enough. Browsers have exploits with sandbox escapes. Any link to a file that is automatically downloaded and opened in an application (office doc or PDF for example) can exploit vulnerabilities in the underlying application and allow for anything including remote code execution.
mrguyorama 2y ago
Early on in COVID, Zoom was doing very sketchy shit so they could "one click" install from a web browser on MacOS

MARK1947 2y ago

[flagged]

rldjbpin 2y ago

using ai might be bringing out some low-effort success but at the end of the day, it is skill issue on our front. a common heuristic to look out for is "badly"-written/spoken communication. the "AI vs Actual Indian" comment and nigerian prince emails stand out for most people, but they still ended up working well enough to become this wide-spread. you just need to employ some critical thinking now for most external communication now. it is no different from some highly-motivated scammers doing it the old-fashioned way. at the end of the day, we are trying to replicate the success of some native-speaking teens (https://news.ycombinator.com/item?id=32959001 https://news.ycombinator.com/item?id=32959001).

mmaniac 2y ago

I've already experienced two AI-powered phishing attempts personally in the last few weeks. One was pretty transparent, but the other almost got me. I expect we'll all see a lot more of these soon.

dough101 2y ago

[dead]