2 ms·

You misunderstood the problem: it’s not that their server’s spam filter missed a message but rather that the phishing attempt was made more plausible by the Mai

by acdha 2y ago

You misunderstood the problem: it’s not that their server’s spam filter missed a message but rather that the phishing attempt was made more plausible by the Mail client treating the message as important and displaying an excerpt without some of the cues which would alert people to it being suspicious. Think of it this way: some companies send junk mail designed to look like renewal offers or messages from your bank or insurance company. Would more or fewer people fall for those scams if, instead of seeing it in the general mail pile, their personal assistant handed them the letter inside and said “your car’s warranty is about to expire, you need to renew it”?

samatman 2y ago
I don't misunderstand the problem at all. Gmail has been putting those little yellow tags on emails that are supposed to be important for longer than I can remember. Do they do that to spam which happens to get through the cordon? You bet they do. Do they use machine learning? Also yes. Because "you forgot to renew your account" is... important. It's the spam filter's job to catch that. The only thing which makes this interesting is artificial intelligence fairy dust. It's what caused you to misunderstand a branded pile of matrix math as though it was a person, capable of showing judgement, who personally handed you a piece of mail. A mistake, I am sure, you would not make about Gmail's machine-assisted prioritization algorithm, because of mere familiarity, and due to no other difference in the intention or behavior of the software whatsoever. It's clickbait.
- consteval 2y ago
  I don't understand how it's clickbait when the title perfectly and completely describes a real problem. Sure you could argue the problem isn't a big deal or doesn't matter (tough argument btw). But you can't say stuff is clickbait when it's not. Clickbait is like "The best brownie recipe that'll make your family stop hating you!" And then you click and it's 1% brownie recipe and 99% filler story and ads. Oh and also they're box brownies. Clickbait is NOT "the grass is green and tree bark is usually brown" and then you click and it tells you about the color of grass. No, you knew what you were getting into when you clicked and it's all true.
- acdha 2y ago
  > Because "you forgot to renew your account" is... important. It's the spam filter's job to catch that. Yes, but we know that spam filters will never be perfect. This is a UI issue where an LLM is amplifying the impact of that failure - the opposite of the goal we should have as engineers to make things fail safely and avoid situations where the only thing preventing a problem is consistent high human diligence. That’s what makes it more than clickbait because it’s an existing problem being made worse by removing some of the cues which people rely on.

by acdha 2y ago

samatman 2y ago

I don't misunderstand the problem at all. Gmail has been putting those little yellow tags on emails that are supposed to be important for longer than I can remember. Do they do that to spam which happens to get through the cordon? You bet they do. Do they use machine learning? Also yes. Because "you forgot to renew your account" is... important. It's the spam filter's job to catch that. The only thing which makes this interesting is artificial intelligence fairy dust. It's what caused you to misunderstand a branded pile of matrix math as though it was a person, capable of showing judgement, who personally handed you a piece of mail. A mistake, I am sure, you would not make about Gmail's machine-assisted prioritization algorithm, because of mere familiarity, and due to no other difference in the intention or behavior of the software whatsoever. It's clickbait.

consteval 2y ago
I don't understand how it's clickbait when the title perfectly and completely describes a real problem. Sure you could argue the problem isn't a big deal or doesn't matter (tough argument btw). But you can't say stuff is clickbait when it's not. Clickbait is like "The best brownie recipe that'll make your family stop hating you!" And then you click and it's 1% brownie recipe and 99% filler story and ads. Oh and also they're box brownies. Clickbait is NOT "the grass is green and tree bark is usually brown" and then you click and it tells you about the color of grass. No, you knew what you were getting into when you clicked and it's all true.
acdha 2y ago
> Because "you forgot to renew your account" is... important. It's the spam filter's job to catch that. Yes, but we know that spam filters will never be perfect. This is a UI issue where an LLM is amplifying the impact of that failure - the opposite of the goal we should have as engineers to make things fail safely and avoid situations where the only thing preventing a problem is consistent high human diligence. That’s what makes it more than clickbait because it’s an existing problem being made worse by removing some of the cues which people rely on.