4 ms·

This would make a much less compelling headline if it read "Spam filter let through a phishing email", would it not? Machine learning is just software. If huma

This would make a much less compelling headline if it read "Spam filter let through a phishing email", would it not? Machine learning is just software. If human intelligence didn't fall for phishing from time to time, no one would bother doing it. Sprinkling a bit of AI magic dust on a spam filter doesn't make it foolproof, but it does make for a rippin' clickbait headline. The assumption I'm making here, for the record, is that Apple Mail has a spam filter, and it isn't Apple Intelligence. The spam filter failed, and the AI® saw an important email and moved it to the top. That seems like an appropriate division of labor to me. If I have a funky LLM trying to guess what's important in my inbox, that might even be useful, and if not, there's the chronological order to fall back on. But does anyone want it second-guessing the spam filter? Not I for one.

acdha 2y ago
You misunderstood the problem: it’s not that their server’s spam filter missed a message but rather that the phishing attempt was made more plausible by the Mail client treating the message as important and displaying an excerpt without some of the cues which would alert people to it being suspicious. Think of it this way: some companies send junk mail designed to look like renewal offers or messages from your bank or insurance company. Would more or fewer people fall for those scams if, instead of seeing it in the general mail pile, their personal assistant handed them the letter inside and said “your car’s warranty is about to expire, you need to renew it”?
- samatman 2y ago
  I don't misunderstand the problem at all. Gmail has been putting those little yellow tags on emails that are supposed to be important for longer than I can remember. Do they do that to spam which happens to get through the cordon? You bet they do. Do they use machine learning? Also yes. Because "you forgot to renew your account" is... important. It's the spam filter's job to catch that. The only thing which makes this interesting is artificial intelligence fairy dust. It's what caused you to misunderstand a branded pile of matrix math as though it was a person, capable of showing judgement, who personally handed you a piece of mail. A mistake, I am sure, you would not make about Gmail's machine-assisted prioritization algorithm, because of mere familiarity, and due to no other difference in the intention or behavior of the software whatsoever. It's clickbait.
  - consteval 2y ago
    I don't understand how it's clickbait when the title perfectly and completely describes a real problem. Sure you could argue the problem isn't a big deal or doesn't matter (tough argument btw). But you can't say stuff is clickbait when it's not. Clickbait is like "The best brownie recipe that'll make your family stop hating you!" And then you click and it's 1% brownie recipe and 99% filler story and ads. Oh and also they're box brownies. Clickbait is NOT "the grass is green and tree bark is usually brown" and then you click and it tells you about the color of grass. No, you knew what you were getting into when you clicked and it's all true.
  - acdha 2y ago
    > Because "you forgot to renew your account" is... important. It's the spam filter's job to catch that. Yes, but we know that spam filters will never be perfect. This is a UI issue where an LLM is amplifying the impact of that failure - the opposite of the goal we should have as engineers to make things fail safely and avoid situations where the only thing preventing a problem is consistent high human diligence. That’s what makes it more than clickbait because it’s an existing problem being made worse by removing some of the cues which people rely on.

acdha 2y ago

You misunderstood the problem: it’s not that their server’s spam filter missed a message but rather that the phishing attempt was made more plausible by the Mail client treating the message as important and displaying an excerpt without some of the cues which would alert people to it being suspicious. Think of it this way: some companies send junk mail designed to look like renewal offers or messages from your bank or insurance company. Would more or fewer people fall for those scams if, instead of seeing it in the general mail pile, their personal assistant handed them the letter inside and said “your car’s warranty is about to expire, you need to renew it”?

samatman 2y ago
I don't misunderstand the problem at all. Gmail has been putting those little yellow tags on emails that are supposed to be important for longer than I can remember. Do they do that to spam which happens to get through the cordon? You bet they do. Do they use machine learning? Also yes. Because "you forgot to renew your account" is... important. It's the spam filter's job to catch that. The only thing which makes this interesting is artificial intelligence fairy dust. It's what caused you to misunderstand a branded pile of matrix math as though it was a person, capable of showing judgement, who personally handed you a piece of mail. A mistake, I am sure, you would not make about Gmail's machine-assisted prioritization algorithm, because of mere familiarity, and due to no other difference in the intention or behavior of the software whatsoever. It's clickbait.
- consteval 2y ago
  I don't understand how it's clickbait when the title perfectly and completely describes a real problem. Sure you could argue the problem isn't a big deal or doesn't matter (tough argument btw). But you can't say stuff is clickbait when it's not. Clickbait is like "The best brownie recipe that'll make your family stop hating you!" And then you click and it's 1% brownie recipe and 99% filler story and ads. Oh and also they're box brownies. Clickbait is NOT "the grass is green and tree bark is usually brown" and then you click and it tells you about the color of grass. No, you knew what you were getting into when you clicked and it's all true.
- acdha 2y ago
  > Because "you forgot to renew your account" is... important. It's the spam filter's job to catch that. Yes, but we know that spam filters will never be perfect. This is a UI issue where an LLM is amplifying the impact of that failure - the opposite of the goal we should have as engineers to make things fail safely and avoid situations where the only thing preventing a problem is consistent high human diligence. That’s what makes it more than clickbait because it’s an existing problem being made worse by removing some of the cues which people rely on.