3 ms·

> It's not like this is unsolved problem. My god, are you joking or what? Spam, a solved problem? Far from it.

SoftTalker 4y ago
Why can I identify spam at a glance, with greater than 90% accuracy, from the subject line alone, and to Google it's an unsolved problem? Google reads and indexes every email that they deliver to a gmail inbox. They probably have the worlds largest and highest paid staff of ML experts, and they resort to auto-flagging based on IP address? Fix it.
- Avamander 4y ago
  > to Google it's an unsolved problem? It's an unsolved problem for anyone. > Why can I identify spam at a glance, with greater than 90% accuracy I don't think you understand how terrible even 99.998% accuracy is at their scale. > They probably have the worlds largest and highest paid staff of ML experts, and they resort to auto-flagging based on IP address? You think they aren't using them for this? IP addresses very likely go into their model as well, when determining something is spam. > Fix it. They are and adversaries are breaking them again.
  - kingjojo23 4y ago
    > I don't think you understand how terrible even 99.998% accuracy is at their scale. What does scale have to do with it, am I missing something? Wouldn’t a spam detection rate of 99.999% mean that only every 100000th spam mail would get through on an account basis, how’s that terrible?
    - Avamander 4y ago
      > What does scale have to do with it, am I missing something? The person I replied to said they can identify spam with 90% accuracy with a glance, my point was that it's abysmal as accuracy. Even orders of magnitude better it'd still be bad. > Wouldn’t a spam detection rate of 99.999% mean that only every 100000th spam mail would get through on an account basis, how’s that terrible? I didn't specify if it's per-account or a false positive rate, it's also actually not relevant to the point I'm trying to make. When you receive billions of letters a month small errors start to matter and they're doing a better job than the person I replied to thinks they do.
- denton-scratch 4y ago
  You can identify the spam because it's your inbox; you know what your goodmail is supposed to look like. For that reason you can use locally-trained naive Bayes on the client, and get good recognition (you don't need ML). Honestly, the best filter in my experience is Spamhaus Zen; it spots most spam with very few FPs. It's an IP blocklist.
  - Avamander 4y ago
    If you use something like rspamd, then doing ML is not terribly hard. Then it becomes a problem of providing a proper teaching set. Combining RBL's (be it URI, IP or domain WL's or BL's) will help with that. In the end it's still a hard problem, so takes a bunch of work.
megous 4y ago
Yes, spam classification is a solved problem. Perhaps even at the level of "this is new IP address with no reputation, is it sending 80% spam or 80% ham?" kind of questions with some defined precision at scale, but ceratinly at personal mailbox level, if you own and train your own clasification engine with thousands of samples of ham and spam you actually receive. That's very effective, and much better than my past experience with gmail.

Why can I identify spam at a glance, with greater than 90% accuracy, from the subject line alone, and to Google it's an unsolved problem? Google reads and indexes every email that they deliver to a gmail inbox. They probably have the worlds largest and highest paid staff of ML experts, and they resort to auto-flagging based on IP address? Fix it.

Avamander 4y ago
> to Google it's an unsolved problem? It's an unsolved problem for anyone. > Why can I identify spam at a glance, with greater than 90% accuracy I don't think you understand how terrible even 99.998% accuracy is at their scale. > They probably have the worlds largest and highest paid staff of ML experts, and they resort to auto-flagging based on IP address? You think they aren't using them for this? IP addresses very likely go into their model as well, when determining something is spam. > Fix it. They are and adversaries are breaking them again.
- kingjojo23 4y ago
  > I don't think you understand how terrible even 99.998% accuracy is at their scale. What does scale have to do with it, am I missing something? Wouldn’t a spam detection rate of 99.999% mean that only every 100000th spam mail would get through on an account basis, how’s that terrible?
  - Avamander 4y ago
    > What does scale have to do with it, am I missing something? The person I replied to said they can identify spam with 90% accuracy with a glance, my point was that it's abysmal as accuracy. Even orders of magnitude better it'd still be bad. > Wouldn’t a spam detection rate of 99.999% mean that only every 100000th spam mail would get through on an account basis, how’s that terrible? I didn't specify if it's per-account or a false positive rate, it's also actually not relevant to the point I'm trying to make. When you receive billions of letters a month small errors start to matter and they're doing a better job than the person I replied to thinks they do.
denton-scratch 4y ago
You can identify the spam because it's your inbox; you know what your goodmail is supposed to look like. For that reason you can use locally-trained naive Bayes on the client, and get good recognition (you don't need ML). Honestly, the best filter in my experience is Spamhaus Zen; it spots most spam with very few FPs. It's an IP blocklist.
- Avamander 4y ago
  If you use something like rspamd, then doing ML is not terribly hard. Then it becomes a problem of providing a proper teaching set. Combining RBL's (be it URI, IP or domain WL's or BL's) will help with that. In the end it's still a hard problem, so takes a bunch of work.

megous 4y ago

Yes, spam classification is a solved problem. Perhaps even at the level of "this is new IP address with no reputation, is it sending 80% spam or 80% ham?" kind of questions with some defined precision at scale, but ceratinly at personal mailbox level, if you own and train your own clasification engine with thousands of samples of ham and spam you actually receive. That's very effective, and much better than my past experience with gmail.