4 ms·

How a blog spammer got past Akismet's filters

10 points by Sam_Odio 19y ago

imp 19y ago
Thanks for the information, very useful.
mojuba 19y ago
For how long will people believe Bayesian filtering stops spam? It's just a cute theorem that in no way can compete with human intelligence. Or do you think some probability calculations can outsmart a spammer?
- brett 19y ago
  Did tlb add a pseudonym? http://tlb.org/whyspamcannotbestopped.html http://tlb.org/whyspamcannotbestopped.html Stop may be a little strong of a word, but people will keep believing that Bayesian filtering helps fight spam a great deal until someone comes up with a better idea.
  - mojuba 19y ago
    Bayesian filtering or any other heuristics comes at a price of occasionally loosing important messages, no matter how good those methods could be in reducing spam. And to my understanding, the best method so far is the statistical (voting-based) method that many large email servers run. And there is one philosophical problem that TLB doesn't mention: we ourselves haven't figured out yet what exactly is spam. Is there a good universal definition of unwanted mail that should be filtered unconditionally?
    - brett 19y ago
      And to my understanding, the best method so far is the statistical (voting-based) method that many large email servers run. That's Bayesian filtering. http://www.paulgraham.com/spam.html http://www.paulgraham.com/spam.html As to your last point, I know what I consider spam. That's all that is important to me. That's part of the whole point of Bayesian filtering. Each user can personally teach the filter.
      - mojuba 19y ago
        Voting is not Bayesian filtering. Not everything is Bayesian filtering in this world. And yes, I read that article. Self-learning algorithms suck, because they are not intelligent. Even if you had a human spam filter and you had to teach him/her what's important for you, even that kind of filter would give you false positives from time to time. The whole Bayesian filtering business was doomed to failure for very obvious reasons.
- byrneseyeview 19y ago
  The nice thing about Bayesian filtering is that it makes spam possible if and only if it's worth reading anyway. If spammers realize that the only way to get past a filter is to make their message statistically indistinguishable from something useful, they may decide that the best way to do that is to make it something useful.
  - mojuba 19y ago
    Spam was never absolutely useless (except those special messages that check your address for availability), and some people do buy fake Rolex watches or cheap viagra pills. These offers may or may not be legal, and that's a different kind of problem. Unsolicited mail can be useful to someone, or otherwise it wouldn't exist at all. The real problem with spam, again, is that you will always receive it unless you clearly define what's important for you, or maybe what's not important - either way. Is that possible?
    - byrneseyeview 19y ago
      Are you confusing gross and net use? Spammers value responses, and they can maximize those by either a) sending more spam, or b) getting more readers per message. I suspect that a) is finally becoming prohibitive because every new message trains all the Bayesian filters out there -- so it might work better for them to have fewer, weirder spams. And why just inject randomness into your message when you can inject usable order? The kind of people who read spam now are the kind of people who mass-forward jokes, so I can imagine a savvy spammer hiring comedians to write topical jokes, and appending a Viagra ad to end. They can 'beat' Bayesian filters by wrapping their spam in a message people will actually read and possibly forward.
      - mojuba 19y ago
        I'm not talking about fighting spam. I'm trying to understand what is spam, because unless we understand that we won't be able to stop it. Let's say you got a message that offers SEO optimization for your web site, and you are not a SEO expert. The message has a decent business writing style, there's a signature with a decent-looking contact information with an postal address, email and everything. And the message invites you to click on a link. Is it spam or not?
photomatt 19y ago
Akismet doesn't use Bayesian filtering.
- brett 19y ago
  That's interesting to know. Feel comfortable elaborating any?
- Sam_Odio 19y ago
  Updated the title to take bayesian out. I too would definitely want to hear more about what Akismet does use... That is, if you want to tell us ;)
- dood 19y ago
  Do you mean they don't use a naive Bayesian classifier, or that they don't use Bayesian probability at all? I'd have a hard time believing the latter.
sethjohn 19y ago
About 5 years ago I heard Bill Gates give a talk proposing a system to charge 1/10th penny charge for anyone to send you an e-mail. After the e-mail is accepted, you can excuse the charge for anyone sending legitimate e-mail...but spammers would wind up with many thousands of dollars in bills. Seems like a genius idea, wonder why I haven't heard anything more about it?
- randallsquared 19y ago
  There are a number of problems with it. First, it won't actually affect spammers unless nearly everyone is using it... what could force them to accept the charges? Second, it's turned out that even the email systems that ask people to confirm they sent something (which spambots can't understand in general) are too much for many people to bother with, so asking them to join a system in order to pay a fraction of a cent for each email they write would likely be too much trouble. Lastly, spam filters actually work really well. I get roughly a thousand spams a day to my primary address which has been continuously in use for ~7 years, and have to "mark as junk" maybe two spams a day -- the rest go straight to my junk folder.
- phil 19y ago
  Here's Schneier on why not: http://www.schneier.com/blog/archives/2007/05/1933_antispam_d.html http://www.schneier.com/blog/archives/2007/05/1933_antispam_d.html

10 points by Sam_Odio 19y ago

imp 19y ago

Thanks for the information, very useful.

For how long will people believe Bayesian filtering stops spam? It's just a cute theorem that in no way can compete with human intelligence. Or do you think some probability calculations can outsmart a spammer?

brett 19y ago
Did tlb add a pseudonym? http://tlb.org/whyspamcannotbestopped.html http://tlb.org/whyspamcannotbestopped.html Stop may be a little strong of a word, but people will keep believing that Bayesian filtering helps fight spam a great deal until someone comes up with a better idea.
- mojuba 19y ago
  Bayesian filtering or any other heuristics comes at a price of occasionally loosing important messages, no matter how good those methods could be in reducing spam. And to my understanding, the best method so far is the statistical (voting-based) method that many large email servers run. And there is one philosophical problem that TLB doesn't mention: we ourselves haven't figured out yet what exactly is spam. Is there a good universal definition of unwanted mail that should be filtered unconditionally?
  - brett 19y ago
    And to my understanding, the best method so far is the statistical (voting-based) method that many large email servers run. That's Bayesian filtering. http://www.paulgraham.com/spam.html http://www.paulgraham.com/spam.html As to your last point, I know what I consider spam. That's all that is important to me. That's part of the whole point of Bayesian filtering. Each user can personally teach the filter.
    - mojuba 19y ago
      Voting is not Bayesian filtering. Not everything is Bayesian filtering in this world. And yes, I read that article. Self-learning algorithms suck, because they are not intelligent. Even if you had a human spam filter and you had to teach him/her what's important for you, even that kind of filter would give you false positives from time to time. The whole Bayesian filtering business was doomed to failure for very obvious reasons.
byrneseyeview 19y ago
The nice thing about Bayesian filtering is that it makes spam possible if and only if it's worth reading anyway. If spammers realize that the only way to get past a filter is to make their message statistically indistinguishable from something useful, they may decide that the best way to do that is to make it something useful.
- mojuba 19y ago
  Spam was never absolutely useless (except those special messages that check your address for availability), and some people do buy fake Rolex watches or cheap viagra pills. These offers may or may not be legal, and that's a different kind of problem. Unsolicited mail can be useful to someone, or otherwise it wouldn't exist at all. The real problem with spam, again, is that you will always receive it unless you clearly define what's important for you, or maybe what's not important - either way. Is that possible?
  - byrneseyeview 19y ago
    Are you confusing gross and net use? Spammers value responses, and they can maximize those by either a) sending more spam, or b) getting more readers per message. I suspect that a) is finally becoming prohibitive because every new message trains all the Bayesian filters out there -- so it might work better for them to have fewer, weirder spams. And why just inject randomness into your message when you can inject usable order? The kind of people who read spam now are the kind of people who mass-forward jokes, so I can imagine a savvy spammer hiring comedians to write topical jokes, and appending a Viagra ad to end. They can 'beat' Bayesian filters by wrapping their spam in a message people will actually read and possibly forward.
    - mojuba 19y ago
      I'm not talking about fighting spam. I'm trying to understand what is spam, because unless we understand that we won't be able to stop it. Let's say you got a message that offers SEO optimization for your web site, and you are not a SEO expert. The message has a decent business writing style, there's a signature with a decent-looking contact information with an postal address, email and everything. And the message invites you to click on a link. Is it spam or not?

photomatt 19y ago

Akismet doesn't use Bayesian filtering.

brett 19y ago
That's interesting to know. Feel comfortable elaborating any?
Sam_Odio 19y ago
Updated the title to take bayesian out. I too would definitely want to hear more about what Akismet does use... That is, if you want to tell us ;)
dood 19y ago
Do you mean they don't use a naive Bayesian classifier, or that they don't use Bayesian probability at all? I'd have a hard time believing the latter.

sethjohn 19y ago

About 5 years ago I heard Bill Gates give a talk proposing a system to charge 1/10th penny charge for anyone to send you an e-mail. After the e-mail is accepted, you can excuse the charge for anyone sending legitimate e-mail...but spammers would wind up with many thousands of dollars in bills. Seems like a genius idea, wonder why I haven't heard anything more about it?

randallsquared 19y ago
There are a number of problems with it. First, it won't actually affect spammers unless nearly everyone is using it... what could force them to accept the charges? Second, it's turned out that even the email systems that ask people to confirm they sent something (which spambots can't understand in general) are too much for many people to bother with, so asking them to join a system in order to pay a fraction of a cent for each email they write would likely be too much trouble. Lastly, spam filters actually work really well. I get roughly a thousand spams a day to my primary address which has been continuously in use for ~7 years, and have to "mark as junk" maybe two spams a day -- the rest go straight to my junk folder.
phil 19y ago
Here's Schneier on why not: http://www.schneier.com/blog/archives/2007/05/1933_antispam_d.html http://www.schneier.com/blog/archives/2007/05/1933_antispam_d.html