2 ms·
One thing that's become clear (at least in my experience with Bayesian filtering) is that less is more. You can actually waste a lot of time trying to reverse
by jgrahamc 19y ago
One thing that's become clear (at least in my experience with Bayesian filtering) is that less is more. You can actually waste a lot of time trying to reverse spammer trickery like misspelling. Dumb beats out smart very often.
In POPFile we do an enormous amount of work to detect spammer trickery and a couple of years back I did an analysis (published at Virus Bulletin) of the effectiveness of this extra work.
Quick summary of the highlights:
1. Looking at words in the headers was more important than the body
2. Of those the Subject line is the most important (most important words were free, what, only, huge, judge, visa and soft)
3. Most useful metadata was image height and width, font color and size, Euclidean distance between foreground and background color (bucketized), and the background color itself.
4. The only trick worth spending the time on was Invisible Ink (same background and foreground color). Camouflage was close to Invisible Ink.
5. The charset is interesting
In my commercial spam filter I do as little intelligent work as possible. Everything else is done by the Bayesian magic.