5 ms·
A Plan for Spam (2002)
- lukego 8y agoCandidate for best blog post of all time? Quickly made a big impact on a real problem in the computing world. I remember reading this at the time, installing one of the implementations that immediately popped up (SpamAssassin), and finally having my spam separated from my ham. Edit: I had not considered until now the possibility that the "IKEA effect" might have made me overestimate the quality of filtering because of the effort that I put into training the classifier!
- TicklishTiger 8y agoIt sounds good but ... ...it did not work out. 17 years later and spam is still annoying. I use ThunderBird and tag every spam message as spam. Still, some spam messages get through. And because there are false positives sometimes, I always look through my spam folder before I empty it.
- coldtea 8y ago>17 years later and spam is still annoying. I use ThunderBird and tag every spam message as spam. With Gmail you hardly see any spam. >Still, some spam messages get through. Some? Back in the day it was in the hundreds every day if you were active online...
- detritus 8y agoGmail also dumps a lot of non-spam emails into its spam folders. The ones that drive me nuts (and have led me to having to sign up to fastmail for my business emails, rather than sending from my own shared server) are the ones that are valid responses to emails sent from GMail in the first place. Excellent logic (or lack thereof), Google. I don't actually believe so, but I have often wondered whether GMail has such an overly fastiduous spam filter simply to encourage people into its fold... .
- PakG1 8y agoSwitched from gmail because receiving too much spam. Part of the reason seems to be people using my generic name as a fake email address when they sign up for stuff, probably credit card promotions and the like. Wonder what it would be like if I used a custom domain with gmail. But not going back....
- stevekemp 8y agoIronically I run my own mail-server, and the majority of incoming spam I receive is routed through gmail.
- TicklishTiger 8y agoGmail can do that because they can see into the inboxes of many million of people. So if a mail is flagged as spam somewhere, it can use that info as a signal to classify it as spam in my inbox. That is not the content based plan for spam PG describes. And it comes at a cost. Complete loss of control over your inbox.
- zimpenfish 8y ago> Still, some spam messages get through. I believe spammers actively test their messages against things like SpamAssassin to sneak them through.
- shin_lao 8y agoI have a very different experience, I have only a couple of spams that don't end up in the spam folder. I do get promotional emails but when you unsubscribe they don't come back. I use Office 365 and Gmail. Both do a good job.
- jorangreef 8y agoI remember reading pg's post, it was a classic. Anyone here working on anti-spam? Where are things at today?
- amaccuish 8y agoNot "working" on anti-spam, but for my family's email server, we run rspamd and it works really well.
- Felz 8y agoI'm not directly working on spam, but it seems like there's a heavy reliance on ipv4 address reputation these days, which is probably one reason mailservers won't use ipv6 any time soon.
- thatsaguy 8y agoAddress reputation was always big, even back then. In the last years though, I cannot really recommend to use any of the DNSBL anymore. I've encountered more cases where legitimate servers were blocked due to netblock vicinity or indeed previous ownership than actual spam issues. Greylisting will still catch dynamic allocations almost as effectively, while you won't reject legitimate mail due to server and/or DNSBL issues.
- jorangreef 8y agoHave you tried using a quorum of DNSBL (e.g. barracudacentral.org, cbl.abuseat.org, truncate.gbudb.net) to reduce false positives? In other words, if at least two DNSBL queries agree, then reject, or feed this information to the rest of the spam pipeline?
- thatsaguy 8y agoThis is pretty common, and systems like SA do this for you by batching responses and calculating a score. I found this to be pretty much worthless if you already have greylisting, even for high-quality curated lists such as spamhaus SBL/XBL.
- dalbasal 8y agoThis really was the golden age of blogging. That unpretentious style of writing made me engage with so many ideas I never would have, reading professional writing.
- tw1010 8y agoThis still exists, you just need to know where to look.
- dalbasal 8y agoYou can't just write a comment like that. links please. ;)
- earenndil 8y agohttps://danluu.com/ https://danluu.com/ https://www.stilldrinking.org/ https://www.stilldrinking.org/ Steve yegge's various blogs (https://sites.google.com/site/steveyegge2/blog-rants https://sites.google.com/site/steveyegge2/blog-rants, https://steve-yegge.blogspot.com/ https://steve-yegge.blogspot.com/, https://medium.com/@steve.yegge https://medium.com/@steve.yegge) https://waitbutwhy.com/ https://waitbutwhy.com/ (bit cheesy but alright).
- peterwwillis 8y agoWhere's the Gmail option to let me turn a label into a unique bayesian filter? In the past year I have gotten >1,750 mails from recruiters, most of them unique addresses. I don't want to mark them as "Spam" because this is a type of "Spam" that I want to keep so I can refer to it later. I'd like to also un-train "not-so-Spammy" messages so I can see the jobs I'd be interested in, but I'm afraid of false positives making these harder to retain.
- tyingq 8y agoThey used to have smart labels in the gmail labs config section, but dropped it. It appeared to be a Bayesian classifier. You couldn't add your own labels, but it did work well. No idea why they dropped it. You could use popfile, it supports moving mail to named folders, which isn't the same as just applying a label, but works. http://getpopfile.org http://getpopfile.org
- nestorD 8y agoThere is a "filter similar message" option available (it is hidden with the answer /transfer / print / etc options).
- goblin89 8y agoPersonally I disabled Gmail’s spam filter long time ago[0]. (False positives plus I have some weak morbid curiosity as what comes in spam.) Not getting too much junk so far. Recently I started to give a different email to each service so that I can see if any get compromised, but so far I don’t think I’ve noticed anything like that. [0] I borrowed someone’s workaround where you create a filter that excludes emails matching a specific random UUID, which no email would match, with action “never send to spam”. Perhaps there’s now a more straightforward option.
- anticodon 8y agoThere's no way to disable spam filtering in Gmail because most of the filtering happens long before user filters are checked. If it weren't true, or spam folders would contain thousands of messages. My spam folder receives no more than a dozen of messages per month and I know that there're tens of thousands attempts to send me a spam.
- tw1010 8y agoFunny thing is that even to this day, most AI companies employs algorithms no sophisticated than in this almost 20-year old article.
- dvirsky 8y agoThis post blew my mind at the time, it was my first exposure to ML techniques.
- tw1010 8y agoWhen I first read this article it felt terribly old. An article from 10 years ago? That's ancient! Now it's almost 20 years old and for some reason it feels a lot fresher today than it did even back then. (Maybe history starts to compact the older you get.)