6 ms·

If I could find a way to filter spam as well as gmail does I would love to back to self-hosting my email. I just can't imagine a time when we'll get anything cl

by pmx 13y ago

If I could find a way to filter spam as well as gmail does I would love to back to self-hosting my email. I just can't imagine a time when we'll get anything close to what gmail can do without their huge resources.

thaumaturgy 13y ago
Yeah, Gmail has probably the best spam filtering of any mail service in the world. But! You can do pretty well at it, too, if you feel like putting in the effort. I host email for some clients, and most of my sysadmin time these days is spent dealing with spam in some fashion or another. 1. Greylisting. Greylisting is fantastic, and the package you're looking for here is sqlgrey. It's easy to use and admin and it has sensible defaults. You can pretty much just drop it in and immediately get an 80% reduction in spam. 2. Use the right MTA stack: currently I think that's Postfix and Dovecot. Postfix and Dovecot go together like ice cream and apple pie. Postfix is far easier to admin and configure than Sendmail, and Dovecot has good support for managesieve, which'll be important in a moment. 3. Install SpamAssassin and AMaViS. Getting it working correctly with Postfix in a multi-domain environment, so that each user can have their own SA settings, is a little bit tricky, but totally worth it. AMaViS by default wants to manage SA's settings. Don't let it. 4. Install the managesieve, password, and sauserprefs plugins. Now you can change your email password, your filters, and your SpamAssassin preferences right from the webmail interface. 5. Configure SpamAssassin to label messages "[SPAM]" (it also sets the X-Spam-Flag header) on junk messages, and then set up a default filter in managesieve to redirect those things to Junk folder. Bam, no missing emails, no cluttered inbox. Use imap on your favorite devices. 6. A couple of extras, like Fail2Ban, will help too. You can modify Fail2Ban to watch your mail.log for frequent attempts at nonexistent user accounts and then iptables those spammers. As with most things, you can spend as much additional time as you'd like fine tuning this, writing some custom software, and so on. But, the above will get you about 98% of the way to Gmail's level of spam filtering in about a day's worth of work.
- uaygsfdbzf 13y ago
  This is great, I use my own mail server but I was scared off when it came to trying to get SpamAssassin working with my Postfix/Dovecot setup, I'll have to give it another try soon!
nileshtrivedi 13y ago
Here's a somewhat aggressive solution: Allow incoming emails only from previously approved senders. Direct strangers to reach to you via Twitter.
- uaygsfdbzf 13y ago
  But what about those who do not have (or want) a Twitter account?
Yetanfou 13y ago
I have always self-hosted my mail. When spam started to become an issue I began looking around for a solution. I happened upon the newly minted spamassassin project, made it work to my liking, sent in a few patches and never looked back. Now, many years later, I still run SA in combination with Exim and Dovecot, filing unrecognised spam into a 'SPAM-it' folder from where it is picked up every night to train the bayes filter. Everything which is 'definitely' spam is refused at the gate, everything which is most certainly spam is filed in a spam folder which is out of sight and mostly out of mind, everything which is probably spam is automatically dumped in the aforementioned 'SPAM-it' folder (with autolearning disabled to avoid erroneous positive feedback). There is a 'HAM-it' folder to tell the filter about mail I care about as well as a 'FORGET-it' folder to untrain the filter on given messages. I use sieve to file messages in their designated folders. No other filtering is applied, no greylisting, no country blocks (I need to be able to communicate with people all over the world), no fail2ban, no nothin'. By keeping the filtering concentrated in one spot I avoid the trap of turning the mail system into a Rube Goldberg machine loaded with unintended consequences. On average, one or two spammy messages make it through the portcullis per day. I simply dump them in the 'SPAM-it' folder and forget about them. False positives are exceedingly rare, fortunately. On the whole I consider my mail setup to be functional and above all manageable. Everything needed to implement this is available in Debian. There is a handy Sieve editor available as a plugin to the Roundcube web mail client for those who like to click pretty boxes instead of writing filter scripts.
drchaos 13y ago
I do the following: * don't accept mail for non-existing recipient adresses * don't accept mail where the sender domain is one of my domains, except when whitelisted manually * don't accept mail from hosts on the NiXSpam list[1] (unlike some others, NixSpam has pretty reasonable policies with automatic delisting after 12 hours) These three rules together work pretty well, I get only a handful false negatives every day, and virtually no false positives, even though I publish my real mail address on many places in the web, and even used it in usenet, back in the days. Next step would be using SpamAssassin, but [2]. [1] http://www.heise.de/ix/NiX-Spam-DNSBL-and-blacklist-for-download-499637.html http://www.heise.de/ix/NiX-Spam-DNSBL-and-blacklist-for-down... [2] http://xkcd.com/1205/ http://xkcd.com/1205/
- gog 13y ago
  Please do not use NiXSpam or encourage others to use it. If I remember correctly they are pretty trigger happy with honeypot email addresses that are pretty easy to find on the internet. All your competition needs to do is register to your service using the honeypot email addresses and your legitimate email server/domain will be blacklisted on hosts that check NIXSpam.
- baghali 13y ago
  Your [2] link is awesome :)
carletes 13y ago
I'm very happy with dspam (http://dspam.nuclearelephant.com/ http://dspam.nuclearelephant.com/), and the Dovecot integration plugin (http://johannes.sipsolutions.net/Projects/dovecot-antispam http://johannes.sipsolutions.net/Projects/dovecot-antispam)
drzaiusapelord 13y ago
I half-assed this once and just made postfix check spamhaus and spamcop. Very little spam gets through. I'm thinking about turning on greylisting as well, but right now its not needed.

by pmx 13y ago

thaumaturgy 13y ago

Yeah, Gmail has probably the best spam filtering of any mail service in the world. But! You can do pretty well at it, too, if you feel like putting in the effort. I host email for some clients, and most of my sysadmin time these days is spent dealing with spam in some fashion or another. 1. Greylisting. Greylisting is fantastic, and the package you're looking for here is sqlgrey. It's easy to use and admin and it has sensible defaults. You can pretty much just drop it in and immediately get an 80% reduction in spam. 2. Use the right MTA stack: currently I think that's Postfix and Dovecot. Postfix and Dovecot go together like ice cream and apple pie. Postfix is far easier to admin and configure than Sendmail, and Dovecot has good support for managesieve, which'll be important in a moment. 3. Install SpamAssassin and AMaViS. Getting it working correctly with Postfix in a multi-domain environment, so that each user can have their own SA settings, is a little bit tricky, but totally worth it. AMaViS by default wants to manage SA's settings. Don't let it. 4. Install the managesieve, password, and sauserprefs plugins. Now you can change your email password, your filters, and your SpamAssassin preferences right from the webmail interface. 5. Configure SpamAssassin to label messages "[SPAM]" (it also sets the X-Spam-Flag header) on junk messages, and then set up a default filter in managesieve to redirect those things to Junk folder. Bam, no missing emails, no cluttered inbox. Use imap on your favorite devices. 6. A couple of extras, like Fail2Ban, will help too. You can modify Fail2Ban to watch your mail.log for frequent attempts at nonexistent user accounts and then iptables those spammers. As with most things, you can spend as much additional time as you'd like fine tuning this, writing some custom software, and so on. But, the above will get you about 98% of the way to Gmail's level of spam filtering in about a day's worth of work.

uaygsfdbzf 13y ago
This is great, I use my own mail server but I was scared off when it came to trying to get SpamAssassin working with my Postfix/Dovecot setup, I'll have to give it another try soon!

nileshtrivedi 13y ago

Here's a somewhat aggressive solution: Allow incoming emails only from previously approved senders. Direct strangers to reach to you via Twitter.

uaygsfdbzf 13y ago
But what about those who do not have (or want) a Twitter account?

Yetanfou 13y ago

I have always self-hosted my mail. When spam started to become an issue I began looking around for a solution. I happened upon the newly minted spamassassin project, made it work to my liking, sent in a few patches and never looked back. Now, many years later, I still run SA in combination with Exim and Dovecot, filing unrecognised spam into a 'SPAM-it' folder from where it is picked up every night to train the bayes filter. Everything which is 'definitely' spam is refused at the gate, everything which is most certainly spam is filed in a spam folder which is out of sight and mostly out of mind, everything which is probably spam is automatically dumped in the aforementioned 'SPAM-it' folder (with autolearning disabled to avoid erroneous positive feedback). There is a 'HAM-it' folder to tell the filter about mail I care about as well as a 'FORGET-it' folder to untrain the filter on given messages. I use sieve to file messages in their designated folders. No other filtering is applied, no greylisting, no country blocks (I need to be able to communicate with people all over the world), no fail2ban, no nothin'. By keeping the filtering concentrated in one spot I avoid the trap of turning the mail system into a Rube Goldberg machine loaded with unintended consequences. On average, one or two spammy messages make it through the portcullis per day. I simply dump them in the 'SPAM-it' folder and forget about them. False positives are exceedingly rare, fortunately. On the whole I consider my mail setup to be functional and above all manageable. Everything needed to implement this is available in Debian. There is a handy Sieve editor available as a plugin to the Roundcube web mail client for those who like to click pretty boxes instead of writing filter scripts.

drchaos 13y ago

I do the following: * don't accept mail for non-existing recipient adresses * don't accept mail where the sender domain is one of my domains, except when whitelisted manually * don't accept mail from hosts on the NiXSpam list[1] (unlike some others, NixSpam has pretty reasonable policies with automatic delisting after 12 hours) These three rules together work pretty well, I get only a handful false negatives every day, and virtually no false positives, even though I publish my real mail address on many places in the web, and even used it in usenet, back in the days. Next step would be using SpamAssassin, but [2]. [1] http://www.heise.de/ix/NiX-Spam-DNSBL-and-blacklist-for-download-499637.html http://www.heise.de/ix/NiX-Spam-DNSBL-and-blacklist-for-down... [2] http://xkcd.com/1205/ http://xkcd.com/1205/

gog 13y ago
Please do not use NiXSpam or encourage others to use it. If I remember correctly they are pretty trigger happy with honeypot email addresses that are pretty easy to find on the internet. All your competition needs to do is register to your service using the honeypot email addresses and your legitimate email server/domain will be blacklisted on hosts that check NIXSpam.
baghali 13y ago
Your [2] link is awesome :)

carletes 13y ago

I'm very happy with dspam (http://dspam.nuclearelephant.com/ http://dspam.nuclearelephant.com/), and the Dovecot integration plugin (http://johannes.sipsolutions.net/Projects/dovecot-antispam http://johannes.sipsolutions.net/Projects/dovecot-antispam)

drzaiusapelord 13y ago

I half-assed this once and just made postfix check spamhaus and spamcop. Very little spam gets through. I'm thinking about turning on greylisting as well, but right now its not needed.