5 ms·

I have a mail server for my personal email that I've been maintaining for the better part of a decade. It started out as Qmail/Courier, changed to Postfix/Couri

by ThJ 8y ago

I have a mail server for my personal email that I've been maintaining for the better part of a decade. It started out as Qmail/Courier, changed to Postfix/Courier and finally Postfix/Dovecot, always with Spamassassin as the spam filter. For a while, it was configured to discard spam. Later, I configured it to put the spam in a special IMAP folder. I have the same system today, but on top of that, I have a background shell script that watches my mail folders and runs sa-learn on them. Anything that appears in the spam folder is learned as spam. Anything that appears elsewhere is learned as ham. Most of the time, this will be email that has already been classified correctly, but if I move an email, it will reclassify it. Some time ago, it began leaking spam into the inbox. It turned out that I needed to make the Bayes scores more aggressive, and after I did that, it's been more or less perfect. More or less zero false positives and maybe a couple of false negatives per week. I find that the good old Bayes classifier is still the most powerful tool in my spam filtering toolbox. You just have to be persistent and consistent in how you train it and tune it. For example, you shouldn't train it to classify legitimate newsletters as spam even if they are undesirable. Instead, you should unsubscribe and put those in the trash folder. I find that this considerably reduces the false positives.

bitexploder 8y ago
I used to operate a mail server for my company, but it was just too much extra work to get sa working company-wide. It just got pinched out of the list of things I was willing to spend any time on because it is such a nuisance.
baud147258 8y ago
Cool setup! How do you check false positives? You take a look at the spam folder for legit emails?
- deleted 8y ago
  [deleted]
- systemfreund 8y ago
  Not the OP, but I have a similar setup, however instead of Postfix I am using OpenSMTPd, which I found much easier to configure. Checking for false positives is a manual process. I've never had legitimate mail marked as spam though, only the other way round. My experience with Gmail's filter is not as nice btw. At work, I had a couple of situations where important mails from a customer were flagged as Spam, which went unnoticed for two weeks.
- ThJ 8y ago
  Pretty much. I just check it occasionally. Did the same thing back when I was a Gmail user, because Gmail has false positives too from time to time.
ahje 8y ago
I use a similar setup. Spam is simply discarded, whereas mails from all whitelisted users will force autolearn. Any mails thrown into the spam folder will be passed on to sa-learn as spam (and submitted to SpamCop and Razor2). So far, no users have complained about false positives being discarded, and moving mails to the spam folder usually results in similar mails being blocked in the future. The Bayes filter is very, very effective on it's own, but as you mentioned it's not perfect. I've found that using Razor2 as a compliment is very effective.
joosters 8y ago
I'm a happy SpamAssassin user too, just wanted to add that I've found using sa-learn to teach it about ham/spam makes the spam detection a lot better. When spam gets through, I move it to a junk folder, and from time to time, use that folder to train SA on. The level of false positives is tiny, perhaps 1-2 emails a year.
sweetp 8y ago
I have the same postfix dovecot setup and would love to setup Spamassassin correctly. Do you know of any tutorials which outlines how?
3pt14159 8y ago
I use Gmail because I know I'm not going to be able to keep an email server secure over time. But I still have to deal with shitty newsletters and other junk. After putting up with it for too long I decided to just domain-filter people that send me newsletters. The only thing that caused me trouble was LinkedIn, where people would occasionally message me expecting a fast response. So I solved that problem by deleting my LinkedIn account. If I reset my, say, Twitter password I know where to find it. All other email from them and countless others is useless.
- diminoten 8y ago
  I'm no l33t hax0r or anything, but when I see "I've hosted a personal mail server for 10 years", I think, "I could probably own that person." Almost certainly false (I couldn't even think of a more technical word than "own"), but I just dunno why people take that risk. I get that the threat profile of Joe Schmoe isn't the same as Car Company, Inc., but still...
z92 8y ago
Did the same. From Qmail/Courier to now using Postfix/Dovecot. But am not using Spamassassin. Instead, filter all TLDs except those that I white listed. Then by reverse ip, dmarc and sfp check. And lastly block domains from which I have received a new mail and marked it as spam from the client side within 24 hours.
Twirrim 8y ago
I've more recently switched to rspamd from spamassassin. I've been using SA for more than a decade on my own server, but have been getting frustrated with its spam matching more recently. I've tweaked it and tweaked it (including doing things like telling it to trust pyzor more) and still obvious stuff is leaking through. That's even with a cron job running to train itself on email I move to a specific folder. There's certain patterns of spam that just won't go away. CPU usage was noticeable for it, too. I started looking for compiled alternatives, and rspamd fit that need. It's faster than SA, and using less resources, and seems to be hitting better than SA filtering.
wpietri 8y ago
My use followed a similar arc. One tip: look at your mail server logs for the top 100 bad addresses. For me they were a combination of incorrect first names (doug@, joe@) and incorrect roles (sales@, support@). Deliver all of that mail directly to a spam trainer. That gives me good odds the Bayesian filter will have repeatedly seen novel spam before they try my account. I also subaddress [1] any address I give to a vendor. So when those leak to spammers, I route those to sa-learn as well. It's a real pleasure to use the key spammer advantage -- indiscriminate volume -- against them. [1] https://tools.ietf.org/html/rfc5233 https://tools.ietf.org/html/rfc5233
mherdeg 8y ago
Hey - could you share some of your SA rules/config that are working well for you? I tend to find that when I manually categorize spam/ham and sa-learn, spam goes to 0 for a couple of weeks, then creeps upward as trends change (?!) or as they learn (?!?!).
excalibur 8y ago
I wish I had the patience to train Bayes properly. The problem in my case is coming up with a good method of feeding it ham (not just for my own mailbox, but for every user in the company, who obviously don't all correspond with the same people or utilize their mailboxes in the same way). I end up doing a LOT of manual configuration, my local.cf is currently 178 KB (which doesn't sound that big, but it's a lot of rules to run against each and every message).
- julianz 8y ago
  I use Roundcube for webmail, which auto-collects outgoing addresses in an address book (and you can obviously manually add addresses as well). I have a script that periodically exports the address book for each user and uses that as a post-Bayes filter; anything from a known address is automatically whitelisted, and if it would have been marked as spam by Bayes then it is also trained as ham. It works pretty well.
autoexec 8y ago
conventional wisdom is to filter unwanted newsletters and never click the unsubscribe links. Clicking those links confirms that your email address is valid and being monitored which can make it more valuable when the company shares and sells your email address to others. Also attackers can forge newsletters from respectable companies or just made up something new entirely and the unsubscribe links can be used to send you to malicious sites that can open the door to malware infections and phishing. When someone sends you trash, you should assume it is toxic and avoid interacting with it as much as possible.
- davchana 8y ago
  I encountered same thing, malicious spam, & logged it at https://gitlab.com/davchana/gmail-indian-spam-domains/blob/master/readme.md https://gitlab.com/davchana/gmail-indian-spam-domains/blob/m...
- ThJ 8y ago
  When I said legit newsletters, I meant the sort you get from sites you've already signed up for. Newsletters that arrive out of the blue are not legit and you should of course never interact with those, but that's self-evident for the reasons you stated, and the whole point of me mentioning legit newsletters specifically was to point out that these shouldn't go in your spam folder because they tend to look too similar to legitimate notification emails, and you only end up skewing the filter toward false positives.
SwellJoe 8y ago
I find the "anything I don't like goes into spam" is among the biggest problems for classifying spam using rules shared across users, and has been forever. We support training the classifier in our products, via a couple of different mechanisms, but don't really make it too evident how to enable it and use it, because the result is often a lot of confusion as users break their classifier over time by putting legitimate (but no longer wanted) mail into the spam folder. Unsurprisingly, GMail also exhibits this problem. I have to check my spam folder regularly because it gets a ton of false positives these days. Nearly always newsletters and such I've signed up for. Another category of common false positive, which is admittedly harder to cope with, is moderation notifications from my WordPress sites...it's often actually a spam post, but it seems to learn the shape of those messages and start to treat them all as spam, even the legitimate ones. Spam filtering has always been, and continues to be, hard. It seemed, for a brief shining moment a few years ago, that spam was on the run...but, it's come roaring back in recent years, I guess as cloud-based mail services made it feasible for spammers to bounce from one provider to another, to keep sending through "known trustworthy" senders. There probably ought to be some way to punish those providers that look the other way when used for spam, but blocking them completely is likely to impact non-spammers (but might be a necessary pain to reduce spam).
- PopeDotNinja 8y ago
  I forget at times that I originally started using Gmail because of the spam filtering. My main problem wasn't getting spam, it was getting alerts on my damn phone that I got spam. I don't mind checking the spam folder, as it's like having a bunch of people at the front door waiting for me to come outside. If they all knocked, I'd never go outside.
z3t4 8y ago
You can also use the same technique to categorize your e-mail in more categories then just spam and not spam, I for example have a category for "commercial", eg. newsletters and offers, kinda like Gmail started with some time ago. But I also have more categories, like a category for each project.

by ThJ 8y ago

bitexploder 8y ago

I used to operate a mail server for my company, but it was just too much extra work to get sa working company-wide. It just got pinched out of the list of things I was willing to spend any time on because it is such a nuisance.

baud147258 8y ago

Cool setup! How do you check false positives? You take a look at the spam folder for legit emails?

deleted 8y ago
[deleted]
systemfreund 8y ago
Not the OP, but I have a similar setup, however instead of Postfix I am using OpenSMTPd, which I found much easier to configure. Checking for false positives is a manual process. I've never had legitimate mail marked as spam though, only the other way round. My experience with Gmail's filter is not as nice btw. At work, I had a couple of situations where important mails from a customer were flagged as Spam, which went unnoticed for two weeks.
ThJ 8y ago
Pretty much. I just check it occasionally. Did the same thing back when I was a Gmail user, because Gmail has false positives too from time to time.

ahje 8y ago

I use a similar setup. Spam is simply discarded, whereas mails from all whitelisted users will force autolearn. Any mails thrown into the spam folder will be passed on to sa-learn as spam (and submitted to SpamCop and Razor2). So far, no users have complained about false positives being discarded, and moving mails to the spam folder usually results in similar mails being blocked in the future. The Bayes filter is very, very effective on it's own, but as you mentioned it's not perfect. I've found that using Razor2 as a compliment is very effective.

joosters 8y ago

I'm a happy SpamAssassin user too, just wanted to add that I've found using sa-learn to teach it about ham/spam makes the spam detection a lot better. When spam gets through, I move it to a junk folder, and from time to time, use that folder to train SA on. The level of false positives is tiny, perhaps 1-2 emails a year.

sweetp 8y ago

I have the same postfix dovecot setup and would love to setup Spamassassin correctly. Do you know of any tutorials which outlines how?

3pt14159 8y ago

I use Gmail because I know I'm not going to be able to keep an email server secure over time. But I still have to deal with shitty newsletters and other junk. After putting up with it for too long I decided to just domain-filter people that send me newsletters. The only thing that caused me trouble was LinkedIn, where people would occasionally message me expecting a fast response. So I solved that problem by deleting my LinkedIn account. If I reset my, say, Twitter password I know where to find it. All other email from them and countless others is useless.

diminoten 8y ago
I'm no l33t hax0r or anything, but when I see "I've hosted a personal mail server for 10 years", I think, "I could probably own that person." Almost certainly false (I couldn't even think of a more technical word than "own"), but I just dunno why people take that risk. I get that the threat profile of Joe Schmoe isn't the same as Car Company, Inc., but still...

z92 8y ago

Did the same. From Qmail/Courier to now using Postfix/Dovecot. But am not using Spamassassin. Instead, filter all TLDs except those that I white listed. Then by reverse ip, dmarc and sfp check. And lastly block domains from which I have received a new mail and marked it as spam from the client side within 24 hours.

Twirrim 8y ago

I've more recently switched to rspamd from spamassassin. I've been using SA for more than a decade on my own server, but have been getting frustrated with its spam matching more recently. I've tweaked it and tweaked it (including doing things like telling it to trust pyzor more) and still obvious stuff is leaking through. That's even with a cron job running to train itself on email I move to a specific folder. There's certain patterns of spam that just won't go away. CPU usage was noticeable for it, too. I started looking for compiled alternatives, and rspamd fit that need. It's faster than SA, and using less resources, and seems to be hitting better than SA filtering.

wpietri 8y ago

My use followed a similar arc. One tip: look at your mail server logs for the top 100 bad addresses. For me they were a combination of incorrect first names (doug@, joe@) and incorrect roles (sales@, support@). Deliver all of that mail directly to a spam trainer. That gives me good odds the Bayesian filter will have repeatedly seen novel spam before they try my account. I also subaddress [1] any address I give to a vendor. So when those leak to spammers, I route those to sa-learn as well. It's a real pleasure to use the key spammer advantage -- indiscriminate volume -- against them. [1] https://tools.ietf.org/html/rfc5233 https://tools.ietf.org/html/rfc5233

mherdeg 8y ago

Hey - could you share some of your SA rules/config that are working well for you? I tend to find that when I manually categorize spam/ham and sa-learn, spam goes to 0 for a couple of weeks, then creeps upward as trends change (?!) or as they learn (?!?!).

excalibur 8y ago

I wish I had the patience to train Bayes properly. The problem in my case is coming up with a good method of feeding it ham (not just for my own mailbox, but for every user in the company, who obviously don't all correspond with the same people or utilize their mailboxes in the same way). I end up doing a LOT of manual configuration, my local.cf is currently 178 KB (which doesn't sound that big, but it's a lot of rules to run against each and every message).

julianz 8y ago
I use Roundcube for webmail, which auto-collects outgoing addresses in an address book (and you can obviously manually add addresses as well). I have a script that periodically exports the address book for each user and uses that as a post-Bayes filter; anything from a known address is automatically whitelisted, and if it would have been marked as spam by Bayes then it is also trained as ham. It works pretty well.

autoexec 8y ago

conventional wisdom is to filter unwanted newsletters and never click the unsubscribe links. Clicking those links confirms that your email address is valid and being monitored which can make it more valuable when the company shares and sells your email address to others. Also attackers can forge newsletters from respectable companies or just made up something new entirely and the unsubscribe links can be used to send you to malicious sites that can open the door to malware infections and phishing. When someone sends you trash, you should assume it is toxic and avoid interacting with it as much as possible.

davchana 8y ago
I encountered same thing, malicious spam, & logged it at https://gitlab.com/davchana/gmail-indian-spam-domains/blob/master/readme.md https://gitlab.com/davchana/gmail-indian-spam-domains/blob/m...
ThJ 8y ago
When I said legit newsletters, I meant the sort you get from sites you've already signed up for. Newsletters that arrive out of the blue are not legit and you should of course never interact with those, but that's self-evident for the reasons you stated, and the whole point of me mentioning legit newsletters specifically was to point out that these shouldn't go in your spam folder because they tend to look too similar to legitimate notification emails, and you only end up skewing the filter toward false positives.

SwellJoe 8y ago

I find the "anything I don't like goes into spam" is among the biggest problems for classifying spam using rules shared across users, and has been forever. We support training the classifier in our products, via a couple of different mechanisms, but don't really make it too evident how to enable it and use it, because the result is often a lot of confusion as users break their classifier over time by putting legitimate (but no longer wanted) mail into the spam folder. Unsurprisingly, GMail also exhibits this problem. I have to check my spam folder regularly because it gets a ton of false positives these days. Nearly always newsletters and such I've signed up for. Another category of common false positive, which is admittedly harder to cope with, is moderation notifications from my WordPress sites...it's often actually a spam post, but it seems to learn the shape of those messages and start to treat them all as spam, even the legitimate ones. Spam filtering has always been, and continues to be, hard. It seemed, for a brief shining moment a few years ago, that spam was on the run...but, it's come roaring back in recent years, I guess as cloud-based mail services made it feasible for spammers to bounce from one provider to another, to keep sending through "known trustworthy" senders. There probably ought to be some way to punish those providers that look the other way when used for spam, but blocking them completely is likely to impact non-spammers (but might be a necessary pain to reduce spam).

PopeDotNinja 8y ago
I forget at times that I originally started using Gmail because of the spam filtering. My main problem wasn't getting spam, it was getting alerts on my damn phone that I got spam. I don't mind checking the spam folder, as it's like having a bunch of people at the front door waiting for me to come outside. If they all knocked, I'd never go outside.

z3t4 8y ago

You can also use the same technique to categorize your e-mail in more categories then just spam and not spam, I for example have a category for "commercial", eg. newsletters and offers, kinda like Gmail started with some time ago. But I also have more categories, like a category for each project.