6 ms·

I run a mail monitoring service [0], and we hear this complaint every now and then. A couple of things that regularly seems to trigger false positives in spam

by LeonM 8y ago

I run a mail monitoring service [0], and we hear this complaint every now and then. A couple of things that regularly seems to trigger false positives in spam algorithms: - no or misconfigured SPF and/or DKIM - no or misconfigured reverse-DNS - automatically included footer texts (confidentiality, copyright, safe a tree don't print, etc) - regular automatic replies from the domain (such as out of office notifications) - the use of embedded images (logos, human signatures, etc) We sometimes joke that these triggers were built in by the algorithm developers as a means of punishing those who litter their email with pointless texts and images. [0] https://www.mailhardener.com https://www.mailhardener.com

gambler 8y ago
And with all this corporate shills chant that "Hosting your own things is easier than ever before".
- LeonM 8y ago
  I know it's going to be an unpopular opinion in this thread, but my opinion is that email is one of the things that most businesses shouldn't run themselves.
  - feanaro 8y ago
    Why?
    - LeonM 8y ago
      Multiple reasons, but here are a few of them: - Email is hard to get right. There are many subtle configuration mistakes you can make, and it's pretty hard to detect those mistakes (hence one of the reasons why I started Mailhardener) - Your users will have high expectations. For email to be used effectively users expect email to work on many devices, have a webmail client, calendar integration, a good search function, good spam detection, infinite backups, ability to have huge attachments, etc, etc. - Your spam filter is never going to be as effective as that of the big email services. They have ML based spam filters that are constantly evolving. Even if you were to have access to their ML models, you wouldn't have enough data to effectively train that model. - Hosted email solutions are almost always more cost effective, regardless of the size of organisations. Running and maintaining an email service is labour intensive. Of course, there are also many legit use cases where you want to run your own email server. But for >99% of organisations, it just won't make any sense to run their own email.
      - feanaro 8y ago
        > - Email is hard to get right. There are many subtle configuration mistakes you can make, and it's pretty hard to detect those mistakes (hence one of the reasons why I started Mailhardener) It's not that hard if I've done it right. ;-) Jokes aside, yes, it's a bit overwhelming at the beginning, but I don't find it that cumbersome to keep it running. That is, it wouldn't be if Google was not actively being malicious in this regard. > - Your users will have high expectations. For email to be used effectively users expect email to work on many devices, have a webmail client, calendar integration, a good search function, good spam detection, infinite backups, ability to have huge attachments, etc, etc. From my experience, these aren't that hard to set up. If you're a corporation, your users are your employees. This is a bit different than if you were an email hosting company where users were paying for your product. In the former case, it is your responsibility to handle this well to serve the company's needs. It either serves you well or it does not. > - Your spam filter is never going to be as effective as that of the big email services. They have ML based spam filters that are constantly evolving. Even if you were to have access to their ML models, you wouldn't have enough data to effectively train that model. I'm not convinced by this argument. As others have stated, I've also had very bad experiences with Gmail's spam filtering. It has a very high false positive rate which often makes email go unnoticed until it's too late. I have a basic SpamAssassin setup for my self-hosted email and it works marvelously. It very rarely lets spam through and I haven't had a single false positive yet. > But for >99% of organisations, it just won't make any sense to run their own email. Perhaps, but the solution is decidedly not going to the monopolist. Hiring an email hosting company instead of doing it yourself is reasonable, though.
throwaway2048 8y ago
You can satisfy all of this, have an IP with an absolutely spotless reputation that you have personally controlled for years that has never sent spam, and still be arbitrarily shitlisted with zero recourse whatsoever.
- LeonM 8y ago
  IP addresses are really insignificant in spam detection because spammers can easily switch to a new IP address. Like with Google's search algorithm, the spam detection algorithms are secret and evolve constantly. So I'm not claiming I know how the algorithm works, but it wouldn't surprise me at all if IP addresses aren't even used anymore in Google's spam detection. So no, you are not 'shitlisted' as you called it. There are no lists, there are only algorithms.
  - throwaway2048 8y ago
    Whatever you care to call it is irrelevant, you jump through all the hoops, and your mail dosen't get delivered.

by LeonM 8y ago

gambler 8y ago

And with all this corporate shills chant that "Hosting your own things is easier than ever before".

LeonM 8y ago
I know it's going to be an unpopular opinion in this thread, but my opinion is that email is one of the things that most businesses shouldn't run themselves.
- feanaro 8y ago
  Why?
  - LeonM 8y ago
    Multiple reasons, but here are a few of them: - Email is hard to get right. There are many subtle configuration mistakes you can make, and it's pretty hard to detect those mistakes (hence one of the reasons why I started Mailhardener) - Your users will have high expectations. For email to be used effectively users expect email to work on many devices, have a webmail client, calendar integration, a good search function, good spam detection, infinite backups, ability to have huge attachments, etc, etc. - Your spam filter is never going to be as effective as that of the big email services. They have ML based spam filters that are constantly evolving. Even if you were to have access to their ML models, you wouldn't have enough data to effectively train that model. - Hosted email solutions are almost always more cost effective, regardless of the size of organisations. Running and maintaining an email service is labour intensive. Of course, there are also many legit use cases where you want to run your own email server. But for >99% of organisations, it just won't make any sense to run their own email.
    - feanaro 8y ago
      > - Email is hard to get right. There are many subtle configuration mistakes you can make, and it's pretty hard to detect those mistakes (hence one of the reasons why I started Mailhardener) It's not that hard if I've done it right. ;-) Jokes aside, yes, it's a bit overwhelming at the beginning, but I don't find it that cumbersome to keep it running. That is, it wouldn't be if Google was not actively being malicious in this regard. > - Your users will have high expectations. For email to be used effectively users expect email to work on many devices, have a webmail client, calendar integration, a good search function, good spam detection, infinite backups, ability to have huge attachments, etc, etc. From my experience, these aren't that hard to set up. If you're a corporation, your users are your employees. This is a bit different than if you were an email hosting company where users were paying for your product. In the former case, it is your responsibility to handle this well to serve the company's needs. It either serves you well or it does not. > - Your spam filter is never going to be as effective as that of the big email services. They have ML based spam filters that are constantly evolving. Even if you were to have access to their ML models, you wouldn't have enough data to effectively train that model. I'm not convinced by this argument. As others have stated, I've also had very bad experiences with Gmail's spam filtering. It has a very high false positive rate which often makes email go unnoticed until it's too late. I have a basic SpamAssassin setup for my self-hosted email and it works marvelously. It very rarely lets spam through and I haven't had a single false positive yet. > But for >99% of organisations, it just won't make any sense to run their own email. Perhaps, but the solution is decidedly not going to the monopolist. Hiring an email hosting company instead of doing it yourself is reasonable, though.

throwaway2048 8y ago

You can satisfy all of this, have an IP with an absolutely spotless reputation that you have personally controlled for years that has never sent spam, and still be arbitrarily shitlisted with zero recourse whatsoever.

LeonM 8y ago
IP addresses are really insignificant in spam detection because spammers can easily switch to a new IP address. Like with Google's search algorithm, the spam detection algorithms are secret and evolve constantly. So I'm not claiming I know how the algorithm works, but it wouldn't surprise me at all if IP addresses aren't even used anymore in Google's spam detection. So no, you are not 'shitlisted' as you called it. There are no lists, there are only algorithms.
- throwaway2048 8y ago
  Whatever you care to call it is irrelevant, you jump through all the hoops, and your mail dosen't get delivered.