6 ms·

If you are using external blocklists, you are perpetuating the problem here. Small mail operators end up on blocklists through no fault of their own, and it sou

If you are using external blocklists, you are perpetuating the problem here. Small mail operators end up on blocklists through no fault of their own, and it sounds like your server would reject them just like the big servers do.

EvanAnderson 4y ago
You don’t have to use blocklists to block messages. I use blocklists in my self-hosted setup but only for the purpose of adding header fields that my Bayesian anti-spam filter can use to classify messages. I don’t reject anything out-right, aside from attempted spoofs of the domains my server is authoritative for. Everything is received— it just may end up in an “Unsure” folder if it seems too shady for the filter to put in my inbox.
- jimmydorry 4y ago
  That sounds like a better solution and not at all what I would assume someone means when they say that they use blocklists to filter out spam.
koheripbal 4y ago
Not using block lists will inundate mail servers with spam. It's a necessary evil.
- jimmydorry 4y ago
  And for most people, using a large mail provider is the necessary evil. I don't see much difference between a small mail provider using block lists and a large mail provider using the same block lists + their own proprietary lists and metrics. Both result in other small mail providers getting blocked for arbitrary reasons with limited recourse.
  - friendzis 4y ago
    Using large mail provider is not necessary evil in my book. It is just so much more convenient than self hosting. Today, with ready-made containers and stuff we start having a fighting change, but the moat is possibly too big to fight with email falling out of favor
- jjav 4y ago
  I don't use any kind of block list on my mail server, as I find the concept to be fundamentally flawed. I accept all incoming email as long as basic sanity on the connection is met and then apply bayesian filtering after the email has been accepted. I get just about no spam at all (<10 per month, maybe).
  - whiplash451 4y ago
    One spam every three days is insanely high IMO (compared to what you will get on gmail)
    - jjav 4y ago
      > One spam every three days is insanely high IMO (compared to what you will get on gmail) That's funny since on gmail I get tons more spam than on any of my self-hosted domain addresses. What's worse, I see emails in gmail misfiled as spam when they are not, which is far worse. I don't ever see that happen in my self-hosted system.
    - olau 4y ago
      I get > 1 spam per day on gmail.com. But I don't find that burdensome at all, in fact I'd prefer that gmail let more stuff through to my spam folder instead of swallowing things that it misclassifies, because I have seen several instances of lost legitimate email, something I still don't understand how is deemed acceptable to the people who wrote it. But I need to vote with my feet instead of complaining.
  - zimpenfish 4y ago
    I use block lists, fail2ban, gray listing, most of rspamd's widgets, and Bayesian filtering. Still getting 10s of spams a day just to my accounts.
    - dvdkon 4y ago
      Any guess on why people's experiences with spam are so diverse? I self-host and get fewer than 10 true spam emails per year (not counting marketing newsletters).
- GTP 4y ago
  I think that gray listing is a great idea to filter out spam. Here's how it works: the very first time an email server get an email from a new IP it answers with a "temporary unavaiable" error. According to specs, a server should retry to deliver the email after a while, so a legitimate server will retry and it's IP will be put in a "withelist" (here in quotes because you can still do further processing of the emails to determine if it's good or bad). But a spammer will most likely not retry to send it, as their goal is to quickly send a big amount of mails. I didn't try this personally as I'm not hosting my own emails (I tried but gave up soon), but I heard it works very well, no third party blocklist needed.
  - zimpenfish 4y ago
    > I heard [gray listing] works very well Mostly. But then you get the "click the link in the email within 10 minutes" problem. There's also a non-zero number of "our mail didn't get through first attempt, oh well, give up" people. From running GL on my servers over a couple of years, it mildly cut down spam (on top of blocklists and fail2ban) but I'm now wavering over whether it's worth the hassle.
    - necovek 4y ago
      It cut spam tremendeously for me, but that's on an email address I've used and published openly since early 2000s. Still, I've given up on it since plenty of email senders are not standard-abiding (they fail to retry), and I've kept losing email. I only caved in in the last 12 months after 15+ years of doing graylisting.
  - iggldiggl 4y ago
    The trade off is that even legitimate mail that you are actively expecting (like an account confirmation or password reset mail) will be delayed by however long the sending server's retry interval is. You also either need to apply the greylisting to some larger IP range (rspamd e.g. apparently uses /19 by default for IPv4) or otherwise specially handle some of the bigger mail providers, because some of them rotate through their servers between retries, so you could be in for a quite a long wait if you do per-individual-IP greylisting. The biggest culprit I noticed this with was Amazon SES – a former mail provider of mine used per-individual-IP, non-configurable greylisting, and any mail sent through Amazon (which isn't just Amazon itself – quite a few companies are using Amazon SES for transactional mail and suchlike) would consequently almost always arrive several hours late (however randomly long it would take Amazon to finally re-use an IP during a subsequent retry attempt). Even more infuriating, my mail provider's support would then claim that it wasn't their fault and they didn't know anything about any supposed greylisting.
- jgsaau 4y ago
  What we found 20+ years ago when we started to provide a very poor service to attempted spam delivery was that the spammers realised our servers were very 'slow' and stopped the attempts. We used to get 1000s of new IPV4 tuples / day, now we get < 10. If you put the inbound SMTP sessions to sleep this is wrecking the spammers bus. model, they need to deliver 1,000,000,000+ / day. Don't u fell sorry 4 them. The only time we don't tuck them in 4 a snooze is if they are in a 3rd party BL !!!
Godel_unicode 4y ago
This is pure victim-blaming. The problem is that bulletproof hosting exists, not that people are sick of spam.
- jimmydorry 4y ago
  I don't follow. I am pointing out that using the same kinds of lists that the big providers use will result in smaller providers being arbitrarily blocked with little recourse. Do you disagree with that assessment?
  - deleted 4y ago
    [deleted]
  - Godel_unicode 4y ago
    First, yes. They (the small providers) have recourse, it’s just annoying. They voluntarily signed up for that annoyance because of their ideology. But much more importantly, that question is orthogonal to my argument. Using blocklists is a good way to cut down on spam, the fact that it might block some trivial percentage of people who for ideological reasons might or might not be on those lists isn’t the receivers problem. You want this to not be the case? Contribute meaningfully to solving the problem in a more effective way. Don’t blame the people who just don’t want the spam.
    - vinay_ys 4y ago
      > ideological reasons might or might not be on those lists What's the ideological reasons here?
      - 1337shadow 4y ago
        Apparently people who want to self host email instead of paying some big tech
    - jimmydorry 4y ago
      The small providers typically don't have recourse if they approach a blocklist maintainer and ask to be taken off a list. That "trivial percentage of people" make up a substantial portion of service providers (for every Google there are many <100 user mail servers), which is what this thread was about. I pointed out that adopting blocklists just makes it harder to operate a mail server as a small provider, and nobody in this thread appears to disagree with that assessment. They instead seem to take issue with the tone of the message.

EvanAnderson 4y ago

You don’t have to use blocklists to block messages. I use blocklists in my self-hosted setup but only for the purpose of adding header fields that my Bayesian anti-spam filter can use to classify messages. I don’t reject anything out-right, aside from attempted spoofs of the domains my server is authoritative for. Everything is received— it just may end up in an “Unsure” folder if it seems too shady for the filter to put in my inbox.

jimmydorry 4y ago
That sounds like a better solution and not at all what I would assume someone means when they say that they use blocklists to filter out spam.

koheripbal 4y ago

Not using block lists will inundate mail servers with spam. It's a necessary evil.

jimmydorry 4y ago
And for most people, using a large mail provider is the necessary evil. I don't see much difference between a small mail provider using block lists and a large mail provider using the same block lists + their own proprietary lists and metrics. Both result in other small mail providers getting blocked for arbitrary reasons with limited recourse.
- friendzis 4y ago
  Using large mail provider is not necessary evil in my book. It is just so much more convenient than self hosting. Today, with ready-made containers and stuff we start having a fighting change, but the moat is possibly too big to fight with email falling out of favor
jjav 4y ago
I don't use any kind of block list on my mail server, as I find the concept to be fundamentally flawed. I accept all incoming email as long as basic sanity on the connection is met and then apply bayesian filtering after the email has been accepted. I get just about no spam at all (<10 per month, maybe).
- whiplash451 4y ago
  One spam every three days is insanely high IMO (compared to what you will get on gmail)
  - jjav 4y ago
    > One spam every three days is insanely high IMO (compared to what you will get on gmail) That's funny since on gmail I get tons more spam than on any of my self-hosted domain addresses. What's worse, I see emails in gmail misfiled as spam when they are not, which is far worse. I don't ever see that happen in my self-hosted system.
  - olau 4y ago
    I get > 1 spam per day on gmail.com. But I don't find that burdensome at all, in fact I'd prefer that gmail let more stuff through to my spam folder instead of swallowing things that it misclassifies, because I have seen several instances of lost legitimate email, something I still don't understand how is deemed acceptable to the people who wrote it. But I need to vote with my feet instead of complaining.
- zimpenfish 4y ago
  I use block lists, fail2ban, gray listing, most of rspamd's widgets, and Bayesian filtering. Still getting 10s of spams a day just to my accounts.
  - dvdkon 4y ago
    Any guess on why people's experiences with spam are so diverse? I self-host and get fewer than 10 true spam emails per year (not counting marketing newsletters).
GTP 4y ago
I think that gray listing is a great idea to filter out spam. Here's how it works: the very first time an email server get an email from a new IP it answers with a "temporary unavaiable" error. According to specs, a server should retry to deliver the email after a while, so a legitimate server will retry and it's IP will be put in a "withelist" (here in quotes because you can still do further processing of the emails to determine if it's good or bad). But a spammer will most likely not retry to send it, as their goal is to quickly send a big amount of mails. I didn't try this personally as I'm not hosting my own emails (I tried but gave up soon), but I heard it works very well, no third party blocklist needed.
- zimpenfish 4y ago
  > I heard [gray listing] works very well Mostly. But then you get the "click the link in the email within 10 minutes" problem. There's also a non-zero number of "our mail didn't get through first attempt, oh well, give up" people. From running GL on my servers over a couple of years, it mildly cut down spam (on top of blocklists and fail2ban) but I'm now wavering over whether it's worth the hassle.
  - necovek 4y ago
    It cut spam tremendeously for me, but that's on an email address I've used and published openly since early 2000s. Still, I've given up on it since plenty of email senders are not standard-abiding (they fail to retry), and I've kept losing email. I only caved in in the last 12 months after 15+ years of doing graylisting.
- iggldiggl 4y ago
  The trade off is that even legitimate mail that you are actively expecting (like an account confirmation or password reset mail) will be delayed by however long the sending server's retry interval is. You also either need to apply the greylisting to some larger IP range (rspamd e.g. apparently uses /19 by default for IPv4) or otherwise specially handle some of the bigger mail providers, because some of them rotate through their servers between retries, so you could be in for a quite a long wait if you do per-individual-IP greylisting. The biggest culprit I noticed this with was Amazon SES – a former mail provider of mine used per-individual-IP, non-configurable greylisting, and any mail sent through Amazon (which isn't just Amazon itself – quite a few companies are using Amazon SES for transactional mail and suchlike) would consequently almost always arrive several hours late (however randomly long it would take Amazon to finally re-use an IP during a subsequent retry attempt). Even more infuriating, my mail provider's support would then claim that it wasn't their fault and they didn't know anything about any supposed greylisting.
jgsaau 4y ago
What we found 20+ years ago when we started to provide a very poor service to attempted spam delivery was that the spammers realised our servers were very 'slow' and stopped the attempts. We used to get 1000s of new IPV4 tuples / day, now we get < 10. If you put the inbound SMTP sessions to sleep this is wrecking the spammers bus. model, they need to deliver 1,000,000,000+ / day. Don't u fell sorry 4 them. The only time we don't tuck them in 4 a snooze is if they are in a 3rd party BL !!!

Godel_unicode 4y ago

This is pure victim-blaming. The problem is that bulletproof hosting exists, not that people are sick of spam.

jimmydorry 4y ago
I don't follow. I am pointing out that using the same kinds of lists that the big providers use will result in smaller providers being arbitrarily blocked with little recourse. Do you disagree with that assessment?
- deleted 4y ago
  [deleted]
- Godel_unicode 4y ago
  First, yes. They (the small providers) have recourse, it’s just annoying. They voluntarily signed up for that annoyance because of their ideology. But much more importantly, that question is orthogonal to my argument. Using blocklists is a good way to cut down on spam, the fact that it might block some trivial percentage of people who for ideological reasons might or might not be on those lists isn’t the receivers problem. You want this to not be the case? Contribute meaningfully to solving the problem in a more effective way. Don’t blame the people who just don’t want the spam.
  - vinay_ys 4y ago
    > ideological reasons might or might not be on those lists What's the ideological reasons here?
    - 1337shadow 4y ago
      Apparently people who want to self host email instead of paying some big tech
  - jimmydorry 4y ago
    The small providers typically don't have recourse if they approach a blocklist maintainer and ask to be taken off a list. That "trivial percentage of people" make up a substantial portion of service providers (for every Google there are many <100 user mail servers), which is what this thread was about. I pointed out that adopting blocklists just makes it harder to operate a mail server as a small provider, and nobody in this thread appears to disagree with that assessment. They instead seem to take issue with the tone of the message.