5 ms·

Well I've gotten blocked from being delivered Outlook-hosted mail because my primary MX is on a Linode block that at times get listed with UCEProtect (level 3 e

Well I've gotten blocked from being delivered Outlook-hosted mail because my primary MX is on a Linode block that at times get listed with UCEProtect (level 3 even!!!) Why can't the big ones just check SPF and DKIM etc and let us responsible self-hosters do our thing

Avamander 4y ago
> Well I've gotten blocked from being delivered Outlook-hosted mail because my primary MX is on a Linode block that at times get listed with UCEProtect (level 3 even!!!) I really don't think Microsoft would ever touch that pile of shit, most likely your spammy neighbour got your neighbourhood blacklisted at both places simultaneously. > Why can't the big ones just check SPF and DKIM etc and let us responsible self-hosters do our thing Most of the spammers have valid SPF and DKIM. Quite a few have entire subnets of IP addresses to spew spam from. That's why a random IP and just those two aren't sufficient to just start blasting.
- mrb 4y ago
  «Most of the spammers have valid SPF and DKIM» What porbelm means is that when SPF and DKIM are valid, reputation check should be based on the sending domain, not on the sending IP. This way spammers still get thwarted (because of their domains with zero or low reputation) and legit senders can send from any IP (since their domain CAN build good reputation) Besides, it's also easier to track reputation of ~360M domains than billion of IPv6 and IPv4 ranges.
  - rlpb 4y ago
    That's great in theory. Unfortunately many legitimate sending domains are newly registered, so have no reputation. It's not possible to rely on age either: if requiring domains to be aged becomes commonplace, they're cheap enough that spammers would just register them and wait for a bit, maintaining a pipeline of them. So it becomes necessary to consider the reputation of whoever is enabling the sending of the email. The only way to do that is by IP. If a domain uses DKIM, has a good reputation but comes from a bad IP, then sure, you might trust it. I don't think that'll help much in practice though.
    - mrb 4y ago
      Everything you said applies to IP-reputation as well. There is nothing that makes domain-reputation inherently harder than IP-reputation. Consider this: every legit IP address once started with the problem of having zero reputation. How you build reputation from zero is simple: you start by being allowed to send only small amount of daily emails (rest goes to spam or is outright rejected by the recipient SMTP server). Then adjust reputation based on the usual factors (observe human recipients flagging emails as spam/not spam, watch for keywords like Viagra, etc, all the exact same stuff email providers already do). You gain or loose reputation over time, which affects how much you can send: a lot, or nothing.
      - rlpb 4y ago
        Some hosting providers have a very negative IP reputation. They seem to be unable to stop their users from sending out spam. They don't seem to react to abuse reports, let alone take action to stop abuse from their ASN generally. So I block their entire netblocks, IP-wise. I'm not aware of any legitimate emails being caught up in this. If somebody is unfortunately caught up, my response is that they should use a more reputable provider. According to your previous post, I shouldn't do that for DKIM/SPF -valid emails by IP, but I do. I hope this explains I do, and why purely domain-based reputation doesn't work.
        mrb 4y ago
        Big spammers own multiple ranges of IPs just as easily as they own multiple domains. When you block only 1 of their range, they can re-use their same domains to spam you from other ranges. If you blocked by domain, this would be effective across all their ranges of IPs. So, in your example, blocking by IP isn't necessarily superior to blocking by domain (and vice versa.) Once again: it's possible to have the tools and infrastructure manage domain reputation and do a job as good as (and arguably better than) IP reputation. Also to consider: IPs can be acquired by spammers for nearly $0 (eg: destroy+recreate VPS to get a new IP for free), whereas domain names have a fixed cost of ~$10 USD per year per domain. So if you have a system that manages domain reputation and has 1 million domain in the blacklist, you know for a fact that you are burning ~$10M/year of spammers' money.
soneil 4y ago
> Why can't the big ones just check SPF and DKIM etc and let us responsible self-hosters do our thing I self-host and frequently block entire ISPs. It's very common to get the same mail for days on end from a different IP, and often a different domain each time. Some months DO accounts for over 50% of my spam, for example. I'm sure there's more graceful ways to handle it if I was willing to put far too much of my own time into fighting a losing battle. but adding a REJECT line for a whole netblock takes seconds. I don't begrudge the big hosts for treating whole ISPs as cesspits as I do the very same thing.

Avamander 4y ago

> Well I've gotten blocked from being delivered Outlook-hosted mail because my primary MX is on a Linode block that at times get listed with UCEProtect (level 3 even!!!) I really don't think Microsoft would ever touch that pile of shit, most likely your spammy neighbour got your neighbourhood blacklisted at both places simultaneously. > Why can't the big ones just check SPF and DKIM etc and let us responsible self-hosters do our thing Most of the spammers have valid SPF and DKIM. Quite a few have entire subnets of IP addresses to spew spam from. That's why a random IP and just those two aren't sufficient to just start blasting.

mrb 4y ago
«Most of the spammers have valid SPF and DKIM» What porbelm means is that when SPF and DKIM are valid, reputation check should be based on the sending domain, not on the sending IP. This way spammers still get thwarted (because of their domains with zero or low reputation) and legit senders can send from any IP (since their domain CAN build good reputation) Besides, it's also easier to track reputation of ~360M domains than billion of IPv6 and IPv4 ranges.
- rlpb 4y ago
  That's great in theory. Unfortunately many legitimate sending domains are newly registered, so have no reputation. It's not possible to rely on age either: if requiring domains to be aged becomes commonplace, they're cheap enough that spammers would just register them and wait for a bit, maintaining a pipeline of them. So it becomes necessary to consider the reputation of whoever is enabling the sending of the email. The only way to do that is by IP. If a domain uses DKIM, has a good reputation but comes from a bad IP, then sure, you might trust it. I don't think that'll help much in practice though.
  - mrb 4y ago
    Everything you said applies to IP-reputation as well. There is nothing that makes domain-reputation inherently harder than IP-reputation. Consider this: every legit IP address once started with the problem of having zero reputation. How you build reputation from zero is simple: you start by being allowed to send only small amount of daily emails (rest goes to spam or is outright rejected by the recipient SMTP server). Then adjust reputation based on the usual factors (observe human recipients flagging emails as spam/not spam, watch for keywords like Viagra, etc, all the exact same stuff email providers already do). You gain or loose reputation over time, which affects how much you can send: a lot, or nothing.
    - rlpb 4y ago
      Some hosting providers have a very negative IP reputation. They seem to be unable to stop their users from sending out spam. They don't seem to react to abuse reports, let alone take action to stop abuse from their ASN generally. So I block their entire netblocks, IP-wise. I'm not aware of any legitimate emails being caught up in this. If somebody is unfortunately caught up, my response is that they should use a more reputable provider. According to your previous post, I shouldn't do that for DKIM/SPF -valid emails by IP, but I do. I hope this explains I do, and why purely domain-based reputation doesn't work.
      - mrb 4y ago
        Big spammers own multiple ranges of IPs just as easily as they own multiple domains. When you block only 1 of their range, they can re-use their same domains to spam you from other ranges. If you blocked by domain, this would be effective across all their ranges of IPs. So, in your example, blocking by IP isn't necessarily superior to blocking by domain (and vice versa.) Once again: it's possible to have the tools and infrastructure manage domain reputation and do a job as good as (and arguably better than) IP reputation. Also to consider: IPs can be acquired by spammers for nearly $0 (eg: destroy+recreate VPS to get a new IP for free), whereas domain names have a fixed cost of ~$10 USD per year per domain. So if you have a system that manages domain reputation and has 1 million domain in the blacklist, you know for a fact that you are burning ~$10M/year of spammers' money.

soneil 4y ago

> Why can't the big ones just check SPF and DKIM etc and let us responsible self-hosters do our thing I self-host and frequently block entire ISPs. It's very common to get the same mail for days on end from a different IP, and often a different domain each time. Some months DO accounts for over 50% of my spam, for example. I'm sure there's more graceful ways to handle it if I was willing to put far too much of my own time into fighting a losing battle. but adding a REJECT line for a whole netblock takes seconds. I don't begrudge the big hosts for treating whole ISPs as cesspits as I do the very same thing.