5 ms·

What kind of colossal idiot would block an email because it comes from a certain IP-block IF SPF and DKIM has been properly setup? Is this an urban legend that

by tjoff 9y ago

What kind of colossal idiot would block an email because it comes from a certain IP-block IF SPF and DKIM has been properly setup? Is this an urban legend that just keeps getting repeated?

jdietrich 9y ago
Statistically, a random mailserver on a random IP address is overwhelmingly likely to be sending spam. All the large mail providers use Bayesian filtering and are willing to tolerate a modest false-positive rate to ensure a low false-negative rate. A spam blocklist is not a court of law. As far as the mail providers are concerned, you're guilty until proven innocent. SPF and DKIM are no guarantee that you're not a spammer.
interfixus 9y ago
No, it's real. I gave it up a year ago, after hosting my own mail since forever. Exactly as stated: Hotmail (or whatever MS mail is called these days), GMail etc. are the main culprits. So yes, 'colossal' is the word.
- danieltillett 9y ago
  Yep I did the same. I ran my own email server off the same IP address for 15 years with zero problems, but I had to give up recently because email is not something that you can do on you own anymore. You now have to pay some somebody to vouch for you or else MS will send your emails to spam or even worse just randomly delete them.
  - EGreg 9y ago
    Something should eventually come along and replace email. Like bitcoin and other technologies, what was originally designed to be decentralized became centralized in practice.
    - mbrock 9y ago
      Bitcoin's history involves a plan for combating spam—that's why proof of work was invented. Hashcash, Bitcoin's PoW game, was intended for use as an email attachment to create a real cost to spam without any micropayments. "If you want to mail me, prove that you burned five seconds of CPU time."
      - kuczmama 9y ago
        That is really interesting. Do you know of any articles that go into more details?
  - reitanqild 9y ago
    Could be you had a customer/other recipient like my ex-boss: I'm fairly certain he used the spam button instead of delete on his Google apps for your domain (now gsuite I guess) account.
- gamache 9y ago
  Hosted my own email for 18 years, same experience, switched to pobox.com about two years ago. It really burns me up, but I need my emails to arrive where I sent them.
colejohnson66 9y ago
> E.g. your hosted email server does nothing wrong but some other bad actor on your ip block sends spam which then makes MS/Google block you because of "guilty by association". That’s why
- tjoff 9y ago
  No. That is not why, the reason must be something else. The only explanation that makes sense is deliberate sabotage to eradicate competition. But I'm not suggesting that, it is just that I can not think of any other reason. What is the motivation for the "guilty by association"? Because you can not verify who sent it. With SPF and DKIM you can - that is the whole point, you know beyond any doubt that the machine supposed to send that mail actually did send it. Now, if the person behind that machine is a spammer (or has been hijacked by a spammer) and sends out a lot of spam (and gets reported as such) then just nuke that sender. It is dead easy to do regardless of what IP it was sent from - each mail have identification credentials built in. It could not get any easier to do. The sender did all of the work for you by going through the extra effort of identifying itself.
  - DanBC 9y ago
    > What is the motivation for the "guilty by association"? One that has been given in the past is "Stop paying money to companies that host spammer websites and spammer email servers, or we'll block your email too" (I don't particularly agree with it, but it was common to see that argument on news.admin.net-abuse.email
  - jazoom 9y ago
    Exactly. I also find this ridiculous.
  - bradknowles 9y ago
    Yeah, but they don’t identify themselves correctly. They lie. Sometimes they claim to be real people with real e-mail addresses that they don’t actually own, and maybe that real person is you — in which case it is your reputation that gets permanently dinged. Sometimes they claim to be fake people with fake e-mail addresses that don’t actually exist, in which case all you really know is the IP address they used to send the message. When they do this, they frequently arrange to get a large block of IP addresses they can use, and then cycle through them very quickly — maybe no more than five minutes per IP address. Anyone else who happens to be in that same range now gets their reputation dinged by association. There’s a lot of tricks spammers use to try to hide their activity. And lots of ways to try to detect and reject that on the other end. It’s a never-ending arms race, and on the Internet it is easy to hide and hard to take out miscreants without causing excessive collateral damage. And many big providers don’t care if they cause collateral damage.
nasalgoat 9y ago
My company's email server ends up on RBLs once in a while due to such nonsense, and I have a perfect SPL and DMARC score.
DanBC 9y ago
> What kind of colossal idiot would block an email because it comes from a certain IP-block IF SPF and DKIM has been properly setup? I don't know if anything like SPEWS is still around but they used to do large blocks. I still don't know if it was genuine attempt to reduce spam, or performance art, or god-tier trolling. https://en.wikipedia.org/wiki/Spam_Prevention_Early_Warning_System https://en.wikipedia.org/wiki/Spam_Prevention_Early_Warning_... https://web-beta.archive.org/web/20030609200813/http://www.spews.org/ https://web-beta.archive.org/web/20030609200813/http://www.s...
KyeRussell 9y ago
No, in my extremely limited experience with email deliverability, this happens. I'd imagine a person that actually does this stuff day-to-day sees it a lot more than I do. You can do "everything right" rolling your own email, but you are still relying on the receiving parties to do their job and implement sane spam prevention policies. This is far from what actually happens. I lead development on a project that sends a lot of (legitimate, user-requested, transactional) email to many mailboxes belonging to a certain government department. I used Amazon SES as they have a pretty good track record. The receiving mail servers were configured to use a few overzealous blacklists that were causing my email to be silently swallowed (and reported as delivered). This was based on an IP black-level ban. Simply Googling the names of these blacklists showed report after report after report of them being completely useless, with many seasoned sysadmins saying that nobody worth their salt should pay any attention to them. This was extremely hard to troubleshoot as this department was not a direct client and was of limited helpfulness. It became apparent that using SES's newly-launched dedicated IP functionality wouldn't have helped. I made full use of my AWS Business Support, the multiple people I talked to made it clear that SES put a great deal of effort into ensuring their IP ranges are "clean" - but there are some spam lists that do not play ball. The most important takeaway was that I was doing everything right. I was using a reputable mail provider who have very stringent spam prevention controls. I utilised SPF and DKIM. The content of my mail shouldn't have (and didn't) trigger a content-based spam detector. It was because the people on the other side weren't doing their job to an acceptable standard. When I was eventually able to open a channel of contact with the receiving department, they confirmed it was an IP range-based ban. The fact that I was doing everything in my power to do the right thing had no effect on the situation. I still wasn't getting mail to customers. Saying "darm, these sysadmins really aren't doing their job properly!" had no effect on the situation at all. We still live in the age of almost-but-not-quite-plug-and-play email solutions like Exchange that can be configured by people that probably shouldn't be configuring mail servers. For all I know, there was a list of blacklists on some Hosted Outlook control panel and the person in charge ticked all of them because more is better. As long as this is the reality I live in, I'm going to do all I can to play ball and succumb to the black magic. I can imagine my experience would've been a lot worse if I had been operating a mail server out of a dirtier IP range (generic cloud hosting provider, retail ISP customer IP range.etc). If I recall correctly, AWS severely limits sending mail from EC2 instances because they don't want their IP addresses / IP ranges to end up on mail blacklists.
dboreham 9y ago
There are more collosal idiots around than you allow for. Also, it isn't in said idiot's interest to deliver your email properly. They would rather fart around with machine learning and blame whatever happens on incomplete training data...

by tjoff 9y ago

What kind of colossal idiot would block an email because it comes from a certain IP-block IF SPF and DKIM has been properly setup? Is this an urban legend that just keeps getting repeated?

jdietrich 9y ago

Statistically, a random mailserver on a random IP address is overwhelmingly likely to be sending spam. All the large mail providers use Bayesian filtering and are willing to tolerate a modest false-positive rate to ensure a low false-negative rate. A spam blocklist is not a court of law. As far as the mail providers are concerned, you're guilty until proven innocent. SPF and DKIM are no guarantee that you're not a spammer.

interfixus 9y ago

No, it's real. I gave it up a year ago, after hosting my own mail since forever. Exactly as stated: Hotmail (or whatever MS mail is called these days), GMail etc. are the main culprits. So yes, 'colossal' is the word.

danieltillett 9y ago
Yep I did the same. I ran my own email server off the same IP address for 15 years with zero problems, but I had to give up recently because email is not something that you can do on you own anymore. You now have to pay some somebody to vouch for you or else MS will send your emails to spam or even worse just randomly delete them.
- EGreg 9y ago
  Something should eventually come along and replace email. Like bitcoin and other technologies, what was originally designed to be decentralized became centralized in practice.
  - mbrock 9y ago
    Bitcoin's history involves a plan for combating spam—that's why proof of work was invented. Hashcash, Bitcoin's PoW game, was intended for use as an email attachment to create a real cost to spam without any micropayments. "If you want to mail me, prove that you burned five seconds of CPU time."
    - kuczmama 9y ago
      That is really interesting. Do you know of any articles that go into more details?
- reitanqild 9y ago
  Could be you had a customer/other recipient like my ex-boss: I'm fairly certain he used the spam button instead of delete on his Google apps for your domain (now gsuite I guess) account.
gamache 9y ago
Hosted my own email for 18 years, same experience, switched to pobox.com about two years ago. It really burns me up, but I need my emails to arrive where I sent them.

colejohnson66 9y ago

> E.g. your hosted email server does nothing wrong but some other bad actor on your ip block sends spam which then makes MS/Google block you because of "guilty by association". That’s why

tjoff 9y ago
No. That is not why, the reason must be something else. The only explanation that makes sense is deliberate sabotage to eradicate competition. But I'm not suggesting that, it is just that I can not think of any other reason. What is the motivation for the "guilty by association"? Because you can not verify who sent it. With SPF and DKIM you can - that is the whole point, you know beyond any doubt that the machine supposed to send that mail actually did send it. Now, if the person behind that machine is a spammer (or has been hijacked by a spammer) and sends out a lot of spam (and gets reported as such) then just nuke that sender. It is dead easy to do regardless of what IP it was sent from - each mail have identification credentials built in. It could not get any easier to do. The sender did all of the work for you by going through the extra effort of identifying itself.
- DanBC 9y ago
  > What is the motivation for the "guilty by association"? One that has been given in the past is "Stop paying money to companies that host spammer websites and spammer email servers, or we'll block your email too" (I don't particularly agree with it, but it was common to see that argument on news.admin.net-abuse.email
- jazoom 9y ago
  Exactly. I also find this ridiculous.
- bradknowles 9y ago
  Yeah, but they don’t identify themselves correctly. They lie. Sometimes they claim to be real people with real e-mail addresses that they don’t actually own, and maybe that real person is you — in which case it is your reputation that gets permanently dinged. Sometimes they claim to be fake people with fake e-mail addresses that don’t actually exist, in which case all you really know is the IP address they used to send the message. When they do this, they frequently arrange to get a large block of IP addresses they can use, and then cycle through them very quickly — maybe no more than five minutes per IP address. Anyone else who happens to be in that same range now gets their reputation dinged by association. There’s a lot of tricks spammers use to try to hide their activity. And lots of ways to try to detect and reject that on the other end. It’s a never-ending arms race, and on the Internet it is easy to hide and hard to take out miscreants without causing excessive collateral damage. And many big providers don’t care if they cause collateral damage.

nasalgoat 9y ago

My company's email server ends up on RBLs once in a while due to such nonsense, and I have a perfect SPL and DMARC score.

DanBC 9y ago

> What kind of colossal idiot would block an email because it comes from a certain IP-block IF SPF and DKIM has been properly setup? I don't know if anything like SPEWS is still around but they used to do large blocks. I still don't know if it was genuine attempt to reduce spam, or performance art, or god-tier trolling. https://en.wikipedia.org/wiki/Spam_Prevention_Early_Warning_System https://en.wikipedia.org/wiki/Spam_Prevention_Early_Warning_... https://web-beta.archive.org/web/20030609200813/http://www.spews.org/ https://web-beta.archive.org/web/20030609200813/http://www.s...

KyeRussell 9y ago

No, in my extremely limited experience with email deliverability, this happens. I'd imagine a person that actually does this stuff day-to-day sees it a lot more than I do. You can do "everything right" rolling your own email, but you are still relying on the receiving parties to do their job and implement sane spam prevention policies. This is far from what actually happens. I lead development on a project that sends a lot of (legitimate, user-requested, transactional) email to many mailboxes belonging to a certain government department. I used Amazon SES as they have a pretty good track record. The receiving mail servers were configured to use a few overzealous blacklists that were causing my email to be silently swallowed (and reported as delivered). This was based on an IP black-level ban. Simply Googling the names of these blacklists showed report after report after report of them being completely useless, with many seasoned sysadmins saying that nobody worth their salt should pay any attention to them. This was extremely hard to troubleshoot as this department was not a direct client and was of limited helpfulness. It became apparent that using SES's newly-launched dedicated IP functionality wouldn't have helped. I made full use of my AWS Business Support, the multiple people I talked to made it clear that SES put a great deal of effort into ensuring their IP ranges are "clean" - but there are some spam lists that do not play ball. The most important takeaway was that I was doing everything right. I was using a reputable mail provider who have very stringent spam prevention controls. I utilised SPF and DKIM. The content of my mail shouldn't have (and didn't) trigger a content-based spam detector. It was because the people on the other side weren't doing their job to an acceptable standard. When I was eventually able to open a channel of contact with the receiving department, they confirmed it was an IP range-based ban. The fact that I was doing everything in my power to do the right thing had no effect on the situation. I still wasn't getting mail to customers. Saying "darm, these sysadmins really aren't doing their job properly!" had no effect on the situation at all. We still live in the age of almost-but-not-quite-plug-and-play email solutions like Exchange that can be configured by people that probably shouldn't be configuring mail servers. For all I know, there was a list of blacklists on some Hosted Outlook control panel and the person in charge ticked all of them because more is better. As long as this is the reality I live in, I'm going to do all I can to play ball and succumb to the black magic. I can imagine my experience would've been a lot worse if I had been operating a mail server out of a dirtier IP range (generic cloud hosting provider, retail ISP customer IP range.etc). If I recall correctly, AWS severely limits sending mail from EC2 instances because they don't want their IP addresses / IP ranges to end up on mail blacklists.

dboreham 9y ago

There are more collosal idiots around than you allow for. Also, it isn't in said idiot's interest to deliver your email properly. They would rather fart around with machine learning and blame whatever happens on incomplete training data...