4 ms·

I am my own provider. It's really not that hard to configure a mail server. Other peoples' providers, however... so I tried using PGP. It's somehow manages to b

by arpa 8y ago

I am my own provider. It's really not that hard to configure a mail server. Other peoples' providers, however... so I tried using PGP. It's somehow manages to be more difficult to use than configuring and running your own mailsystem which doesn't deliver straight to gmails' spam folder, but I digress; but then my emails are at mercy of the end devices: who knows if outlook doesn't send a plaintext email home as "telemetry"? Same thing about chrome with all the extensions for using PGP... I mean, security is hard as-is, but security of an ubiquitous thing, especially when people have been conditioned to undervalue security and privacy, is a lost cause. It's best to treat emails as being in the same category as phone calls, conversations in subway and sms'es - public.

selectodude 8y ago
Also all of your emails get kicked to spam when you host your own.
- mrb 8y ago
  You can perfectly "host your own email server" while sending outgoing mail through a smart host. No falsely-tagged-spam issue that way.
  - deleted 8y ago
    [deleted]
  - techsupporter 8y ago
    No, in my experience, selectodude is right. I've written this rant before but I think it's worth sharing repeatedly, so here goes again: The big players (Google, in particular...in case it's not clear, the bulk of this rant is leveled at Google; Microsoft is an offender on perhaps 25% of the level, Yahoo just doesn't seem to care) have ZERO interest in "playing nice" with small e-mail servers. My experience running my own e-mail for almost twenty years then finally throwing in the towel and migrating to Fastmail not only proves this for me, but is, according to other sysadmins I've talked to, quite common. I did everything right when hosting my e-mail. I did it for myself and maybe four others. The volume of mail going out of my server was so low, I could read the outbound mail.log for myself, and routinely did. Every domain had SPF, DKIM, and DMARC records. The outgoing single IPv4 and IPv6 addresses I used had perfectly-matching forward and reverse DNS lookups. I had a script that checked every blacklist I could find every 24 hours. My IP addresses were allocated from my (professional) colocation provider and SWIP'd to me with valid contact information. My IP addresses had been unchanged for seven years and had been stable for ten years prior to that change. My colo provider had a draconian non-spam policy. Nobody ran an e-mail list or anything. I've used the exact same domain for e-mail for over twenty years. Still, every couple of months, Google (sometimes Microsoft, but far and away most likely Google) would decide to simply stop delivering messages from my five users--remember, including me--to Google's users. The messages would just...disappear. Sometimes, Google would stop delivering the messages mid-conversation. Replies would be going back and forth but then my reply to a Gmail user would go to /dev/null while their messages to me would still arrive. Most of the time, the mail would go to spam (even with that little yellow tag meaning "verified sender" or some such) but 20% of the time the mails would just disappear. Worse, the destination would say "250 OK" to every e-mail I sent them so I had no way of knowing that the messages had been sent to the big bitbucket in the sky. I got annoyed. I signed up for Google's Postmaster Tools and Microsoft's similar offering. Microsoft would at least show me telemetry but it wouldn't do any good. Google? Google would proclaim that I wasn't sending them enough e-mail so I couldn't show up on the dashboard. Finally, after three years of fighting this, it came to a head when I was, at all places, at a boardgame meetup during one of these "blackout periods" and got introduced to a Google SRE who worked "adjacent" to Gmail. "Oh, hey, you should talk to techsupporter, he does e-mail too!" was the opening line. We talked, I ranted, and he actually said he'd try to help. He escalated internally and the word came down from on high on their SMTP team or some such: "domain [mydomain] has insufficient reputation." What did that mean? Apparently, their system--at least at that moment--thought my domain was "too new" and they tilted towards just binning the e-mail. I got red, saw stars, then gave Fastmail $390 for three years of service on three e-mail accounts and moved everything to them within five hours. That was the last straw, the final injustice. For me, the "decentralized Internet" died on that day. Why was I so pissed? Observe these dates from WHOIS: Domain Name: MYDOMAIN.ORG [...] Creation Date: 1997-03-10T05:00:00Z Domain Name: GOOGLE.COM [....] Creation Date: 1997-09-15T04:00:00Z
    - walrus01 8y ago
      I wish I could upvote you more than once. No matter how perfectly your mx environment is set up, the IP space internal reputation tools run by Google and MS for office365 will remain totally opaque. I have this issue occasionally with o365 hosted destination smtp, and I control my own ASN and ARIN IP space. Not once in the history of mankind has an open relay run on my IP space, nor has spam come from it.
    - mrb 8y ago
      I don't think you read or understood what I wrote. You were not routing through a smart host (third party SMTP relay.) This is your problem. There is a specific reason I recommend doing so: the reputation of the sending IP address almost always overrides the reputation of the sending domain. A smart host operated by a third party, such as Google: https://support.google.com/a/answer/2956491?hl=en https://support.google.com/a/answer/2956491?hl=en , would have solved your blackholing issues because it's the third party's job to maintain extremely good reputation of their IPs sending mail. Most importantly, they have high volume of legit emails, and volume is key to maintain good reputation. Finally, even if you have a low-volume domain, your emails will benefit of the good rep of the smart host, and thus would not be blackholed. I will repeat it again: if you host your mail server, you definitely should also use a smart host, and all the spam/blackholing issues disappear.
      - CyberShadow 8y ago
        Using Gmail as a smart host defeats the point as mentioned in the root comment of this thread: Google can read your email. Google also rewrites the "From:" address to your Google account's address, unless you jump through some hoops to add it to your account. This has become more difficult recently, and needs to be done for every address (which is an issue if you use single-purpose email addresses).
        mrb 8y ago
        «Using Gmail as a smart host defeats the point as mentioned in the root comment of this thread: Google can read your email.» Many things wrong with this sentence: • Don't like Google? Then use any of the hundreds of other smart host relays: Sendgrid, OutboundSMTP, etc • The smart host can't read your incoming mail (typically more sensitive than outgoing.) • Google, being a mail provider, can usually read your email anyway, because all your conversations with Gmail users end up in Google's hands. «Google also rewrites the "From:"» Google has multiple smart hosts offerings, but the one I linked to does NOT rewrite the From:.
- Erlich_Bachman 8y ago
  Not true. This might be happening if you are trying to host a server on your domestic internet connection, those IP ranges are usually banned in spam filters. But you can easily host your e-mail server on a real datacenter server or something similar, and you'll have almost no problems with spam filtering. (In fact many companies and even individuals already do this and have done this for long time.)
  - CyberShadow 8y ago
    > (In fact many companies and even individuals already do this and have done this for long time.) Yes, with mixed results. For one instance, see the trouble Linux people are having with getting messages delivered to Linus' Gmail inbox. I recall reading that some have needed to sign up for Gmail specifically just to get their patches to Linus. Plus, the 2015 occurrence with Gmail having a 20% false positive rate: https://plus.google.com/+LinusTorvalds/posts/DiG9qANf5PA https://plus.google.com/+LinusTorvalds/posts/DiG9qANf5PA
  - wastholm 8y ago
    Honest question: Where do people get this idea that they state so confidently, that it's "impossible" to run one's own mail server? Have they actually tried or are they just repeating what they've heard? Counter-anecdote: I have run my own mail server for several years. I use it for all my email communication, and also send out daily emails to a couple of thousand subscribers[1] who are on all sorts of email providers. The "canary accounts" I have set up on all the big providers all seem to have their subscription messages delivered straight to the inbox just fine, though I should probably monitor this better somehow. [1] the Aphorism of the Day, https://www.aphorismsgalore.com/ https://www.aphorismsgalore.com/
  - loup-vaillant 8y ago
    > Not true. This might be happening if you are trying to host a server on your domestic internet connection, those IP ranges are usually banned in spam filters. This is pretty much what "hosting your own" really means. Having SSH access to some virtual machine somewhere is much better than letting gmail read all your mail, but it's still isn't self hosting. You could still use the remote host as a mere relay, but it's still not ideal. Real self hosting means having your box at home, sending and receiving mail directly. And that is effectively impossible. Self hosting is dead. I'm not sure how we could resurrect it.
- Yetanfou 8y ago
  Not if you send your mail through a smart host (an outgoing SMTP proxy) which is in good standing. Most sane ISPs/IAPs require this anyway. I've hosted my own mail server since about 1996, moving from Sendmail on a 66MHz Pentium hanging off a cable internet connection to the current Exim on an Intel SS4200 on gigabit fiber - none of it high-powered hardware as such is not needed for the purpose. Mail sent from here arrives just fine, no problems there.
laumars 8y ago
> I am my own provider. It's really not that hard to configure a mail server. Oh yes it is. I mean for professional sysadmins it may not be, but try talking any normal person through buying a domain, getting an SSL certificate, attaching that at your MTA, configuring all those fiddly DNS records that prove you're emails are who you say you are. And all of that isn't even taking into account how needlessly painful the actual process of just installing the MTA is. And what do you have for show from all of that? Still no real assurance that half your mail won't end up in spam, no web front end (that step alone adds several additional layers of complexity), less guarantees regarding up time (since you no have a single server that you need to manage yourself), arguably worse security (since you're now asking a layman to harden a Linux box) and the whole thing comes with a heftier price tag than Gsuite. We're not talking about self hosting an IRC server or other service you can throw up in an hour and then forget about. Email is a frustrating sequence of a multitude of steps where even just one of them could render your set up worthless. Hence why even many of us sysadmins have long since given up self hosting email servers. It's just not worth the effort
- GordonS 8y ago
  I consider myself very technical, and know my way around Windows and Linux pretty well (I've used both for many years), but setting up a mail server is something I do very infrequently and find complicated to get right when I do so, especially when getting it working with TLS, DKIM, SPF and spam filtering.
- arpa 8y ago
  And this is how the world ends, not with a bang, but with a whimper.

by arpa 8y ago

selectodude 8y ago

Also all of your emails get kicked to spam when you host your own.

mrb 8y ago
You can perfectly "host your own email server" while sending outgoing mail through a smart host. No falsely-tagged-spam issue that way.
- deleted 8y ago
  [deleted]
- techsupporter 8y ago
  No, in my experience, selectodude is right. I've written this rant before but I think it's worth sharing repeatedly, so here goes again: The big players (Google, in particular...in case it's not clear, the bulk of this rant is leveled at Google; Microsoft is an offender on perhaps 25% of the level, Yahoo just doesn't seem to care) have ZERO interest in "playing nice" with small e-mail servers. My experience running my own e-mail for almost twenty years then finally throwing in the towel and migrating to Fastmail not only proves this for me, but is, according to other sysadmins I've talked to, quite common. I did everything right when hosting my e-mail. I did it for myself and maybe four others. The volume of mail going out of my server was so low, I could read the outbound mail.log for myself, and routinely did. Every domain had SPF, DKIM, and DMARC records. The outgoing single IPv4 and IPv6 addresses I used had perfectly-matching forward and reverse DNS lookups. I had a script that checked every blacklist I could find every 24 hours. My IP addresses were allocated from my (professional) colocation provider and SWIP'd to me with valid contact information. My IP addresses had been unchanged for seven years and had been stable for ten years prior to that change. My colo provider had a draconian non-spam policy. Nobody ran an e-mail list or anything. I've used the exact same domain for e-mail for over twenty years. Still, every couple of months, Google (sometimes Microsoft, but far and away most likely Google) would decide to simply stop delivering messages from my five users--remember, including me--to Google's users. The messages would just...disappear. Sometimes, Google would stop delivering the messages mid-conversation. Replies would be going back and forth but then my reply to a Gmail user would go to /dev/null while their messages to me would still arrive. Most of the time, the mail would go to spam (even with that little yellow tag meaning "verified sender" or some such) but 20% of the time the mails would just disappear. Worse, the destination would say "250 OK" to every e-mail I sent them so I had no way of knowing that the messages had been sent to the big bitbucket in the sky. I got annoyed. I signed up for Google's Postmaster Tools and Microsoft's similar offering. Microsoft would at least show me telemetry but it wouldn't do any good. Google? Google would proclaim that I wasn't sending them enough e-mail so I couldn't show up on the dashboard. Finally, after three years of fighting this, it came to a head when I was, at all places, at a boardgame meetup during one of these "blackout periods" and got introduced to a Google SRE who worked "adjacent" to Gmail. "Oh, hey, you should talk to techsupporter, he does e-mail too!" was the opening line. We talked, I ranted, and he actually said he'd try to help. He escalated internally and the word came down from on high on their SMTP team or some such: "domain [mydomain] has insufficient reputation." What did that mean? Apparently, their system--at least at that moment--thought my domain was "too new" and they tilted towards just binning the e-mail. I got red, saw stars, then gave Fastmail $390 for three years of service on three e-mail accounts and moved everything to them within five hours. That was the last straw, the final injustice. For me, the "decentralized Internet" died on that day. Why was I so pissed? Observe these dates from WHOIS: Domain Name: MYDOMAIN.ORG [...] Creation Date: 1997-03-10T05:00:00Z Domain Name: GOOGLE.COM [....] Creation Date: 1997-09-15T04:00:00Z
  - walrus01 8y ago
    I wish I could upvote you more than once. No matter how perfectly your mx environment is set up, the IP space internal reputation tools run by Google and MS for office365 will remain totally opaque. I have this issue occasionally with o365 hosted destination smtp, and I control my own ASN and ARIN IP space. Not once in the history of mankind has an open relay run on my IP space, nor has spam come from it.
  - mrb 8y ago
    I don't think you read or understood what I wrote. You were not routing through a smart host (third party SMTP relay.) This is your problem. There is a specific reason I recommend doing so: the reputation of the sending IP address almost always overrides the reputation of the sending domain. A smart host operated by a third party, such as Google: https://support.google.com/a/answer/2956491?hl=en https://support.google.com/a/answer/2956491?hl=en , would have solved your blackholing issues because it's the third party's job to maintain extremely good reputation of their IPs sending mail. Most importantly, they have high volume of legit emails, and volume is key to maintain good reputation. Finally, even if you have a low-volume domain, your emails will benefit of the good rep of the smart host, and thus would not be blackholed. I will repeat it again: if you host your mail server, you definitely should also use a smart host, and all the spam/blackholing issues disappear.
    - CyberShadow 8y ago
      Using Gmail as a smart host defeats the point as mentioned in the root comment of this thread: Google can read your email. Google also rewrites the "From:" address to your Google account's address, unless you jump through some hoops to add it to your account. This has become more difficult recently, and needs to be done for every address (which is an issue if you use single-purpose email addresses).
      - mrb 8y ago
        «Using Gmail as a smart host defeats the point as mentioned in the root comment of this thread: Google can read your email.» Many things wrong with this sentence: • Don't like Google? Then use any of the hundreds of other smart host relays: Sendgrid, OutboundSMTP, etc • The smart host can't read your incoming mail (typically more sensitive than outgoing.) • Google, being a mail provider, can usually read your email anyway, because all your conversations with Gmail users end up in Google's hands. «Google also rewrites the "From:"» Google has multiple smart hosts offerings, but the one I linked to does NOT rewrite the From:.
Erlich_Bachman 8y ago
Not true. This might be happening if you are trying to host a server on your domestic internet connection, those IP ranges are usually banned in spam filters. But you can easily host your e-mail server on a real datacenter server or something similar, and you'll have almost no problems with spam filtering. (In fact many companies and even individuals already do this and have done this for long time.)
- CyberShadow 8y ago
  > (In fact many companies and even individuals already do this and have done this for long time.) Yes, with mixed results. For one instance, see the trouble Linux people are having with getting messages delivered to Linus' Gmail inbox. I recall reading that some have needed to sign up for Gmail specifically just to get their patches to Linus. Plus, the 2015 occurrence with Gmail having a 20% false positive rate: https://plus.google.com/+LinusTorvalds/posts/DiG9qANf5PA https://plus.google.com/+LinusTorvalds/posts/DiG9qANf5PA
- wastholm 8y ago
  Honest question: Where do people get this idea that they state so confidently, that it's "impossible" to run one's own mail server? Have they actually tried or are they just repeating what they've heard? Counter-anecdote: I have run my own mail server for several years. I use it for all my email communication, and also send out daily emails to a couple of thousand subscribers[1] who are on all sorts of email providers. The "canary accounts" I have set up on all the big providers all seem to have their subscription messages delivered straight to the inbox just fine, though I should probably monitor this better somehow. [1] the Aphorism of the Day, https://www.aphorismsgalore.com/ https://www.aphorismsgalore.com/
- loup-vaillant 8y ago
  > Not true. This might be happening if you are trying to host a server on your domestic internet connection, those IP ranges are usually banned in spam filters. This is pretty much what "hosting your own" really means. Having SSH access to some virtual machine somewhere is much better than letting gmail read all your mail, but it's still isn't self hosting. You could still use the remote host as a mere relay, but it's still not ideal. Real self hosting means having your box at home, sending and receiving mail directly. And that is effectively impossible. Self hosting is dead. I'm not sure how we could resurrect it.
Yetanfou 8y ago
Not if you send your mail through a smart host (an outgoing SMTP proxy) which is in good standing. Most sane ISPs/IAPs require this anyway. I've hosted my own mail server since about 1996, moving from Sendmail on a 66MHz Pentium hanging off a cable internet connection to the current Exim on an Intel SS4200 on gigabit fiber - none of it high-powered hardware as such is not needed for the purpose. Mail sent from here arrives just fine, no problems there.

laumars 8y ago

> I am my own provider. It's really not that hard to configure a mail server. Oh yes it is. I mean for professional sysadmins it may not be, but try talking any normal person through buying a domain, getting an SSL certificate, attaching that at your MTA, configuring all those fiddly DNS records that prove you're emails are who you say you are. And all of that isn't even taking into account how needlessly painful the actual process of just installing the MTA is. And what do you have for show from all of that? Still no real assurance that half your mail won't end up in spam, no web front end (that step alone adds several additional layers of complexity), less guarantees regarding up time (since you no have a single server that you need to manage yourself), arguably worse security (since you're now asking a layman to harden a Linux box) and the whole thing comes with a heftier price tag than Gsuite. We're not talking about self hosting an IRC server or other service you can throw up in an hour and then forget about. Email is a frustrating sequence of a multitude of steps where even just one of them could render your set up worthless. Hence why even many of us sysadmins have long since given up self hosting email servers. It's just not worth the effort

GordonS 8y ago
I consider myself very technical, and know my way around Windows and Linux pretty well (I've used both for many years), but setting up a mail server is something I do very infrequently and find complicated to get right when I do so, especially when getting it working with TLS, DKIM, SPF and spam filtering.
arpa 8y ago
And this is how the world ends, not with a bang, but with a whimper.