7 ms·

Break free from traditional email - Host your own server

69 points by kuczmama 9y ago

deleted 9y ago
[deleted]
fiatpandas 9y ago
I wrote my own receive-only Haraka-based server. The web UI is pretty simple: x-y matrix of small blocks representing received emails with basic info (subject and from address). A column is the account. Clicking an email block brings up full text or html mime content (not rendered). It’s setup to receive everything that’s sent to it, which means I occasionally have to delete rando spam. TLS is setup too. But it’s an interesting system because you can keep tabs on what exactly you are receiving per service (e.g. using instagram@mydomain.com), and maybe one day will tip me off to services giving away email addresses. Otherwise I’m using gmail for personal sending.
- extra88 9y ago
  I've been signing up on sites using unique addresses for each one for about 10 years. I've had two addresses that eventually started receiving spam, one for a small local business and one for Adobe. I'm sure both were due to hacks of those sites, not the customer addresses being sold.
clemenspw 9y ago
https://github.com/mailcow/mailcow-dockerized https://github.com/mailcow/mailcow-dockerized
kuczmama 9y ago
I wrote this article because I couldn't really find a good email tutorial that included DNS setup. I hope this helps you out. It's 2017 and it's surprisingly difficult to set up an email server, but hopefully this will make it a little bit easier. Please let me know if you need any help or you think I missed something major :D
- buovjaga 9y ago
  Thanks for writing it. Do you intend to include a reverse DNS record for IPv6? I remember doing it once and the syntax was mind-bending. Here is a generator: http://rdns6.com/hostRecord http://rdns6.com/hostRecord
  - kuczmama 9y ago
    Thank you very much! I never actually learned how to set a reverse DNS record for IPv6 because because vultr doesn't use IPv6 by default. I think that it might be a good idea to look into though.
- interfixus 9y ago
  Alas, you haven't extended the self-hosting idea to your blogging. You publish on Blogger, which throws out some garbled js, and otherwise refuses to show up on my screen.
  - kuczmama 9y ago
    Yeah that's true. I set up blogger a few years ago and I just haven't gotten around to self hosting it yet. I have a todo list of things I want to switch over to.
MrPatan 9y ago
This is very interesting. Recently I realized that if tomorrow the big G decides to take over or close my email address I lose access to all my online accounts. Step one to freedom was to use my own domain, redirected to my old account, but I'm seriously thinking about doing it all myself. But keeping a whole server up to date, secure, etc.... that's a full time job. Is there a good solution?
- qznc 9y ago
  I switched to mailbox.org where I'm a paying customer. That feels better than being a ad target at Gmail.
  - chrisper 9y ago
    >That feels better than being a ad target at Gmail. I believe they stopped this recently. But you could have always become a paying customer of Google.
    - duckMuppet 9y ago
      They stopped scanning for purposes of targeted advertising. However, they still scan and read email. People can't be inconvenienced. I tend to think this is worse than scanning for ads. I also tend to think anyone concerned about privacy moved from Google long ago. After all, its terrifying when you allow a company to determine if perhaps your received/sent links are fake, or malware, maybe they should report that you like certain sites, or have a flight.. People lived the idea of the President maxing out his power with executive orders, until they didn't.. Life is ask about tradeoffs i suppose.. http://variety.com/2017/digital/news/google-gmail-ads-emails-1202477321/ http://variety.com/2017/digital/news/google-gmail-ads-emails...
- innocenat 9y ago
  I have been down that route before. My realization come when I was blocked out of my main GMail account for ~24 hours. I have consider setting up my own email server, but in the end I just use FastMail (paid plan).
jasode 9y ago
Tutorials for setting up a personal email server are fine for educational purposes -- or -- if you're primarily using it to receive emails. On the other hand, if you absolutely depend on the ability to send emails such that your recipients reliably get them, hosting your own email server is extremely tricky. One could carefully go through the checklist of SPF, DKIM, ip blacklists, etc and emails will still be rejected by MS Hotmail, GMail, Yahoo, etc. Those giants are "black boxes" when it comes to their heuristics for rejecting incoming mail as spam. E.g. your hosted email server does nothing wrong but some other bad actor on your ip block sends spam which then makes MS/Google block you because of "guilty by association". Trying to debug your "sender reputation" is not easy. In the 1990s, I hosted my own home email server over ISDN lines. These days, I have a million other things I'd rather do than babysit a personal email server with software updates, SpamAssassin lists, etc. I get the whole decentralized ethos but it's just not worth the effort for email servers.
- tjoff 9y ago
  What kind of colossal idiot would block an email because it comes from a certain IP-block IF SPF and DKIM has been properly setup? Is this an urban legend that just keeps getting repeated?
  - jdietrich 9y ago
    Statistically, a random mailserver on a random IP address is overwhelmingly likely to be sending spam. All the large mail providers use Bayesian filtering and are willing to tolerate a modest false-positive rate to ensure a low false-negative rate. A spam blocklist is not a court of law. As far as the mail providers are concerned, you're guilty until proven innocent. SPF and DKIM are no guarantee that you're not a spammer.
  - interfixus 9y ago
    No, it's real. I gave it up a year ago, after hosting my own mail since forever. Exactly as stated: Hotmail (or whatever MS mail is called these days), GMail etc. are the main culprits. So yes, 'colossal' is the word.
fredsted 9y ago
Cool article. I personally use Virtualmin for this, it's very easy to set up, you can manage hosting and mail accounts for a bunch of different domains, easily do backups via S3, easy to set up SPF and DKIM, has a good web admin interface, easy to set up automatic updates, etc. By the way, here's the obligatory HN-style critic of the format, not the content: the blog template in this is a little annoying, it fiddles with scrolling by making it really slow on Safari on Mac, and Reader mode doesn't work.
- kuczmama 9y ago
  Virtualmin looks pretty cool, that looks like a great option. Yeah thanks for the feedback, I need to get around to self-hosting my blog as well. I set up blogger a few years ago and I never got around to changing it.
azr79 9y ago
Pro tip: don’t do this, use a custom domain name if you want, but never host your own email server.
- deleted 9y ago
  [deleted]
cjsuk 9y ago
Hell no. The moment Yahoo, Microsoft or google decide that they don’t like you, you’re SOL. Yahoo are the worst. If you try and deliver to them you get a deferral with an error message in your log with a URL. Then you have to open the link in the URL and fill in a form. They don’t have to accept the form and they ignore you for 3 months if it goes wrong. This happens even if you’re not on an RBL and have set up DKIM and SPF properly. Edit: you want to see the trouble we had to go to so we could run an SMTP server in AWS for outbound/abuse address inbound only and get that talking to Office 365 for internal use only. Two days of hell.
- stephengillie 9y ago
  I've worked for a real estate webhost and a major cloud provider/isp. Both had nightmarish problems simply ensuring email delivery. It did not help that the real estate "bulk mail newsletters" were legitimate spam. At the ISP, we offered an open internal relay to customers for some reason, and thus were again legitimate spammers. MS has 2 layers of spam blocking - one for Outlook.com, and a much stricter invisible layer for O365 with no support team.
  - cjsuk 9y ago
    Yeah I wouldn’t want to do it now. I worked for a major pollster about 10 years ago and they had monumental problems back then just sending invites out that people had actually signed up for.
- jankins 9y ago
  I've been self-hosting for the past 4 years and this hasn't been my experience. Things like fixing DKIM/SPF eat an hour or two occasionally, but I've never encountered an issue where the time investment overtook the learning + other benefits - it's been hands-off 364 days of the year. I've only seen one blocked send happen -- blocked by my grandma's @att.net account. Since it happens so infrequently and nobody uses @att.net, I just re-sent from a Hotmail account instead. No issues with the other major players. But for my use-case it's easy to mitigate and if the problem persists I can invest more time in it, but one recipient blocking me in 4 years isn't bad. It's the only way to have ownership, which is is one of the benefits I really like - Google, Yahoo, etc. still get pieces of my personal email history because nobody else self-hosts or uses PGP, which is disappointing, but I prefer it over handing one player ownership the full history. BTW, I'm running it on the same 512MB DigitalOcean droplet that I use to host my static sites (personal website, small product sites, etc), so it's basically free since I'd need to host those things anyway, which is nice. Needs some swap though. Edit: Not saying these points are invalid. They're certainly valid, a service like Gmail _will_ be more reliable and easier. If you're blocked for some reason or have any other email probs, there's nobody else to fix it besides you.
tramtrist 9y ago
I moved from gmail to https://mailinabox.email/ https://mailinabox.email/ a few years ago and never looked back...
ryan-c 9y ago
I have been running my own mail server for a number of years. I recently was forced to move to a new IP. I had to get my server listed on DNSWL, then send in manual requests to Microsoft, AOL, and Yahoo to not block my email because they all seem to deny by default. I also ran through a couple of IPs with my provider before I found one that wasn't on any meaningful RBLs (my IP's on SpamGrouper, but that list is clearly run by an insane person and nobody seems to use it). As other people point out, some mail providers are just complete assholes and will blackhole your mail with no indication to the sender or recipient that it happened.
cedivad 9y ago
I've been hosting my own email server for 10 years until a couple of weeks ago, when I switched everything to gsuite. During the sign up process there is a step claiming "email just got awesome". That's as close of a description of my experience so far as you can possibly get. Email just got awesome.
jstimpfle 9y ago
Just another data point: I've been running my own mail for more than 6 years and I've never had a problem with receivers. Hosted on a cheap Strato VServer in Germany, I've never cared for the technical details, could not explain right now what DKIM and SPF are (and they're not configured), and only recently installed a self-signed SSL certificate in my Exim configuration to be able to use it with a Desktop client for submission (pretty sure outbound traffic still runs unencrypted).
prashnts 9y ago
Echoing to other's thoughts that hosting a server on your own would probably flag you in spam and just be a very tiresome process. A reasonably good and cheap service I could not recommend enough is https://www.migadu.com https://www.migadu.com . They allow you to use unlimited email domains, storage, addresses with the only limit being on total daily outgoing emails. The mini plan allows 100 outgoing emails a day which is more than sufficient for most of my purposes.
cygned 9y ago
https://workaround.org/ispmail https://workaround.org/ispmail We used this to setup a multi-domain email server. Works reliably and fast at both sending and receiving mail.
apple4ever 9y ago
Ugh a virtual server running Docker? Dumb. Just install it on the base OS without the overhead.
throwaway2016a 9y ago
You couldn't pay me enough to host my own mail... To me it falls under the same category as assembling my own computer. I can do it but to me it's not worth the trouble. Between DKIM, DMARC, and SPF, security, backup strategy, the fact if you are an open relay for even a day a bot net will find you and get your IP blacklisted for life... or an ISP could just blacklist you because they saw other spam from your same subnet on a shared hosting provider... Granted this article covers a lot of that (it talks about DKIM, DMARC, and SPF) I'm still counting this as one of the things I outsource.
oliwarner 9y ago
Friends don't let friends host their own email. It's just not worth the hassle these days unless you're managing hundreds of accounts and it's a full time endeavour. Even at the hundreds level, I'm sure there are people who'd rather outsource. It's not just email that these guys (Gmail, Outlook, Fastmail, etc) provide.
luord 9y ago
I wasn't aware of mail-tester. Yet another tool for checking my domains. Thanks.

69 points by kuczmama 9y ago

deleted 9y ago

[deleted]

I wrote my own receive-only Haraka-based server. The web UI is pretty simple: x-y matrix of small blocks representing received emails with basic info (subject and from address). A column is the account. Clicking an email block brings up full text or html mime content (not rendered). It’s setup to receive everything that’s sent to it, which means I occasionally have to delete rando spam. TLS is setup too. But it’s an interesting system because you can keep tabs on what exactly you are receiving per service (e.g. using instagram@mydomain.com), and maybe one day will tip me off to services giving away email addresses. Otherwise I’m using gmail for personal sending.

extra88 9y ago
I've been signing up on sites using unique addresses for each one for about 10 years. I've had two addresses that eventually started receiving spam, one for a small local business and one for Adobe. I'm sure both were due to hacks of those sites, not the customer addresses being sold.

clemenspw 9y ago

https://github.com/mailcow/mailcow-dockerized https://github.com/mailcow/mailcow-dockerized

kuczmama 9y ago

I wrote this article because I couldn't really find a good email tutorial that included DNS setup. I hope this helps you out. It's 2017 and it's surprisingly difficult to set up an email server, but hopefully this will make it a little bit easier. Please let me know if you need any help or you think I missed something major :D

buovjaga 9y ago
Thanks for writing it. Do you intend to include a reverse DNS record for IPv6? I remember doing it once and the syntax was mind-bending. Here is a generator: http://rdns6.com/hostRecord http://rdns6.com/hostRecord
- kuczmama 9y ago
  Thank you very much! I never actually learned how to set a reverse DNS record for IPv6 because because vultr doesn't use IPv6 by default. I think that it might be a good idea to look into though.
interfixus 9y ago
Alas, you haven't extended the self-hosting idea to your blogging. You publish on Blogger, which throws out some garbled js, and otherwise refuses to show up on my screen.
- kuczmama 9y ago
  Yeah that's true. I set up blogger a few years ago and I just haven't gotten around to self hosting it yet. I have a todo list of things I want to switch over to.

MrPatan 9y ago

This is very interesting. Recently I realized that if tomorrow the big G decides to take over or close my email address I lose access to all my online accounts. Step one to freedom was to use my own domain, redirected to my old account, but I'm seriously thinking about doing it all myself. But keeping a whole server up to date, secure, etc.... that's a full time job. Is there a good solution?

qznc 9y ago
I switched to mailbox.org where I'm a paying customer. That feels better than being a ad target at Gmail.
- chrisper 9y ago
  >That feels better than being a ad target at Gmail. I believe they stopped this recently. But you could have always become a paying customer of Google.
  - duckMuppet 9y ago
    They stopped scanning for purposes of targeted advertising. However, they still scan and read email. People can't be inconvenienced. I tend to think this is worse than scanning for ads. I also tend to think anyone concerned about privacy moved from Google long ago. After all, its terrifying when you allow a company to determine if perhaps your received/sent links are fake, or malware, maybe they should report that you like certain sites, or have a flight.. People lived the idea of the President maxing out his power with executive orders, until they didn't.. Life is ask about tradeoffs i suppose.. http://variety.com/2017/digital/news/google-gmail-ads-emails-1202477321/ http://variety.com/2017/digital/news/google-gmail-ads-emails...
innocenat 9y ago
I have been down that route before. My realization come when I was blocked out of my main GMail account for ~24 hours. I have consider setting up my own email server, but in the end I just use FastMail (paid plan).

jasode 9y ago

Tutorials for setting up a personal email server are fine for educational purposes -- or -- if you're primarily using it to receive emails. On the other hand, if you absolutely depend on the ability to send emails such that your recipients reliably get them, hosting your own email server is extremely tricky. One could carefully go through the checklist of SPF, DKIM, ip blacklists, etc and emails will still be rejected by MS Hotmail, GMail, Yahoo, etc. Those giants are "black boxes" when it comes to their heuristics for rejecting incoming mail as spam. E.g. your hosted email server does nothing wrong but some other bad actor on your ip block sends spam which then makes MS/Google block you because of "guilty by association". Trying to debug your "sender reputation" is not easy. In the 1990s, I hosted my own home email server over ISDN lines. These days, I have a million other things I'd rather do than babysit a personal email server with software updates, SpamAssassin lists, etc. I get the whole decentralized ethos but it's just not worth the effort for email servers.

tjoff 9y ago
What kind of colossal idiot would block an email because it comes from a certain IP-block IF SPF and DKIM has been properly setup? Is this an urban legend that just keeps getting repeated?
- jdietrich 9y ago
  Statistically, a random mailserver on a random IP address is overwhelmingly likely to be sending spam. All the large mail providers use Bayesian filtering and are willing to tolerate a modest false-positive rate to ensure a low false-negative rate. A spam blocklist is not a court of law. As far as the mail providers are concerned, you're guilty until proven innocent. SPF and DKIM are no guarantee that you're not a spammer.
- interfixus 9y ago
  No, it's real. I gave it up a year ago, after hosting my own mail since forever. Exactly as stated: Hotmail (or whatever MS mail is called these days), GMail etc. are the main culprits. So yes, 'colossal' is the word.

fredsted 9y ago

Cool article. I personally use Virtualmin for this, it's very easy to set up, you can manage hosting and mail accounts for a bunch of different domains, easily do backups via S3, easy to set up SPF and DKIM, has a good web admin interface, easy to set up automatic updates, etc. By the way, here's the obligatory HN-style critic of the format, not the content: the blog template in this is a little annoying, it fiddles with scrolling by making it really slow on Safari on Mac, and Reader mode doesn't work.

kuczmama 9y ago
Virtualmin looks pretty cool, that looks like a great option. Yeah thanks for the feedback, I need to get around to self-hosting my blog as well. I set up blogger a few years ago and I never got around to changing it.

azr79 9y ago

Pro tip: don’t do this, use a custom domain name if you want, but never host your own email server.

deleted 9y ago
[deleted]

cjsuk 9y ago

Hell no. The moment Yahoo, Microsoft or google decide that they don’t like you, you’re SOL. Yahoo are the worst. If you try and deliver to them you get a deferral with an error message in your log with a URL. Then you have to open the link in the URL and fill in a form. They don’t have to accept the form and they ignore you for 3 months if it goes wrong. This happens even if you’re not on an RBL and have set up DKIM and SPF properly. Edit: you want to see the trouble we had to go to so we could run an SMTP server in AWS for outbound/abuse address inbound only and get that talking to Office 365 for internal use only. Two days of hell.

stephengillie 9y ago
I've worked for a real estate webhost and a major cloud provider/isp. Both had nightmarish problems simply ensuring email delivery. It did not help that the real estate "bulk mail newsletters" were legitimate spam. At the ISP, we offered an open internal relay to customers for some reason, and thus were again legitimate spammers. MS has 2 layers of spam blocking - one for Outlook.com, and a much stricter invisible layer for O365 with no support team.
- cjsuk 9y ago
  Yeah I wouldn’t want to do it now. I worked for a major pollster about 10 years ago and they had monumental problems back then just sending invites out that people had actually signed up for.
jankins 9y ago
I've been self-hosting for the past 4 years and this hasn't been my experience. Things like fixing DKIM/SPF eat an hour or two occasionally, but I've never encountered an issue where the time investment overtook the learning + other benefits - it's been hands-off 364 days of the year. I've only seen one blocked send happen -- blocked by my grandma's @att.net account. Since it happens so infrequently and nobody uses @att.net, I just re-sent from a Hotmail account instead. No issues with the other major players. But for my use-case it's easy to mitigate and if the problem persists I can invest more time in it, but one recipient blocking me in 4 years isn't bad. It's the only way to have ownership, which is is one of the benefits I really like - Google, Yahoo, etc. still get pieces of my personal email history because nobody else self-hosts or uses PGP, which is disappointing, but I prefer it over handing one player ownership the full history. BTW, I'm running it on the same 512MB DigitalOcean droplet that I use to host my static sites (personal website, small product sites, etc), so it's basically free since I'd need to host those things anyway, which is nice. Needs some swap though. Edit: Not saying these points are invalid. They're certainly valid, a service like Gmail _will_ be more reliable and easier. If you're blocked for some reason or have any other email probs, there's nobody else to fix it besides you.

tramtrist 9y ago

I moved from gmail to https://mailinabox.email/ https://mailinabox.email/ a few years ago and never looked back...

ryan-c 9y ago

I have been running my own mail server for a number of years. I recently was forced to move to a new IP. I had to get my server listed on DNSWL, then send in manual requests to Microsoft, AOL, and Yahoo to not block my email because they all seem to deny by default. I also ran through a couple of IPs with my provider before I found one that wasn't on any meaningful RBLs (my IP's on SpamGrouper, but that list is clearly run by an insane person and nobody seems to use it). As other people point out, some mail providers are just complete assholes and will blackhole your mail with no indication to the sender or recipient that it happened.

cedivad 9y ago

I've been hosting my own email server for 10 years until a couple of weeks ago, when I switched everything to gsuite. During the sign up process there is a step claiming "email just got awesome". That's as close of a description of my experience so far as you can possibly get. Email just got awesome.

jstimpfle 9y ago

Just another data point: I've been running my own mail for more than 6 years and I've never had a problem with receivers. Hosted on a cheap Strato VServer in Germany, I've never cared for the technical details, could not explain right now what DKIM and SPF are (and they're not configured), and only recently installed a self-signed SSL certificate in my Exim configuration to be able to use it with a Desktop client for submission (pretty sure outbound traffic still runs unencrypted).

prashnts 9y ago

Echoing to other's thoughts that hosting a server on your own would probably flag you in spam and just be a very tiresome process. A reasonably good and cheap service I could not recommend enough is https://www.migadu.com https://www.migadu.com . They allow you to use unlimited email domains, storage, addresses with the only limit being on total daily outgoing emails. The mini plan allows 100 outgoing emails a day which is more than sufficient for most of my purposes.

cygned 9y ago

https://workaround.org/ispmail https://workaround.org/ispmail We used this to setup a multi-domain email server. Works reliably and fast at both sending and receiving mail.

apple4ever 9y ago

Ugh a virtual server running Docker? Dumb. Just install it on the base OS without the overhead.

throwaway2016a 9y ago

You couldn't pay me enough to host my own mail... To me it falls under the same category as assembling my own computer. I can do it but to me it's not worth the trouble. Between DKIM, DMARC, and SPF, security, backup strategy, the fact if you are an open relay for even a day a bot net will find you and get your IP blacklisted for life... or an ISP could just blacklist you because they saw other spam from your same subnet on a shared hosting provider... Granted this article covers a lot of that (it talks about DKIM, DMARC, and SPF) I'm still counting this as one of the things I outsource.

oliwarner 9y ago

Friends don't let friends host their own email. It's just not worth the hassle these days unless you're managing hundreds of accounts and it's a full time endeavour. Even at the hundreds level, I'm sure there are people who'd rather outsource. It's not just email that these guys (Gmail, Outlook, Fastmail, etc) provide.

luord 9y ago

I wasn't aware of mail-tester. Yet another tool for checking my domains. Thanks.