7 ms·

I've wanted to take complete ownership of my email for a while now, but it seems risky, given the need for 100% uptime, and the anti-spam/deliverability ecosyst

I've wanted to take complete ownership of my email for a while now, but it seems risky, given the need for 100% uptime, and the anti-spam/deliverability ecosystem of the major providers and senders (to say nothing of security concerns, given that email is de facto auth for the vast majority of services). Has anyone done this? What were the pain points?

deleted 7y ago
[deleted]
stas2k 7y ago
You do not need 100% uptime to receive mail. Because email is store and forward, other server will retry for a reasonable time. The most complex part is setting up all the DKIM and friends to make your outgoing mail authenticated.
- GordonS 7y ago
  Was going to say much the same - other mail servers will retry failed sends for several days. Setting up the basics is relatively straightforward, but I found it tricky to setup TLS correctly for SMTP and IMAP, and DKIM and SPF were horrible to configure and get working right! I've been running my own mail server for 15 years or so - I really wish Mail In a Box had existed back then!
- pseudalopex 7y ago
  Bulk senders give up in hours.[1][2] Some will even unsubscribe you.[3] [1] https://postmarkapp.com/migration-guides/mailgun#processing-inbound-email https://postmarkapp.com/migration-guides/mailgun#processing-... [2] https://forums.aws.amazon.com/thread.jspa?threadID=145330 https://forums.aws.amazon.com/thread.jspa?threadID=145330 [3] https://mailchimp.com/help/soft-vs-hard-bounces/ https://mailchimp.com/help/soft-vs-hard-bounces/
dvko 7y ago
I've been running Mailinabox since late 2016 and have experienced 0 issues with downtime or my emails being flagged as spam. It did take a few weeks to train SpamAssassin to properly flag incoming email as spam but that worked very well and it. just. works. nowadays. Recently I closed down my lingering Gmail account and used imapsync to copy over all my email and contacts to my Mailinabox. I've also moved it to a different VPS provider once without any issues. Setup mailinabox on new server, close ports on old server, create final back-up, import back-up, switch DNS, done. 0 emails missed.
- thelittleone 7y ago
  I’ve tried getting out of gmail a few times with various sync tools which all seemed to fail before completion. I’m curious if it’s a mailbox size issue, or perhaps scripts have matured since I last tried (2 years ago). Any idea how many emails you synced over?
  - dvko 7y ago
    Close to 20.000 emails, ~4.5 GB in size. Tried using Google's data takeout first, but imapsync worked way better (while slower).
    - thelittleone 7y ago
      Thank you, that’s about the same size as my account. I struggled with Google’s takeout also. Going to make the move and feeling pretty good about it. Did you do anything to migrate accounts signed up using the gmail?
lisper 7y ago
I've been running my own mail server for about 15 years now. Getting it set up is a fair bit of work. The hardest part is just deciding how you want things configured. It has been pretty much hands-off for ten years now. Dealing with spam has been the biggest annoyance, but even that hasn't been too horrible, and it forced me to learn how to write milters. I pay for commercial hosting so my mail comes from a "reputable" IP address. I once got blacklisted by Spamhaus because I accidentally enabled an open relay, but that was easy enough to fix and it hasn't happened since. AMA if you decide to take this on, I'm happy to help.
- stephenr 7y ago
  ... can you expand on what you mean by “commercial” hosting?
  - lisper 7y ago
    Sorry, that was a very poor choice of words. What I actually meant was "a small commercial hosting company", i.e. not Amazon. Amazon IPs can have a dicey provenance. (Actually, the company was recently acquired so they may not be so small any more. The company is called ZeroLag and they were acquired by Total Server Solutions.)
    - stephenr 7y ago
      Right that’s what I figured you were using, thanks for confirming!
- velosol 7y ago
  When you say 'reputable' IP do you mean non-residential or is something like AWS, Linode, etc enough? Edit: duplicate to sibling, apparently I had this tab open longer than I thought.
  - lisper 7y ago
    Yes, I meant non-residential, and also not AWS. The problem with AWS is that a lot of spammers use it, and so a lot of AWS IP addresses get tainted. If you go with a smaller ISP that charges more it's less likely that their clients will be spammers. But to be fair, I've never tried running a mail server on AWS. It's possible that you could do it and not have any problems, I don't actually know. It's probably an experiment worth doing.
stinos 7y ago
I ran mailinabox for like 5 years and there were no major painpoints, some quirks while setting up which I don't even remember. And it's best to keep it up to date, updating to new releases if you're far behind can get messy. Essentially it all boils down to reading instructions and reproduce the commands in them and hope they don't fail. If they do, standard search internet-try something-cyle. Normally if you've ever touched linux or similar and/or ran a server you know the deal. Also if you don't know much about email on linux it all remains a black box which isn't necessarily a problem. So in the end it depends on what type of work you like, I guess. Personally I don't really enjoy fiddling with things to get/keep them working anymore; I've done that more than enough in the past, building your own linux from scratch and whatnot, but in the end it's just not my cup of tea and turns out I just don't want to waste time on it, no matter how little, so I siwtched to fastmail. Honestly, no clue if that makes my mail more or less secure.
doctorpangloss 7y ago
> What were the pain points? If Apple can't even convince Google to provide push for the native Mail client, draining the batteries of hundreds of millions of users everywhere for no good reason, all at the altar of driving downloads of the Gmail app, some pathetic self-hosted emailer isn't going to be able to get Google to accept their mails once it's marked as spam by an automated system (or a change in strategy decided by a PM in 10 minutes).
throwanem 7y ago
Use https://mxtoolbox.com/blacklists.aspx https://mxtoolbox.com/blacklists.aspx or something like it to verify that you have a clean VPS IP before putting any effort into setting it up as a mail host. Use https://www.mail-tester.com/ https://www.mail-tester.com/ or something like it to check how completely you're set up to maintain a good sender reputation. Having one doesn't guarantee you'll be able to deliver to places like Gmail, but not having one does guarantee that you won't, so keep fixing things until you get a 10/10 score. (The tool will tell you what it doesn't like, so this is easier than it might sound at first.)
copperx 7y ago
> given the need for 100% uptime When I was a teen, I ran a personal mail server on a dynamic IP (using DynDNS) over dialup with about 25% uptime. Miraculously, mail still got delivered to my server. I was in awe of the reliability of SMTP. The greatest problem today affecting reliablity is that mail gets dropped silently by the major providers.
- pwg 7y ago
  Yes, SMTP was designed in the days of much less reliable internet connections (either systems that only had connectivity at intermittent times, or just basic unreliability). The protocol has a built in assumption that a receiver might not be accessible "now" and so servers are required (in order to be compliant to the standard) to hold, and retry, over several days, to deliver an email before giving up and returning it to the sender. So it is quite feasible to run email over dialup. You just have to periodically connect, and remain connected for a reasonable period of time, and SMTP will work just fine (only with higher latency than we are accustomed to in 2019).

deleted 7y ago

[deleted]

stas2k 7y ago

You do not need 100% uptime to receive mail. Because email is store and forward, other server will retry for a reasonable time. The most complex part is setting up all the DKIM and friends to make your outgoing mail authenticated.

GordonS 7y ago
Was going to say much the same - other mail servers will retry failed sends for several days. Setting up the basics is relatively straightforward, but I found it tricky to setup TLS correctly for SMTP and IMAP, and DKIM and SPF were horrible to configure and get working right! I've been running my own mail server for 15 years or so - I really wish Mail In a Box had existed back then!
pseudalopex 7y ago
Bulk senders give up in hours.[1][2] Some will even unsubscribe you.[3] [1] https://postmarkapp.com/migration-guides/mailgun#processing-inbound-email https://postmarkapp.com/migration-guides/mailgun#processing-... [2] https://forums.aws.amazon.com/thread.jspa?threadID=145330 https://forums.aws.amazon.com/thread.jspa?threadID=145330 [3] https://mailchimp.com/help/soft-vs-hard-bounces/ https://mailchimp.com/help/soft-vs-hard-bounces/

dvko 7y ago

I've been running Mailinabox since late 2016 and have experienced 0 issues with downtime or my emails being flagged as spam. It did take a few weeks to train SpamAssassin to properly flag incoming email as spam but that worked very well and it. just. works. nowadays. Recently I closed down my lingering Gmail account and used imapsync to copy over all my email and contacts to my Mailinabox. I've also moved it to a different VPS provider once without any issues. Setup mailinabox on new server, close ports on old server, create final back-up, import back-up, switch DNS, done. 0 emails missed.

thelittleone 7y ago
I’ve tried getting out of gmail a few times with various sync tools which all seemed to fail before completion. I’m curious if it’s a mailbox size issue, or perhaps scripts have matured since I last tried (2 years ago). Any idea how many emails you synced over?
- dvko 7y ago
  Close to 20.000 emails, ~4.5 GB in size. Tried using Google's data takeout first, but imapsync worked way better (while slower).
  - thelittleone 7y ago
    Thank you, that’s about the same size as my account. I struggled with Google’s takeout also. Going to make the move and feeling pretty good about it. Did you do anything to migrate accounts signed up using the gmail?

lisper 7y ago

I've been running my own mail server for about 15 years now. Getting it set up is a fair bit of work. The hardest part is just deciding how you want things configured. It has been pretty much hands-off for ten years now. Dealing with spam has been the biggest annoyance, but even that hasn't been too horrible, and it forced me to learn how to write milters. I pay for commercial hosting so my mail comes from a "reputable" IP address. I once got blacklisted by Spamhaus because I accidentally enabled an open relay, but that was easy enough to fix and it hasn't happened since. AMA if you decide to take this on, I'm happy to help.

stephenr 7y ago
... can you expand on what you mean by “commercial” hosting?
- lisper 7y ago
  Sorry, that was a very poor choice of words. What I actually meant was "a small commercial hosting company", i.e. not Amazon. Amazon IPs can have a dicey provenance. (Actually, the company was recently acquired so they may not be so small any more. The company is called ZeroLag and they were acquired by Total Server Solutions.)
  - stephenr 7y ago
    Right that’s what I figured you were using, thanks for confirming!
velosol 7y ago
When you say 'reputable' IP do you mean non-residential or is something like AWS, Linode, etc enough? Edit: duplicate to sibling, apparently I had this tab open longer than I thought.
- lisper 7y ago
  Yes, I meant non-residential, and also not AWS. The problem with AWS is that a lot of spammers use it, and so a lot of AWS IP addresses get tainted. If you go with a smaller ISP that charges more it's less likely that their clients will be spammers. But to be fair, I've never tried running a mail server on AWS. It's possible that you could do it and not have any problems, I don't actually know. It's probably an experiment worth doing.

stinos 7y ago

I ran mailinabox for like 5 years and there were no major painpoints, some quirks while setting up which I don't even remember. And it's best to keep it up to date, updating to new releases if you're far behind can get messy. Essentially it all boils down to reading instructions and reproduce the commands in them and hope they don't fail. If they do, standard search internet-try something-cyle. Normally if you've ever touched linux or similar and/or ran a server you know the deal. Also if you don't know much about email on linux it all remains a black box which isn't necessarily a problem. So in the end it depends on what type of work you like, I guess. Personally I don't really enjoy fiddling with things to get/keep them working anymore; I've done that more than enough in the past, building your own linux from scratch and whatnot, but in the end it's just not my cup of tea and turns out I just don't want to waste time on it, no matter how little, so I siwtched to fastmail. Honestly, no clue if that makes my mail more or less secure.

doctorpangloss 7y ago

> What were the pain points? If Apple can't even convince Google to provide push for the native Mail client, draining the batteries of hundreds of millions of users everywhere for no good reason, all at the altar of driving downloads of the Gmail app, some pathetic self-hosted emailer isn't going to be able to get Google to accept their mails once it's marked as spam by an automated system (or a change in strategy decided by a PM in 10 minutes).

throwanem 7y ago

Use https://mxtoolbox.com/blacklists.aspx https://mxtoolbox.com/blacklists.aspx or something like it to verify that you have a clean VPS IP before putting any effort into setting it up as a mail host. Use https://www.mail-tester.com/ https://www.mail-tester.com/ or something like it to check how completely you're set up to maintain a good sender reputation. Having one doesn't guarantee you'll be able to deliver to places like Gmail, but not having one does guarantee that you won't, so keep fixing things until you get a 10/10 score. (The tool will tell you what it doesn't like, so this is easier than it might sound at first.)

copperx 7y ago

> given the need for 100% uptime When I was a teen, I ran a personal mail server on a dynamic IP (using DynDNS) over dialup with about 25% uptime. Miraculously, mail still got delivered to my server. I was in awe of the reliability of SMTP. The greatest problem today affecting reliablity is that mail gets dropped silently by the major providers.

pwg 7y ago
Yes, SMTP was designed in the days of much less reliable internet connections (either systems that only had connectivity at intermittent times, or just basic unreliability). The protocol has a built in assumption that a receiver might not be accessible "now" and so servers are required (in order to be compliant to the standard) to hold, and retry, over several days, to deliver an email before giving up and returning it to the sender. So it is quite feasible to run email over dialup. You just have to periodically connect, and remain connected for a reasonable period of time, and SMTP will work just fine (only with higher latency than we are accustomed to in 2019).