12 ms·

Ask HN: The middle ground for email self-hosting?

75 points by kajiryoji 4y ago

There were many threads previously on how bad/impractical it is to self-host your email. For example, this thread[1] is just a few days ago. I was wondering if anyone had any experience combining the microsoft 365 business basic (6$ a month) with self-hosted email server? By relaying SMTP through the Microsoft provided outlook server, would my custom domain be free from being marked by spam? [1]: https://news.ycombinator.com/item?id=31180379 https://news.ycombinator.com/item?id=31180379

abofh 4y ago
I've done similar with AWS's SES before with no trouble - I imagine you'd have similar results with MSFT.
- uuyi 4y ago
  I’ve had severe deliverability issues with SES. You’re pooled with a lot of bad actors and occasionally you get a blacklisted outbound. They have a reputation system but it’s far from perfect and requires manual intervention if you’re blocked from sending which is a real pain. For personal stuff I’m using iCloud+ domain hosting now after moving from fastmail which was also trouble free.
johnvalger 4y ago
[flagged]
deleted 4y ago
[deleted]
asdfqwertzxcv 4y ago
Do the same but use a http://postmarkapp.com http://postmarkapp.com or https://www.smtp2go.com https://www.smtp2go.com account where you likely pay nothing unless you're a heavy sender. Had an issue with my self hosted email going to spam and these services solved it.
tiborsaas 4y ago
I'm still wondering why there's no go-to choice to spin up a docker container that has everything preconfigured.
- rglullis 4y ago
  Because the problem is mostly about the reputation of the IP address. Setting up an email server is somewhat easy. Ensuring that other servers consider yours as legitimate that is the problem.
  - cubesnooper 4y ago
    Spam is often sent from big providers, too. For several years I hosted my email the “middle ground” way (i.e., relaying outgoing mail via Google Workspace), and despite using DMARC correctly it was not infrequent that my emails would go to spam (even to GMail boxes) or never show up at all. Obviously GMail is such a giant that email providers have to be very careful when blocking it, but enough spam comes from there that receivers clearly use some heuristics to block some of it. I’ve even received multiple rejection notices because the GMail server my email was sent through happened to be on a blacklist! I switched last year to sending directly from my VPS. It was partly for privacy from Google, but moreso so I could enforce outgoing TLS. For the first few days they went to spam boxes or moderation queues, but I made sure they were rescued, and ever since I’ve had no deliverability issues sending to Google, some local ISPs, and even Microsoft (which seems crazy, as I never got a mail from my domain to show up in Outlook when I was relaying through Google). I can only speak for my own experience, of course. But that is what I experienced.
    - Avamander 4y ago
      I have had very similar experiences. Doing things perfectly and building up "reputation" helps a lot.
- elorant 4y ago
  Would you trust it if there was one?
jfernandezr 4y ago
You can also try to use your domain registrar SMTP as relay, in case they offer you a mail service, and setting their SPF records to the domain. I have set it up with Gandi and works pretty good.
huhtenberg 4y ago
Don't know about the effect on the spam ranking, but on the flip side you'd lose some privacy. For example, email notifications sent by Stripe are delivered over TLS'd connections. My bank does this too. If you are to proxy these, the relay will obviously be in the loop on all emails that aren't local to your mail server.
outsomnia 4y ago
Incoming email is simple, MTAs have no problem at all delivering to residential IPs if that's what your MX says. So ideally you should run your own postfix + dovecot at your premises and point your MX to that. You have to take additional steps for sending email. > would my custom domain be free from being marked by spam? The receipient's mail service gets to choose if it thinks your email is spam, this will happen whatever your sending arrangements, outlook is not immune from sending spam and is no magic guarantee others will give it a free pass somehow. Recipients score your email on a variety of characteristics, many of which are under your control. A major consideration is the sending netblock, eg, residential ADSL blocks are likely to be rejected or scored to hell. Garbage netblocks like linode with a terrible reputation likewise. A clean (no history of spamming) IP in a clean (reputable) netblock will be scored higher. You can look up sender reputations here, which is the service the big email providers use. https://senderscore.org/ https://senderscore.org/ So to send your own mail, you should rent a dedicated server on your own IP, you can do this for $30/mo or so. All you need to run there is postfix + SASL auth to forward your (and only your) emails. Then you must configure DKIM etc correctly and check your emails are validly signed, DKIM requires being able to add TXT fields to your DNS. It's very possible to do this yourself securely after a bit of a learning curve and have it require minimal ongoing maintenance.
- raegis 4y ago
  I think $30 a month is high just for email. I do $5/month now and have been doing this for over 20 years on my domain. (It was more expensive in the past.)
  - outsomnia 4y ago
    Yes, you can do it on a VPS much cheaper. But this is your outgoing email authorized by DKIM... an attacker can use it to take over most of your accounts via Forgot Password flow. I think it is a false economy to have that depend on a shared VM.
    - cubesnooper 4y ago
      You don’t have to store your DKIM keys on the VPS. I keep my signing infrastructure local, and send outgoing mail over a WireGuard tunnel so it looks like it was sent from the VPS.
adhikasp 4y ago
I have good luck using https://forwardemail.net https://forwardemail.net for having email address on my own custom domain. This will catch inbound email (configured from DNS MX record) to my gmail, and for outbound I set the custom domain as gmail alias. Doesn't solve privacy, data ownership, nor google lock-in issue (but at least if I lost my gmail, I can move to a real email selfhost solution and keep my address). As my need is just to have custom domain address for the cool factor of it, this simple setup works flawlessly.
- jokethrowaway 4y ago
  I do the same and I can't thank niftylettuce enough for it
- libertine 4y ago
  The problem I'm having with a similar solution is that for some of my emails (that I used to test), are labeling the emails sent with the custom domain as gmail alias as "Promotion", so it doesn't go into the Inbox on gmail, goes to the Promotions tab - and makes no sense, since the only thing in my email structure I have is my name as a signature and the domain.
mattbee 4y ago
That definitely outsources the most painful part of the problem. Though I'd probably use a forwarding-specific service - e.g. a lot of people swear by SES, I've used smtp2go just for Hotmail deliveries, I'm sure there are others. You might have a bit of SPF fiddling to do, just because you might be fighting the default self-hosting assumption that incoming and outgoing servers are the same.
sam_goody 4y ago
I have experience self hosting for incoming mails; outgoing mails are sent via mailchimp. We have had this setup for several years. It is not difficult to setup, emails are delivered reliably, and email delivery cost is negligible. Some of our users use Outlook / Thunderbird / Apple Mail as a client, some use GMail as a client (check external mail / send as user) and some use Rainloop which I set up on the mail server.
dmje 4y ago
I just don't understand the attraction of self hosting email. The pain seems extreme, even for those who understand the considerable number of nuances. To me the happy middle ground is email on your own domain but using an existing provider such as G / MS or whoever. That way you've got control but don't need to worry about the pain. It does require paying for but really on balance not much. If you're spending more than an hour a year maintaining your self hosted email (which you will, big time!) then your Google Workspace / O365 is paid for. The situation I've found frustrating is about family email on same domain. I've gone in a huge loop that has ended up back with GWorkspace which is quite costly for 3-4 family users. But still - not even close to the horror of self hosting...
- cubesnooper 4y ago
  > I just don't understand the attraction of self hosting email. For several years I’ve hosted in the “middle ground” sense described by the OP, running my own incoming mail server and relaying outgoing mail through a big provider. The main benefit for me (compared to using a big provider with my own domain) is personal privacy. When I used Google for mail, Google had access to so many pieces that make up my personal life: Purchase receipts. Flight itineraries. Conference registrations. Emails from my university. Emails from my realtor. Utility bills. Notifications for subscribed forum threads, GitHub repositories, Wikipedia pages. Whatever newsletters I chose to subscribe to. Theoretical access to any site with password reset by email. Running my own MX eliminates Google’s access to most of these things. There are other some other benefits too. Free infinite aliases I can use to sign up on any website. No fear of dependence on features that might get paywalled. No sudden danger of having to migrate data to another provider. > If you're spending more than an hour a year maintaining your self hosted email (which you will, big time!) then your Google Workspace / O365 is paid for. Reducing my data footprint is something I care about enough to spend my spare time on.
  - dmje 4y ago
    In my world (running a small digital agency), I've realised that even if I do all the things to reduce my data footprint (for instance, migrate all my docs to NextCloud, self host email, etc), it actually all breaks almost instantly - all it takes is a client to share a doc or folder with me where they use GDocs / Dropbox / whatever, and I'm effectively straight back in it. My basic strategy is one of slight defeatism, I have to admit. I am 100% in to Google for their (really quite excellent) tools in Google Workspace: nothing is as good as GDocs, nothing is as good as Gmail, nothing is as good as Google Meet; but I do things to ensure I'm not utterly f**d if the Random Google AI Best happens to decide I'm some sort of unspecified menace. So for instance - I use Google Docs but only with .docx / .xlsx files rather than native .gdoc .gsheet files. I back this up automatically to my self-hosted NAS. I do this on a domain which I own, so can step away if things do happen to go south, or costs double or whatever. Then I use kagi.com for search, and have a piHole / ublock / Brave to minimise footprint from a tracking POV. I know, it's all probably moot given I just open up my inbox to Google, but I've tried and failed to find a provider that is even close to being the same balance of low price + utility. I got excited about Fastmail but turned out it was a combination of not very good AND really expensive once I factored in having several accounts on the same domain. I had a horrific experience with iCloud+ (they have a weird YEAR long account blocking issue thing that I won't go into now). M$ was awful and required me to send everything through GoDaddy's DNS. All the others were just underwhelming or expensive or both. So - sadly - I'm back in the G stable where I'll stay for the time being... :-)
derkades 4y ago
This is how I do it, I can highly recommend it. My residential ISP provides a free relay service so I use that.
caeruleus 4y ago
I have been selfhosting my email stack on a cheap VPS (~ $10) for many years now, probably since 2009 or so. I used to set up everything manually, but that was quite painful and there were some rough edges. Nowadays, I just use mailcow (https://github.com/mailcow/mailcow-dockerized https://github.com/mailcow/mailcow-dockerized) for the setup part and have a much more polished experience. Email deliverability is not a problem. Generally, you just have to make sure to correctly setup DKIM/SPF (and DMARC) and check if your IP is on some blacklist. You can get it removed easily. (Edit: Also required is forward-confirmed reverse DNS, see below). There was one provider that denied incoming mails from me, even though I got the IP removed from every blacklist I could find. I wrote a short mail to the admin contact and got told I had to host a web page with contact information on the same IP. Since being whitelisted there, everything works like a charm, couldn't be happier.
- ssl232 4y ago
  > I wrote a short mail to the admin contact and got told I had to host a web page with contact information on the same IP. Interesting. Was that just to prove to that particular provider that you, the emailer, own the domain? Or is it some more widely used (beyond that provider) practice?
  - caeruleus 4y ago
    Afaik, it is specific to this provider. I'm not entirely sure why exactly tbh, probably to require some kind of associated identity (the policy is likely targeted to larger providers). It was a manual process and I never had to do anything similar for a different one. They require that a) the sending IP address has a PTR record b) from a domain that you own c) that resolves to the same IP address. This is also very important for general deliverability (https://en.wikipedia.org/wiki/Forward-confirmed_reverse_DNS https://en.wikipedia.org/wiki/Forward-confirmed_reverse_DNS). They furthermore recommend d) that your host name should clearly mark it as a mail server and e) to make sure the domain leads to a web page that contains provider details and contact information.
    - ssl232 4y ago
pxeger1 4y ago
I've done this with Amazon SES, which is PAYG and costs me pennies. https://www.pxeger.com/2020-07-02-hybrid-cloud-email-with-amazon-ses-and-dovecot/ https://www.pxeger.com/2020-07-02-hybrid-cloud-email-with-am... It is a bit overcomplicated, because I also set up SES to receive email, but I could run that instead with an ordinary Postfix server. It would be much simpler for outgoing only, I think
- noduerme 4y ago
  SES is good for loads of outbound mail. People don't know this, I mean no one knows this, but if you're running an EC2 instance with some allocated IPs, you can contact Amazon's customer service and ask them to unthrottle outbound mail on it. Typically they clamp down and stop connecting any SMTP deliveries off the EC2s if you do more than 10 emails/hr or something. But if you plan on keeping the IPs for awhile you can set up DKIM and SPF and all that, call Amazon and tell them you're sending and receiving legitimate business emails off that server. They may try to refer you to SES, but if you tell them you need to manage it on a private server for legitimate reasons, they have the ability to lift the block for you. Do check your allocated IPs for blacklisting in advance, and obviously don't give them any cause for being blacklisted in the future.
josh_fyi 4y ago
I do outbound email for my domain from a generic Gmail account, where my domain address is added as an alternate "Send mail as". This gets sent through Mailgun's free SMTP. Incoming email to my domain is forwarded through Cloudflare's free service to the generic Gmail account. This seems to pass all quality checks to avoid being sent to spam.
- nicbou 4y ago
  I do this but with Namecheap + Gmail. It has worked flawlessly for years. The only issue is that emails don't come in instantly. They can take up to 15 minutes. I consider this a feature, but it would be really annoying to some.
  - capableweb 4y ago
    > The only issue is that emails don't come in instantly. They can take up to 15 minutes. I consider this a feature, but it would be really annoying to some. I have a somewhat similar setup, slightly different. Most emails arrive within 10-15 seconds, but sometimes gets stuck somewhere along the line (as it happens with emails), which is normally not a problem. But some platforms force you to use the "send link to login via email" option for login, which again, normally is not a problem. But when they have a timeout of 10 minutes + it takes 15 minutes for it to arrive, you end up not being able to login. Only happened to me a few times during the years of this setup, but when it does happen, it really sucks.
    - noduerme 4y ago
      This is called "greylisting" and it's fairly normal if you don't run your own server. Someone triggers a spam block and gets put on a partial time-out. Then the more times they try to re-send an email to you, the longer their emails get kept in limbo. If you don't control the server, you don't have independent email.
      - capableweb 4y ago
        Well, could be that, could be other things as well. Could be the sender who have implemented their email sending via a queue, and currently they are overloaded. Could be their email sending server/service who is behind and having delivery problems. Could be numerous things thanks to the nature of email. In the end, the UX of having to wait for an email sucks, sometimes.
david_draco 4y ago
The middle ground is to keep copies of all emails on your computer through a email client (via IMAP or POP), so that when you are locked out by your email provider, you can resume your business and communication with your contacts with your new email address. With IMAP, you can even upload your old emails to your next provider.
g105b 4y ago
I know this is slightly off topic from OP's question, but I'm chiming in with one piece of first-hand advice: You can self host mail alongside gmail/outlook on your own domain. More than one email service can run concurrently, without any problems. That often overlooked fact allows you to quickly set up something like gmail on your domain, then use the trial period to see if you can self-host with any success. If you can, then you can shut down the trial, or move on to trial another paid service like 365 while you're still "trialing" your own host. It really helped me make the transition.
hannob 4y ago
You should be aware that Microsoft's SMTP servers parse the mime structure of the mails and restructure it in a non-standard way. I have no idea why, but for example this breaks PGP signatures.
- rockwotj 4y ago
  This is due to a well known fact that Microsoft hates the email ecosystem. I work on a email client and a large majority of "this email is broken" is due to weird outlook behavior. Most recently it's TNEF attachments: https://en.wikipedia.org/wiki/Transport_Neutral_Encapsulation_Format https://en.wikipedia.org/wiki/Transport_Neutral_Encapsulatio...
- Avamander 4y ago
  They break way more than that, it's quite annoying. If you search in various MTA/client mailing lists you'll find a bunch of threads. Though, others do violate standards as well, but not like this.
JoshTriplett 4y ago
I used to do something similar: I hosted my own IMAP, while using Gandi for receiving mail and sending mail. That meant I didn't lose mail on the receiving side if my mail server had an issue, and I didn't fail to deliver mail to others because I hadn't jumped through enough hoops. (Literally everyone I personally know who has run their own mail server has had one or both of those problems at least once.) This worked well for me because it gave me the feeling of having more control and privacy and security over my email. I switched away from that solution when I realized that in practice I have less ability to effectively provide security than the whole security and product teams of a major email provider.
- sirnoggin 4y ago
  Whats's Gandi please?
  - JoshTriplett 4y ago
    https://www.gandi.net/ https://www.gandi.net/ They're a domain registrar that also supplies email and hosting and similar services.
defanor 4y ago
A major theme in self-hosted email discussions is deliverability issues (particularly to larger email service providers), and I tend to be unsure how bad it actually is: sometimes it does seem pretty bad, other times it sounds like it's fine, and possibly the chatter about failed deliveries is caused by misconfigured servers and/or misunderstandings. Seems like it shouldn't be hard to check and collect reference statistics with a survey, though I'm failing to find surveys of that kind, and getting accounts on public services would be the tricky part for me personally (since I don't like to provide my phone number), so not doing that myself either. Only occasionally tried to check it with others, and messages were delivered fine in those cases -- but that's just a few samples.
- michaelt 4y ago
  The problem with deliverability issues is the impossibility of proving a negative. If I send an e-mail to a company's customer support, or to my senator, or I reply to a potential client, or I contact an open source mailing list and I don't receive a reply - do I know if my message made it to them or not? I mean, it's plausible that JohnDoe@senate.gov just didn't deign to reply to my e-mail. But it's equally plausible there's some subtle misconfiguration - like an e-mail forwarder that breaks the SPF signature. It's not like I can sign up for a senate.gov e-mail address to test with. Meanwhile, to paraphrase an old joke, when your senator rejects your e-mails you have a problem. When your senator rejects @gmail.com they have a problem.
  - defanor 4y ago
    Sure, strictly speaking it's impossible to ensure that a message was actually read by a user even with automated end-to-end delivery acknowledgements and/or in centralized systems: UIs manage to gobble/hide messages, users fail to find how to open attached documents (and declare that those are missing), etc. But I imagine that a survey/statistics would still help to estimate how bad deliverability in general (in a variety of common cases) is: without that there are differing and even more vague ideas of its state.
  - noduerme 4y ago
    I can prove a different negative with my own mailserver - when I've sent things to @gov, they've always been responded to. I think that just proves the government reads ALL their spam.
Neil44 4y ago
I’d go AuthSMTP or similar rather than 365 for pure outbound as it will be much more cost effective.
- saurik 4y ago
  (This is exactly what I do.)
  - Neil44 4y ago
    I think it's a good service, I like that it can do DKIM on the outbound emails even though your 'main' MTA might not support it or might be a pain to set up.
fxtentacle 4y ago
I'd say just rent hosted dovecot from a reputable company and you're then free to build your own filters, tools, pipelines on top of that. My favorite is mailbox.org
pkalinowski 4y ago
I did different setups over time, but currently settled on forwardemail.net for incoming catch-all custom domain and iCloud+ SMTP servers for sending.
ulrikrasmussen 4y ago
Many VPS providers have an SMTP gateway that you can use to ensure that your email does not get marked as spam. I use transip.eu and never have problems. Prior to setting up my mail server to use their gateway, I ONLY had problems with Microsoft-based receivers such as @outlook.com. So please, don't use Microsoft for anything email related, as they are currently one of the worst offenders in making it hard for people to run their own mail server.
- deleted 4y ago
  [deleted]
noduerme 4y ago
I'm going to write this assuming you're a non-technical professional, lawyer perhaps, looking for a private email solution that doesn't rely on third parties. Bottom line: There's no "middle ground", any middle ground you cede is allowing a third party some kind of access. Hosting your own email has become expensive and time-consuming (although IMHO it's still extremely worthwhile, and I do it in spite of what a pain in the ass it is). Be prepared to spend at least $50/mo and at least 6 hours in setup and 1-2 hours a month debugging if you do it personally. Or you can find someone to help (see below). You need your own IP address. You need a dedicated box, not a VPS. And check the IP address in advance to make sure it's clean, and not blacklisted. Tell the datacenter you're going to be doing email and ask them if they're okay with that for a clean IP. Use https://mxtoolbox.com/blacklists.aspx https://mxtoolbox.com/blacklists.aspx to test the IP address they're offering you, or IPs in their range. Unlike some people are saying, you should never do this off a VPS if you have an interest in keeping the email secure and functioning for a long time. My personal go-to would be dedicated hosting in the Netherlands, Switzerland, Isle of Man or Norway. Clean IPs, your own box, start with a clean server. But then you're talking $250/mo or so. If you don't know how to set it up, there are people who can do it for you. You will need to essentially trust that person with access to all your correspondence, but if they do it properly, no one at the server farm[0] or elsewhere will have access to your correspondence... which puts you in the 0.01% of people on earth whose email isn't read by big tech companies. [0] -who doesn't physically access the server: Look for ones in cages and ask who has physical access and why.
- twobitshifter 4y ago
  To prevent 3rd party access you need to not only host your own email but all recipients of your mail need to be self hosted as well. Unless you are providing everyone you correspond with their own account, 3rd parties will be involved.
  - noduerme 4y ago
    No, you just have to be aware of who you're sending to, and encrypt appropriately. I mean obviously if I send to someone @gmail, they're going to know I sent that person an email. But they can't scan my inbox for keywords.
p0d 4y ago
I use a free Gmail account as my mail client and direct all my custom domains from Fastmail to Gmail. Messages are delivered very quickly and I don't have spam issues. Truth is I like Gmail but I think Google have dropped the ball with, "Let me point all my custom domains to a Gmail account. I would even pay you but I don't want Workspace".
superasn 4y ago
I think your best bet is Amazon ses. In my recent testing, all mail always go through - unless the recipient marks it thus. It's damn cheap too, like almost free for low volumes. I think Amazon uses this for their workmail also and has become pretty strict at policing abuse. I am only speaking for gmail though, so ymmv for hotmail et al which I haven't checked.
- quickthrower2 4y ago
  I agree. Amazingly cheap. Worth a little bit more technical hassle (looking at templating and batching for example IIRC you had to do for marketing or bulk transactional emails) but still amazing value for money.
- sirnoggin 4y ago
  Interesting just met a client who us using Amazon SES with a few domains, all of his emails go to spam, across all domains. Curious aye! Almost like these sender's aren't inpregnable and there is something else going on.
fsflover 4y ago
Another alternative is https://thehelm.com https://thehelm.com: Helm is a personal, private email server that won't share your data. The Verge
tbyehl 4y ago
FWIW, a $4/m Exchange Online Plan 1 is probably enough. Or even the $1/m Exchange Online Protection if you don't have any need for a hosted mailbox. Anything that'll get you ongoing access to the Exchange Admin Center. I use a single M365 Business Basic account, as a conventional mailbox, for one of my domains. From within the Exchange Admin Center there's extensive control over mail flow -- domains to accept mail for, inbound and outbound connectors for routing mail between on-prem mail servers. Best as I can tell, literally ongoing any subscription that gets you an account with access to EAC ought be enough to route any or all of your email through EO in either direction. https://www.microsoft.com/en-us/microsoft-365/exchange/compare-microsoft-exchange-online-plans https://www.microsoft.com/en-us/microsoft-365/exchange/compa... https://www.microsoft.com/en-us/microsoft-365/exchange/exchange-email-security-spam-protection https://www.microsoft.com/en-us/microsoft-365/exchange/excha... https://docs.microsoft.com/en-us/exchange/standalone-eop/standalone-eop https://docs.microsoft.com/en-us/exchange/standalone-eop/sta... https://docs.microsoft.com/en-us/exchange/mail-flow-best-practices/use-connectors-to-configure-mail-flow/use-connectors-to-configure-mail-flow https://docs.microsoft.com/en-us/exchange/mail-flow-best-pra...

nottorp 4y ago

> If you're spending more than an hour a year maintaining your self hosted email (which you will, big time!) then your Google Workspace / O365 is paid for. But what's the cost when Google's "AI" bans your account? For paid email hosting I'd go to some provider with actual support...

laurent123456 4y ago
Since he uses his own domain, he can move elsewhere if there's a problem with Google. I do that too, and it's indeed much simpler than self-hosting.
- nottorp 4y ago
  It sounded like he's actually keeping the emails on google, even though they come on a custom domain. And that doesn't seem safe to me.
  - laurent123456 4y ago
    For backup he could get Thunderbird or similar to fetch the emails are regular interval. I think it's generally a good idea to do so.
  - dmje 4y ago
    I use a client, so have a local IMAP backup, and regularly back this up to my NAS. That's not really my worry, though. What scares me about being heavily invested in blah@gmail.com or blah@notmydomain.com is lockout from services. As long as I have my own domain then I can just switch out the DNS and I'll still be able to get into whatever web service I signed up to using my email address.
- hedora 4y ago
  On that note, I have amazon manage my custom domain. Now I'm terrified that I'll return one too many pairs of socks or something, and get the retail consumer part of my account banned (we buy a lot of stuff on Amazon). Does anyone have experience with what happens to AWS resources (specifically, domains) when that happens?
b3kart 4y ago
If you use your own domain (as the parent suggests), the cost is creating an account with any other e-mail provider that supports custom domains and updating DNS entries.
gnopgnip 4y ago
The same thing camps happen with any company. If you have your own domain you can get a new email host, change your mx record. Update your email client to use the new credentials. If you use mobile email, upload existing recent email from your desktop or restore/ import from backup. This means not registering the domain with your email host to avoid problematic delays

dspillett 4y ago

> and I tend to be unsure how bad it actually is: sometimes it does seem pretty bad, other times it sounds like it's fine That is pretty much it. One factor is that once you are on a blacklist it can spread like wildfire and be much faf to get off them all again, so the risk is small but the hassle if it happens is high. Also if you send mail for numerous people there is going to be a much higher risk: every extra user/account/address is an extra hack target (do all your users have good, non-shared, passwords?) or just extra volume that might be accidentally classified as junk (and once something from your server gets classed that way, future content may get more aggressively analysed and more mistakes may happen). I've run my own mail server, including sending mail directly, for many years and to my knowledge not had a significant delivery problem. But I have a few mitigating factors: the IPv4 address is essentially on a commercial ISP range, not one that looks like a residential account or a VPS service provider, and the ISP is one that takes junk mail seriously, so there is less “splash damage” potential, and the same range has been used this way for several years (the main sender has moved around that small range, when testing upgrades on a copy VM for instance, but never away from it entirely) so it never looks like a brand new mail server these days, I only serve myself and a very small number of other users, our outgoing mail volume is pretty low. It is a bigger problem for hosting services (much bigger user-base and little control over what they might send) or if you are sending from one of their ranges, if sending from a residential ISP address range, if your volume is high (perhaps you have apps that send mail as well as your personal mail?), etc., but it can be a problem for everyone. I'm rebuilding my mail service soon (moving off Zimbra to just configuring the parts myself, as we don't need the extra features these days, it is too chunky for just a mail server, and at the end of next year they stop releasing easy install packages for the non-paid users (they already have for v9., next year v8. hits EOL)) at which point I might reconsider where it is hosted and if I should be sending via a paid SMTP relay to let them worry about deliverability, though as far as I know I've not had a problem.

jasode 4y ago

>, and I tend to be unsure how bad it actually is: sometimes it does seem pretty bad, other times it sounds like it's fine, and possibly the chatter about failed deliveries is caused by misconfigured servers and/or misunderstandings. It's not just misconfigured email server settings like DKIM, SPF, DMARC etc. One can correctly set all of those and still have the outgoing emails rejected or spamholed. Why? Because the big email players like GMail, Microsoft Outlook.com, etc use black-box heuristics of reputation datapoints that exist outside the boundaries of email settings such as... "amount of email volume", "# of spam abuse reports from ip block", etc. Because "sender reputation" cannot be encoded into an email configuration (DKIM/SPF/DMARC/etc), that's why nobody can provide a convenient Docker container with a perfectly working self-hosted email server that can reliably send email. If such a thing existed, the spammers would use it as well! A datapoint such as "volume of email from this ip" is an unstated behavior/activity number and not an identity setting like DKIM. And the invisible heuristics keep changing which causes previous email setups that worked -- to later stop working for no obvious reason. Why? Because there's a constant arms race between spammers and email filter algorithms. This means others' email spam heuristics that keep evolving and that you don't control -- blocks your self-hosted outbound emails without warning. That's why you have example of skilled admins who know what they're doing and had a working self-hosted setup for years suddenly getting their emails rejected: https://www.tablix.org/~avian/blog/archives/2019/04/google_is_eating_our_mail/ https://www.tablix.org/~avian/blog/archives/2019/04/google_i... As to the contradicting anecdotes about the difficulties of self-hosting email, the issue is that the conversation shares the same unstated environments in comments about Uber or umbrellas that affects how the writer perceives the truth or relevance of their anecdote. - "The problems of self-hosted email getting blocked is overstated. I've been doing it and it's working fine." - "I'm not sure what value Uber provides. Taxi services have smartphone apps." - "I'm not sure why people use umbrellas. Every time I walk outside, it's not raining." As an example of evangelists and advice-givers not noticing their unstated environments... Back in October 2017, a commenter (lucb1e) argued[1] that I was exaggerating the difficulties of reliably sending email but a year later in 2019, he eventually confirmed the same difficulties! [2] [1] https://news.ycombinator.com/item?id=15525505 https://news.ycombinator.com/item?id=15525505 [2] https://news.ycombinator.com/item?id=19757607 https://news.ycombinator.com/item?id=19757607

noduerme 4y ago
Oh yeah. This is very true, and getting worse every year. I've had this discussion on HN before. It's gotten to the point where I've had to have my clients and their corporate lawyers go to bat against mail providers to maintain deliverability. No mail provider has any interest whatsoever in allowing an independent mailserver to continue delivering now. So far, legal threats have worked when push came to shove against certain networks. But I imagine the difficulty is only going to increase.
- sirnoggin 4y ago
  Im extremely curious the legal precedents you used to accomplish this, particularly around forcing certain providers to un-spam or un-block your emails. What was the condition your legals found in order to do that?

cubesnooper 4y ago

> You need a dedicated box, not a VPS. … Unlike some people are saying, you should never do this off a VPS if you have an interest in keeping the email secure and functioning for a long time. I agree that hosting a mail server directly on a VPS compromises privacy and control. But there’s a better alternative: use VPSes for cheap static IPs, while hosting the server locally on hardware you physically control, using WireGuard tunnels and port forwarding to connect things. Port forward incoming SMTP over WireGuard to your real MX, and use MTA‐STS and DANE so that as many senders as possible will TLS‐encrypt mail they send you. Have your outgoing SMTP server handle DKIM signing, then send it out via WireGuard so it looks like it came from the VPS, while enforcing TLS encryption. The VPS won’t be able to forge mail from you without your DKIM keys. It won’t be able to read your outgoing mail due to TLS. It won’t be able to read incoming mail that’s TLS encrypted. It will be able to read unencrypted mail, but the big providers that follow MTA‐STS will abort if the VPS attempts to block encrypted connections. This has the added benefit of reducing your dependence on an external provider (the VPS company) for server setup. If you’re unhappy with a particular provider, just switch to another one. The issues associated with sending email from a brand new IP will be there, but you won’t have to set up complicated infrastructure on the new host, only a few WireGuard tunnels and firewall rules.

noduerme 4y ago
That's a cool idea I'd never considered. And it's easy to set up. But then the main thing you're getting out of the VPS is the static IP. The main reason I was advising against VPS's, besides obvious vulnerabilities, was that their IP blocks get banned all the time. If you want to run a mailserver for a long time you need to cultivate that IP address's reputation for years, and you don't want it to be anywhere sketchy. If you're paying for that, why not just pay for a static IP at home?
- cubesnooper 4y ago
  > the main thing you're getting out of the VPS is the static IP. Yes, that and RDNS. > If you're paying for that, why not just pay for a static IP at home? That’s a good question. I too hear that mail providers consider IP blocks assigned to VPS providers less trustworthy than others. The reasons I don’t take the ISP/dedicated server route, aside from price, are: • VPS providers are not tied to my physical location. If I move, I probably can’t take my ISP’s static IP with me (I may even move to somewhere they don’t service). Conversely, if I want to switch away from a local ISP, the selection of alternatives is extremely limited. • Risk of neighboring IPs reducing the reputation of the block exists with server companies and local ISPs as well. I concede that the problem is probably worse with VPSes, but I hope to mitigate it somewhat by avoiding bottom‐of‐the‐barrel providers and by the fact that my own IP will never be used to spam. • I’m somewhat worried about the possibility of DDOS, and VPS companies provide a lot of cheap bandwidth, so in case of attack I might be able to salvage the situation with careful firewalling on the VPS.
  - noduerme 4y ago
    That all makes sense. Although in my experience, if you're really being careful about not sending spam, you've got a lot more to worry about from your VPS being blacklisted than your own IP personally. >> I’m somewhat worried about the possibility of DDOS The one time I got severely dDoS'd, because I'd let a friend run a tiny static website off my server that attracted that kind of attention, the hosting company I was with shut my account down immediately and asked for $5000 in reparations. I had to backdoor into the server and salvage whatever I could. That was a hardened box in a military grade facility. I don't think a VPS is going to be kind. Push comes to shove, if it's in your house you can pull the cable.