4 ms·

When I tried a similar project, every email I sent from a self-hosted email server were considered spam from Gmail. In some cases, websites weren't even allowin

When I tried a similar project, every email I sent from a self-hosted email server were considered spam from Gmail. In some cases, websites weren't even allowing email registration with a self-hosted email. Is there any general advice about avoiding such problems?

mjl- 4y ago
Get a machine/VM at a hosting provider that takes their network seriously. Typically not the cheapest you can find. And make sure all the DNS config is set correctly. Mox helps you with that. I've done a few mox quickstarts at fresh VM's, and mail was accepted by gmail just fine.
creshal 4y ago
Email reputation is a complex topic, but generally as long as you • don't use an IP range classified as "consumer" (known dynamic IP DSL/cable ranges) • set up SPF • set up DKIM • don't send actual spam …it generally gets through. Mox is so far the easiest solution for SPF/DKIM I've seen, but if you don't mind spending a weekend tinkering, it should be solveable with any email stack. > In some cases, websites weren't even allowing email registration with a self-hosted email. I never ran into that issue. Not much you can do there, other than writing the owner angry emails via gmail.
- danpalmer 4y ago
  I did all of this when self hosting for a few years. It was mostly fine, but for a couple of undelivered emails. Those were just enough that I lost trust in the system and eventually stopped running my own email. People mostly expect email to be fully reliable. It’s not, but that doesn’t stop people treating it like that. As a result, even if you only fail to deliver 1 in 1000 emails, that could have significant consequences. Gmail et. al have a high enough deliverability rate that most people can get away with treating it as reliable. I’m not saying people shouldn’t self host, but I think they need to be very aware of how it will change their trust of email.
  - 1vuio0pswjnm7 4y ago
    This sums up the problem quite nicely. IMO, what needs to happen is for people to see the 1 in 1000 as a problem with Gmail et al. rather than a problem with someone hosting their own STMP server. Generally speaking, people I know who self-host their email do not have problems. However, inevitably there is the 1 in 1000. IMO, unreached recipients using Gmail et al. are likely to see the problem as with the self-hosted sender, not their own third party email provider. One idea is to have a backup for emergencies. That is, a Gmail user could have an alternative, self-hosted means of receiving email on port 587 or 2525. By default, this mail drop accepts mail from no one. When there's a 1 in 1000 problem, the recipient could add the sender to a list of acceptable senders, either temporaily or permanently. The sender can then use the alternative instead of Gmail.
    - creshal 4y ago
      To be honest, in the last 20 years or so of self-hosting emails, the most common failure points were • Gmail marked an email as spam, but still delivered it. Easily cleared up, no hard feelings on either side. • Someone else self-hosts some abysmally maintained Exchange 2003 setup that just about barely works with GMail and Office365, after the MSP irresponsible for it gets enough verbal beatings. Impossible to get to work.
  - drpixie 4y ago
    Your right, but email is never fully reliable. How many times have you lost things into your gmail/outlook/etc's spam folder? And with the amount of genuine spam being sent FROM GMAIL to my backup gmail address, it's unthinkable to actually check through the spam folder with any regularity.
- pmoriarty 4y ago
  > don't use an IP range classified as "consumer" (known dynamic IP DSL/cable ranges) How can you tell if the IP you use is or has been in one of those ranges?
  - gboone 4y ago
    I think "consumer" is being contrasted with a business contract offering static ip. You should know because it is currently blocked - found by using mxtoolbox.com for example. Or, suspect it will be because the ip to your residence is not a designated static ip. (In practice it may be ipv6 and never changes but applying a PTR to the ip requires assistance from the ISP and it would likely be outside of the contract.)
- drpixie 4y ago
  > In some cases, websites weren't even allowing email registration with a self-hosted email. I've seen that once or twice, but it was because my self-hosted is on a .one domain. There are (not surprisingly) a few organisations out there that validate against the old set of top-level domains (.com, .net, etc, and the country codes). They just won't accept any of the new top-levels as real!!
callesgg 4y ago
You need to get a proper self owned ip range with your own WHOIS info for your mail server. On google I found that one can get a IPv4 /30 subnet for about 11 USD a year.
- scandox 4y ago
  Not a huge amount of hosting companies support Bring Your Own IP do they?
  - xolox 4y ago
    By definition most of them won't because it's a complicated mess.
  - tomatocracy 4y ago
    More do than you'd think. But for IPv4, true BYOIP isn't really possible for less than a /24 block, which is overkill for just selfhosting personal email. It also usually involves needing to set up a company to own/lease the block if you haven't already got one.
- msravi 4y ago
  Can you link to this please? Everything I see seems much more expensive...
dspillett 4y ago
The largest issue is often where you are hosting it – the address/range might not be well trusted due to either how it is categorised (if it is address known to be assigned to residential accounts then all mail may be dismissed as junk from a compromised host without any inspection) or who has used it before you (if using an inexpensive VPS (or dedicated server) provider the addresses you have may have been used by a spammer in the past). I've had very few problems with my home server sending mail, but I'm with an ISP more known for commercial users (so they take keeping their ranges clean seriously, and they aren't listed in IP databases as residential), I've had these addresses for over a decade (back when you could get a /29 fairly easily if not using a cheap ISP) so no one has sent junk from them (actually a friend was infected by something that sent web junk out, but that goes over the primary address not the range the mail service is on), and so forth. The easiest way to solve this if you aren't as lucky as I am, is unfortunately not a free solution: send via a 3rd party relay such as companies like mxroute offer – you still control your entire mail server, your store all the mail, etc, but someone else deals with final delivery of outgoing mail (a good provider will multiple outgoing feeds themselves and deal with resending in response to some types of rejection, as well as making sure all their outgoing routes are reputation clean). For small volumes this is actually quite cheap, potentially far less expensive than your admin time dealing with delivery failures yourself. Other common issues are not having SPF records set properly, or not implementing DKIM signing. Both of these are now essential and have been for some time, whether you send directly or via a 3rd party MTA, so if you didn't verify those that could be your issue. Also, some mail receivers don't trust brand-new domains (the newest domain my main mail server regularly sends for was registered in 2010) so if you tested the mail server with a throwaway domain that could be a significant part of the problem. If you want to use a throwaway domain for testing, at very least wait until after the 5-day “domain tasting” grace period. Also pick a throwaway domain name that is relatively long and otherwise unlikely to have been used before (and unlikely to have ever been used as a from address in junk messages).

mjl- 4y ago

Get a machine/VM at a hosting provider that takes their network seriously. Typically not the cheapest you can find. And make sure all the DNS config is set correctly. Mox helps you with that. I've done a few mox quickstarts at fresh VM's, and mail was accepted by gmail just fine.

creshal 4y ago

Email reputation is a complex topic, but generally as long as you • don't use an IP range classified as "consumer" (known dynamic IP DSL/cable ranges) • set up SPF • set up DKIM • don't send actual spam …it generally gets through. Mox is so far the easiest solution for SPF/DKIM I've seen, but if you don't mind spending a weekend tinkering, it should be solveable with any email stack. > In some cases, websites weren't even allowing email registration with a self-hosted email. I never ran into that issue. Not much you can do there, other than writing the owner angry emails via gmail.

danpalmer 4y ago
I did all of this when self hosting for a few years. It was mostly fine, but for a couple of undelivered emails. Those were just enough that I lost trust in the system and eventually stopped running my own email. People mostly expect email to be fully reliable. It’s not, but that doesn’t stop people treating it like that. As a result, even if you only fail to deliver 1 in 1000 emails, that could have significant consequences. Gmail et. al have a high enough deliverability rate that most people can get away with treating it as reliable. I’m not saying people shouldn’t self host, but I think they need to be very aware of how it will change their trust of email.
- 1vuio0pswjnm7 4y ago
  This sums up the problem quite nicely. IMO, what needs to happen is for people to see the 1 in 1000 as a problem with Gmail et al. rather than a problem with someone hosting their own STMP server. Generally speaking, people I know who self-host their email do not have problems. However, inevitably there is the 1 in 1000. IMO, unreached recipients using Gmail et al. are likely to see the problem as with the self-hosted sender, not their own third party email provider. One idea is to have a backup for emergencies. That is, a Gmail user could have an alternative, self-hosted means of receiving email on port 587 or 2525. By default, this mail drop accepts mail from no one. When there's a 1 in 1000 problem, the recipient could add the sender to a list of acceptable senders, either temporaily or permanently. The sender can then use the alternative instead of Gmail.
  - creshal 4y ago
    To be honest, in the last 20 years or so of self-hosting emails, the most common failure points were • Gmail marked an email as spam, but still delivered it. Easily cleared up, no hard feelings on either side. • Someone else self-hosts some abysmally maintained Exchange 2003 setup that just about barely works with GMail and Office365, after the MSP irresponsible for it gets enough verbal beatings. Impossible to get to work.
- drpixie 4y ago
  Your right, but email is never fully reliable. How many times have you lost things into your gmail/outlook/etc's spam folder? And with the amount of genuine spam being sent FROM GMAIL to my backup gmail address, it's unthinkable to actually check through the spam folder with any regularity.
pmoriarty 4y ago
> don't use an IP range classified as "consumer" (known dynamic IP DSL/cable ranges) How can you tell if the IP you use is or has been in one of those ranges?
- gboone 4y ago
  I think "consumer" is being contrasted with a business contract offering static ip. You should know because it is currently blocked - found by using mxtoolbox.com for example. Or, suspect it will be because the ip to your residence is not a designated static ip. (In practice it may be ipv6 and never changes but applying a PTR to the ip requires assistance from the ISP and it would likely be outside of the contract.)
drpixie 4y ago
> In some cases, websites weren't even allowing email registration with a self-hosted email. I've seen that once or twice, but it was because my self-hosted is on a .one domain. There are (not surprisingly) a few organisations out there that validate against the old set of top-level domains (.com, .net, etc, and the country codes). They just won't accept any of the new top-levels as real!!

callesgg 4y ago

You need to get a proper self owned ip range with your own WHOIS info for your mail server. On google I found that one can get a IPv4 /30 subnet for about 11 USD a year.

scandox 4y ago
Not a huge amount of hosting companies support Bring Your Own IP do they?
- xolox 4y ago
  By definition most of them won't because it's a complicated mess.
- tomatocracy 4y ago
  More do than you'd think. But for IPv4, true BYOIP isn't really possible for less than a /24 block, which is overkill for just selfhosting personal email. It also usually involves needing to set up a company to own/lease the block if you haven't already got one.
msravi 4y ago
Can you link to this please? Everything I see seems much more expensive...

dspillett 4y ago

The largest issue is often where you are hosting it – the address/range might not be well trusted due to either how it is categorised (if it is address known to be assigned to residential accounts then all mail may be dismissed as junk from a compromised host without any inspection) or who has used it before you (if using an inexpensive VPS (or dedicated server) provider the addresses you have may have been used by a spammer in the past). I've had very few problems with my home server sending mail, but I'm with an ISP more known for commercial users (so they take keeping their ranges clean seriously, and they aren't listed in IP databases as residential), I've had these addresses for over a decade (back when you could get a /29 fairly easily if not using a cheap ISP) so no one has sent junk from them (actually a friend was infected by something that sent web junk out, but that goes over the primary address not the range the mail service is on), and so forth. The easiest way to solve this if you aren't as lucky as I am, is unfortunately not a free solution: send via a 3rd party relay such as companies like mxroute offer – you still control your entire mail server, your store all the mail, etc, but someone else deals with final delivery of outgoing mail (a good provider will multiple outgoing feeds themselves and deal with resending in response to some types of rejection, as well as making sure all their outgoing routes are reputation clean). For small volumes this is actually quite cheap, potentially far less expensive than your admin time dealing with delivery failures yourself. Other common issues are not having SPF records set properly, or not implementing DKIM signing. Both of these are now essential and have been for some time, whether you send directly or via a 3rd party MTA, so if you didn't verify those that could be your issue. Also, some mail receivers don't trust brand-new domains (the newest domain my main mail server regularly sends for was registered in 2010) so if you tested the mail server with a throwaway domain that could be a significant part of the problem. If you want to use a throwaway domain for testing, at very least wait until after the 5-day “domain tasting” grace period. Also pick a throwaway domain name that is relatively long and otherwise unlikely to have been used before (and unlikely to have ever been used as a from address in junk messages).