18 ms·

DIY Mail

173 points by zdw 5y ago

meling 5y ago
This was a great read and goes to show how broken email has become… I often wonder if we will ever migrate to something better collectively, or we end up on a variety of different communication platforms.
- vbezhenar 5y ago
  I very rarely use e-mail for anything. Most interactions are via whatsapp, telegram. Not sure if it counts as a better alternative.
- jiggunjer 5y ago
  Yes, if you think about it it's weird how govt. application forms have an email field. Perhaps email should be nationalized.
  - alex_duf 5y ago
    nationalized by which government?
    - jiggunjer 5y ago
      All of them ofc. Instead of a few companies controlling an essential service, what's stopping a government from offering every citizen an address at yourid@govmail.country?
    - 0xbadcafebee 5y ago
      Here's a list of nationalized postal services: https://en.wikipedia.org/wiki/List_of_national_postal_services https://en.wikipedia.org/wiki/List_of_national_postal_servic... Nationalized digital mail isn't that much of a stretch. You can still run private digital mail, just like you can hire private physical mail couriers. But probably it won't happen. Capitalist countries prefer private industry does things. E-mail could become a regulated utility, though.
      - alex_duf 5y ago
        My point (albeit subtly made) was that there isn't a central authority that controls emails, you can't just buy emails and then run it as a national service. And that's before you look into which government runs what.
pydry 5y ago
These stories have hammered home two points for me: * You probably shouldnt use email for sensitive communications ever again. * While it's not worth self hosting a mail server to send and receive email there's still a huge privacy value in hosting email storage and using a service that only passes on emails via POP3/SMTP (ideally with minimal logs).
simonebrunozzi 5y ago
Should be Email, not Mail.
YPPH 5y ago
Pretty much reflects my view. Mail is hard to set up properly, but it's not ridiculously hard to set up. Anyone with average Unix administration skills can do it. I might add that OpenSMTPd is a lot easier than postfix and makes a dangerous open relay misconfiguration hard to do, so it's my recommendation to the uninitiated. It requires maintenance attention, but not a ridiculous level like some suggest. What is ridiculous is the effort you will go to to stop your connections being bounced, or (worse) your emails going to spam folders. It's this third reason why I use Fastmail to run my email. Technical accumen is of some but limited assistance here. In essence you will you will be struggling to build an IP reputation with low traffic flow. This article has some interesting insights on this topic.
- akerl_ 5y ago
  Saying it’s not ridiculously hard to set up your own mail server, unless you want recipients to actually get your emails when you send them, feels a bit irrelevant? Most things are easy if you’re willing to accept an end result that doesn’t do one of the two (3? Maybe? Send, receive, search?) primary functions.
  - walrus01 5y ago
    The frustrating part is that you can have the most impeccable Postfix configuration for your smtpd, and your system has never once emitted a single spam outbound to anyone, but your mail to office365 or other places might disappear into a black hole. I know probably 4 or 5 people who are far more than capable of running their own mail server and have given up on it these days, bowing to the inexorable market pressure of gsuite/google workspace/office365 for their MX, because they just can't be bothered by having outbound mail deliver-ability issues. These are people who were running smtpd for regional ISPs in 1997. It almost seems like the google and microsoft spam filters are a bit of a protection racket, forcing people to buy their services. The author of the article covers this: > If you self-host email, you will run into an elevated number of delivery problems. That is a guarantee. Fully implementing trust and authentication measures will help, but it will not eliminate the problem because providers weight their IP reputation information more than your ability to configure DKIM correctly. Whether or not it becomes a noticeable problem for you depends on a few factors, and it's hard to say in advance without just trying it. Federated systems like email tend to rely on a high degree of informal social infrastructure. Unfortunately, as email has become centralized into a small number of major providers, that infrastructure has mostly decayed. It was not that long ago that you could often resolve a deliverability problem with a politely worded note to postmaster @ the problematic destination server. Today, many email providers have some kind of method of contacting them, but I have never once received a response or even evidence of action due to one of these messages... both for complaints of abuse on their end and deliverability problems [3].
deleted 5y ago
[deleted]
walrus01 5y ago
I actually clicked this thread after reading the first part of the article, and saw that the author has already made 3 very important points (in ASCII bullet points) that I came here to say basically the same thing. All of this is extremely important in terms of real world outbound mail deliverability. >Do not cheap out on providers. Run your mailserver in the most reputable IP space you can find. Most likely you will use some type of cloud instance or VPS (probably in such a way that the line between the two is blurry). Unfortunately, per-month cost tends to correlate directly with IP reputation quality, as does name recognition. But neither are any guarantee. AWS has plenty of IP reputation problems. Sometimes a little-known, brand-new VPS provider can be a surprisingly good option because they got their IP space off of some staid corporation that maintained an impeccable security program. Do some research to try and find out what kind of results people get from different providers. > From an IP reputation perspective, consistency is key. Mailservers and IP reputation are like credit cards and your credit report: once you open one, you want to keep it as long as possible so that it will build a good IP reputation. Do not move your mailserver around between providers to save money, it will result in headache. > Do not run a mailserver off of IP space that is not intended for commercial services. For example, running a mailserver on a residential ISP is a terrible idea (if it even works, many residential ISPs drop outbound on port 25). You will be guaranteed to have IP reputation problems and, moreover, IP space allocated to end-users usually gets reported to a "policy block list" such as the one Spamhaus maintains... meaning your outbound email will be blocked by recipients simply because it's coming from an IP that should not have mailservers, so it's most likely to be from a compromised end-user device. =================== Now to add my own 2 cents: If you are serious about doing this, find a medium sized long-established ISP where it is still small enough that you can get in contact with the people who run the routers. And that ISP should be one that really cares about its IP space reputation. If you can afford it, anywhere with long-held IP space belonging to its ASN that has never had $5/month VPS customers on it, or any form of shared hosting environment, is much more likely to have "clean" IP space.
- geocrasher 5y ago
  Great. Can you suggest such a provider, or 3?
rsync 5y ago
"Of course requiring a validated telephone number as part of identity is a substantial compromise on privacy and effectively eliminates identity compartmentalization ..." That's what the "2FA mule" is for. It's a stock android phone with no google account and no apps installed except for "SMS Forwarder"[1]. It is configured to forward all SMS to an email address via encrypted SMTP. This means that I can receive these 2FA codes anywhere I have Internet access - such as an airplane or newly arrived in a foreign country where my SIM card does not work. The "2FA Mule" itself is plugged in at my office in a corner. I'm not employing this for anything sensitive but it's interesting to consider that I can use SMS based 2FA while divorcing it from my day to day SIM identity ... [1] https://play.google.com/store/apps/details?id=com.frzinapps.smsforward&hl=en_US&gl=US https://play.google.com/store/apps/details?id=com.frzinapps....
- commoner 5y ago
  A free and open source, ad-free alternative to SMS Forwarder is SMS Gate: - F-Droid: https://f-droid.org/en/packages/com.github.axet.smsgate/ https://f-droid.org/en/packages/com.github.axet.smsgate/ - GitLab: https://gitlab.com/axet/android-sms-gate https://gitlab.com/axet/android-sms-gate
g5095 5y ago
One of the biggest hurdles I've experienced is that a number of large players in the industry (think G..argantuan) systematically flag private mail servers as spam almost as a matter of course and it's up to you to prove your innocence. There is a clear financial benefit for this sort of behavior as it pushes people back onto their services.
- sbuk 5y ago
  The bulk of this post discusses IP reputation which is the biggest hurdle to self hosting. It is the first line defence for spam/phishing/fraudulent mail. SPF and DKIM can align for DMARC, but if the IP has a bad, or no reputation, chances are the message will be bounced. Think of it like having a low or no credit score. As an example, if a “cold” IP address suddenly starts sending mail, mail security will take the (arguably cynical) view that chances are it’s some sort of spam. It even more cynical to assume that this is done as a money grab, shepherding users to big mail services, when in fact, it’s the best way to prevent bulk mail.
  - jcrawfordor 5y ago
    The irony too is that, if you run a mailserver of substantial size, you will become very sympathetic to the other side on this. Dropping email from new/suspect IPs will tremendously reduce your user complaints about spam. It's not really a winnable situation.
- upofadown 5y ago
  Monopolies gotta monopoly. In the end though they can't lose too much of their incoming email before people will move to email services that are reliable.
Autilio 5y ago
jmconfuzeus 5y ago
Scrolling on this web page is really annoying.
tommek4077 5y ago
To whoever always ends up in spam: stop using cloud providers. Spammers use cloud providers, so you get banned, too. I am having my own mail server for over 15 years, no it is not a lot of maintenance. I had to create some spf records at some point in time and thats it.
- jiggunjer 5y ago
  I read some ISP will block port 25, while other mail providers block residential IPs. It seems there is no rule that always applies.
  - tommek4077 5y ago
    The rule is: no residential ip and no cloud provider.
    - Nextgrid 5y ago
      What's left then? Anything easy to access would've been used for spam in the past. This only really leaves services with a high cost/barrier to entry such as leased lines?
      - tommek4077 5y ago
        Real servers, they don't even need to be expensive. But spammers would not create 12 month contracts. Bonus: as much traffic as you like and REAL hardware. Starts at 6$/month.
- amelius 5y ago
  One problem is still if you're sending email on behalf of other people. One of those people might do something (maybe even something innocent) that will put your entire service on a blacklist.
  - NetOpWibby 5y ago
    I think this has happened to me.
normaler 5y ago
i never had an issue with the big mail servers. All the smaller ones broke me. T-online.de etc.
- brongondwana 5y ago
  Agree - the really big providers are pretty good because you can get in contact with somebody who gives a shit at the other end and can fix things up. They do so much volume that they can afford somebody who cares about it. The really painful things are either places that subscribe to dodgy blocklists and never check their configuration, or... yeah, places like t-online.de and free.fr.
messo 5y ago
How is IPv6 coverage in the email provider landscape these days? I guess it would be easier to build up a decent IP reputation if it was on a "virgin" IPv6 address used only for this purpose, and not tainted by prior use etc. IPv6-only for email is probably still a pipe dream, I guess...
- brongondwana 5y ago
  It's pretty bad - nobody is really doing it. On the plus side, since it's all pretty fresh - you can do things like "MUST be DKIM signed from an aligned reverse DNS" on IPv6 and have a chance of it working.
  - LeonM 5y ago
    > It's pretty bad - nobody is really doing it. I'm curious how you came to that conclusion. Messo's question above made me curious, so I just did a quick query on the database of Mailhardener. >99% of email coming from Gmail is delivered over IPv6 (from ~10k samples). This appears to be provider specific though. For example: Yahoo seems to mail exclusively over IPv4, Zoho also IPv4. Most exchange based services appear to prefer IPv6. Microsoft 365 also uses IPv6. So your conclusion doesn't really reflect what I am seeing in our database. Obviously, we process mostly DMARC reports, so maybe regular email is more IPv4 biased to support mail services that are not IPv6 capable.
- LeonM 5y ago
  > I guess it would be easier to build up a decent IP reputation if it was on a "virgin" IPv6 address used only for this purpose, and not tainted by prior use etc. IP addresses (especially IPv6) are ephemeral, email providers know that. Just about every email service has a TTL on IP reputation. Usually, if an email is DKIM signed (and aligned) then domain reputation is used as the key indicator, the sender IP becomes less relevant in that situation. Obviously, each implementation is different. There are no silver bullets here. But what I'm trying to say here is: do not worry about inheriting a 'tainted' IP address. With the limited number of IPv4 addresses available, and the huge volume of spam, just about every publicly available IP would have been 'tainted' by now.
sam_goody 5y ago
I run a mail server [Ubuntu + Postfix + Dovecot + Sieve], and then use a service , (for me - Postmark[1]) to actually send the mail. The integration is extremely easy to setup [2], and solves the deliver-ability issue. Most of these services also take care of the DKIM and DMARC setup, so it is much easier to get things right. And they all offer debugging tools if something isn't sending or returning correctly. On the other hand, they are not in privacy abusing position that Gmail is, and I also don't have to worry that they will close my account with no recourse, as Gmail once did to me. The biggest downside is no email forwarding. There is no way to do email forwarding using a service, since they only send mail from approved senders. (There are Bad Idea solutions, such as rewriting headers with SRS. Don't.) And of course, I need to pay for sending. Mailgun used to have a generous free tier, and Amazon SES is decent at $1/10K mails, but things add up. I chose Postmark for many reasons, but the pricing of different companies is worth looking into if you are sending enough mail. (Sendgrid and Mailchimp Mandril, for example, have a unlimited per deliveree charge, that could conceivably be cheaper for some use cases.) OT, the only reason that I support PoP3 is for Gmail: The only way to "check an external account" to write and send mails in Gmail is PoP3. [1]: postmarkapp.com [2]: https://postmarkapp.com/support/article/832-can-i-configure-postfix-to-send-through-postmark https://postmarkapp.com/support/article/832-can-i-configure-...
- narag 5y ago
  So in case you don't want to send email, just receive for you, there's not much problem... ???
  - z0f 5y ago
    Correct. You can receive emails on any IP, sending is the only time reputation comes into play.
WhyNotHugo 5y ago
Another big issue is silently rejected emails. A client of mine had corporate clients who relied on MS's hosted email solution. Often emails would be accepted from our MTA onto theirs, but never show up in inboxes or spamboxes. They were just silently discarded, even though the IP had pretty good reputation (and had been used to send email for years). Eventually, the problem got so bad they just had to switch to a hosted email solution, since there was no way to work around it. I would have insisted the other company deal with it, but they wouldn't pay my client if the invoices weren't emailed, so we didn't really have a choice.
- southerntofu 5y ago
  The problem in this case is Microsoft. They've been known to do this for years in order to reinforce their email oligopoly. Unfortunately instances where your business lost money is the only thing Nation State regulators care about, but in this specific instance, you could have sued Microsoft and made a ton of money and maybe paved the way for their stopping to do so entirely from fear of further fines. Maybe it's not too late. You'd be doing all of us a major service.
  - tiahura 5y ago
    “in order to reinforce their email oligopoly.” Cite?
    - southerntofu 5y ago
      No specific source for the purpose. That's just classic mafia/oligopoly playbook. A more explicit version would be: "that's a nice email server you got there, would be a shame if your mail were silently dropped; If you just pay for my services that certainly wouldn't happen" Of course, they can't do that with other big players (like Gmail) so they're allowlisted, because they would be afraid of lawsuits and their users would complain about not receiving email from Gmail. By applying this to smaller providers only, they maintain plausible deniability that the problem lies on the other side, a sentiment which is reinforced by maintaining their tech support's lower levels unaware of the scheme (the only ones you can talk to for months after initial complaint).
akshayrajp 5y ago
Does anyone know how to go about building a blog like that? What SSG/theme does the author use? Looks really good.
- jcrawfordor 5y ago
  It's a short and pretty ugly Python script that reads a directory of text files and plugs them into Jinja2 templates (virtually all of the actual logic is in the templates). I find myself using this pattern pretty often because I am very much not a web developer and find most SSGs to be insufferably complicated for my use cases. When I write new posts, I use a shell script that runs them through aspell, then runs the site generator, does a simple concatenation to build the email version, and calls pandoc to build the secret hidden pdf version.
  - akshayrajp 5y ago
    Thank you for describing how you build/develop your blog. Maybe I'll implement some of that for myself.
emptybottle 5y ago
Running mail is complex, and maybe it’s not the best use of your time personally. But please stop dissuading engineers from attempting to run their own mail infrastructure if desired. More people getting familiar with how reliable email systems work is a good thing. With regard to sending mail, a few things go a long way. Check your IPs on dnsbls before sending email from them. Don't expect to send mail successfully from a public cloud address. have at least one backup mail system that isn't listed in any dnsbl. Use separate IP addresses for transactional email, user email and bulk email. Deploy SPF, DKIM and DMARC for your domains.
- deltarholamda 5y ago
  Totally agree. I think nearly everybody of the HN stripe should run an email server, if only on a vanity domain. There has to be some kind of pushback against centralizing email among Google, Microsoft, et al. Email is fundamental to the open Internet. It's the "killer app," as they used to say in ye olden tymes. It does not belong to 3-4 international corporations, it belongs to us. The article pooh-poohs iRedmail and some others, and while he's not wrong, I can say that iRedmail is about as close as you can get to learning all about the inner workings of a mail delivery system and still be reasonably one-click install. It is opinionated to some degree, but it doesn't lock you in. I've used it for years and have been happy. Unfortunately, delivery problems (as the author noted) meant I have to use Mailgun for their relay service. I think this is a fight we should continue to have. It's worth it.
  - nickdothutton 5y ago
    I feel the same way. I wonder if users of hackernews would be interested in a community (semi)Pubnix and email service. Traditional multiuser system with thousands of concurrent users catered for and all the “old” services plus some new ones.
    - lambdaba 5y ago
      Would this be like a modern sdf.org? Sounds cool.
kkjjkgjjgg 5y ago
Reading that article, I had the thought that maybe it is time to abandon mail for all communication and only use it for the required signup processes with web sites. I had actually given up on mail months ago, in the sense of not considering it a reliable channel anymore. But I haven't been strict about enforcing it, and instead live with the risk of missing important messages.
lazyweb 5y ago
A lot of good points. I've been running my own private mail server for almost seven years [1], it's been a very valuable learning experience. [1] https://jschumacher.info/2021/05/running-a-private-mail-server-for-six-years-easy-peasy/ https://jschumacher.info/2021/05/running-a-private-mail-serv...
softwarebeware 5y ago
This pretty much convinced me not to run my own mail server. Thanks! Anyone using Protonmail with a custom domain?
- ipodopt 5y ago
  I have used both pm and migadu with custom domains. Miguda ends being much better on that front feature, pricing, and ease of use wise. https://www.migadu.com/pricing/ https://www.migadu.com/pricing/ You do lose the layer of encryption protonmail provides. But it is essentially boils down Encryption at Rest and e2e when between two protonmail users. It can be mimicked by pulling down the mail from the servers to your machine and using pgp.
- 3np 5y ago
  It needs to be said that you can get most of the benefits of your own mail-server and still route outgoing SMTP through a provider like mailroute/sendgrid/mailgun/AWS/even google. It’s not an all- or nothing question. Similarly, if you’re worried about the risk of potentially losing incoming email if you have issues, you can keep your existing mail provider choice as fallback in the DNS and fetch+delete mail going there into your primary mailbox every 5 min or so using mbsync.
andris9 5y ago
When I started building https://wildduck.email https://wildduck.email (a completely independent email server implementation) then my goal was to build sonething that is deeply integrated, there would be only a single way of doing things, no file based storage, and management would be done over a REST API. I don’t regret a single decision from back then, the non-unix way has proven to have a lot of upside. PS. It’s not mentioned often but when you send out emails you need to send at least 100 emails per IP per large destination _every day_ (100 to Gmail, 100 to O365 etc). Otherwise you’re not even on the radar and can’t build any reputation. You need to have a constant mail flow going out for the big guys to take you even remotely seriously.

173 points by zdw 5y ago

This was a great read and goes to show how broken email has become… I often wonder if we will ever migrate to something better collectively, or we end up on a variety of different communication platforms.

vbezhenar 5y ago
I very rarely use e-mail for anything. Most interactions are via whatsapp, telegram. Not sure if it counts as a better alternative.
jiggunjer 5y ago
Yes, if you think about it it's weird how govt. application forms have an email field. Perhaps email should be nationalized.
- alex_duf 5y ago
  nationalized by which government?
  - jiggunjer 5y ago
    All of them ofc. Instead of a few companies controlling an essential service, what's stopping a government from offering every citizen an address at yourid@govmail.country?
  - 0xbadcafebee 5y ago
    Here's a list of nationalized postal services: https://en.wikipedia.org/wiki/List_of_national_postal_services https://en.wikipedia.org/wiki/List_of_national_postal_servic... Nationalized digital mail isn't that much of a stretch. You can still run private digital mail, just like you can hire private physical mail couriers. But probably it won't happen. Capitalist countries prefer private industry does things. E-mail could become a regulated utility, though.
    - alex_duf 5y ago
      My point (albeit subtly made) was that there isn't a central authority that controls emails, you can't just buy emails and then run it as a national service. And that's before you look into which government runs what.

pydry 5y ago

These stories have hammered home two points for me: * You probably shouldnt use email for sensitive communications ever again. * While it's not worth self hosting a mail server to send and receive email there's still a huge privacy value in hosting email storage and using a service that only passes on emails via POP3/SMTP (ideally with minimal logs).

simonebrunozzi 5y ago

Should be Email, not Mail.

YPPH 5y ago

Pretty much reflects my view. Mail is hard to set up properly, but it's not ridiculously hard to set up. Anyone with average Unix administration skills can do it. I might add that OpenSMTPd is a lot easier than postfix and makes a dangerous open relay misconfiguration hard to do, so it's my recommendation to the uninitiated. It requires maintenance attention, but not a ridiculous level like some suggest. What is ridiculous is the effort you will go to to stop your connections being bounced, or (worse) your emails going to spam folders. It's this third reason why I use Fastmail to run my email. Technical accumen is of some but limited assistance here. In essence you will you will be struggling to build an IP reputation with low traffic flow. This article has some interesting insights on this topic.

akerl_ 5y ago
Saying it’s not ridiculously hard to set up your own mail server, unless you want recipients to actually get your emails when you send them, feels a bit irrelevant? Most things are easy if you’re willing to accept an end result that doesn’t do one of the two (3? Maybe? Send, receive, search?) primary functions.
- walrus01 5y ago
  The frustrating part is that you can have the most impeccable Postfix configuration for your smtpd, and your system has never once emitted a single spam outbound to anyone, but your mail to office365 or other places might disappear into a black hole. I know probably 4 or 5 people who are far more than capable of running their own mail server and have given up on it these days, bowing to the inexorable market pressure of gsuite/google workspace/office365 for their MX, because they just can't be bothered by having outbound mail deliver-ability issues. These are people who were running smtpd for regional ISPs in 1997. It almost seems like the google and microsoft spam filters are a bit of a protection racket, forcing people to buy their services. The author of the article covers this: > If you self-host email, you will run into an elevated number of delivery problems. That is a guarantee. Fully implementing trust and authentication measures will help, but it will not eliminate the problem because providers weight their IP reputation information more than your ability to configure DKIM correctly. Whether or not it becomes a noticeable problem for you depends on a few factors, and it's hard to say in advance without just trying it. Federated systems like email tend to rely on a high degree of informal social infrastructure. Unfortunately, as email has become centralized into a small number of major providers, that infrastructure has mostly decayed. It was not that long ago that you could often resolve a deliverability problem with a politely worded note to postmaster @ the problematic destination server. Today, many email providers have some kind of method of contacting them, but I have never once received a response or even evidence of action due to one of these messages... both for complaints of abuse on their end and deliverability problems [3].

deleted 5y ago

[deleted]

walrus01 5y ago

I actually clicked this thread after reading the first part of the article, and saw that the author has already made 3 very important points (in ASCII bullet points) that I came here to say basically the same thing. All of this is extremely important in terms of real world outbound mail deliverability. >Do not cheap out on providers. Run your mailserver in the most reputable IP space you can find. Most likely you will use some type of cloud instance or VPS (probably in such a way that the line between the two is blurry). Unfortunately, per-month cost tends to correlate directly with IP reputation quality, as does name recognition. But neither are any guarantee. AWS has plenty of IP reputation problems. Sometimes a little-known, brand-new VPS provider can be a surprisingly good option because they got their IP space off of some staid corporation that maintained an impeccable security program. Do some research to try and find out what kind of results people get from different providers. > From an IP reputation perspective, consistency is key. Mailservers and IP reputation are like credit cards and your credit report: once you open one, you want to keep it as long as possible so that it will build a good IP reputation. Do not move your mailserver around between providers to save money, it will result in headache. > Do not run a mailserver off of IP space that is not intended for commercial services. For example, running a mailserver on a residential ISP is a terrible idea (if it even works, many residential ISPs drop outbound on port 25). You will be guaranteed to have IP reputation problems and, moreover, IP space allocated to end-users usually gets reported to a "policy block list" such as the one Spamhaus maintains... meaning your outbound email will be blocked by recipients simply because it's coming from an IP that should not have mailservers, so it's most likely to be from a compromised end-user device. =================== Now to add my own 2 cents: If you are serious about doing this, find a medium sized long-established ISP where it is still small enough that you can get in contact with the people who run the routers. And that ISP should be one that really cares about its IP space reputation. If you can afford it, anywhere with long-held IP space belonging to its ASN that has never had $5/month VPS customers on it, or any form of shared hosting environment, is much more likely to have "clean" IP space.

geocrasher 5y ago
Great. Can you suggest such a provider, or 3?

rsync 5y ago

"Of course requiring a validated telephone number as part of identity is a substantial compromise on privacy and effectively eliminates identity compartmentalization ..." That's what the "2FA mule" is for. It's a stock android phone with no google account and no apps installed except for "SMS Forwarder"[1]. It is configured to forward all SMS to an email address via encrypted SMTP. This means that I can receive these 2FA codes anywhere I have Internet access - such as an airplane or newly arrived in a foreign country where my SIM card does not work. The "2FA Mule" itself is plugged in at my office in a corner. I'm not employing this for anything sensitive but it's interesting to consider that I can use SMS based 2FA while divorcing it from my day to day SIM identity ... [1] https://play.google.com/store/apps/details?id=com.frzinapps.smsforward&hl=en_US&gl=US https://play.google.com/store/apps/details?id=com.frzinapps....

commoner 5y ago
A free and open source, ad-free alternative to SMS Forwarder is SMS Gate: - F-Droid: https://f-droid.org/en/packages/com.github.axet.smsgate/ https://f-droid.org/en/packages/com.github.axet.smsgate/ - GitLab: https://gitlab.com/axet/android-sms-gate https://gitlab.com/axet/android-sms-gate

g5095 5y ago

One of the biggest hurdles I've experienced is that a number of large players in the industry (think G..argantuan) systematically flag private mail servers as spam almost as a matter of course and it's up to you to prove your innocence. There is a clear financial benefit for this sort of behavior as it pushes people back onto their services.

sbuk 5y ago
The bulk of this post discusses IP reputation which is the biggest hurdle to self hosting. It is the first line defence for spam/phishing/fraudulent mail. SPF and DKIM can align for DMARC, but if the IP has a bad, or no reputation, chances are the message will be bounced. Think of it like having a low or no credit score. As an example, if a “cold” IP address suddenly starts sending mail, mail security will take the (arguably cynical) view that chances are it’s some sort of spam. It even more cynical to assume that this is done as a money grab, shepherding users to big mail services, when in fact, it’s the best way to prevent bulk mail.
- jcrawfordor 5y ago
  The irony too is that, if you run a mailserver of substantial size, you will become very sympathetic to the other side on this. Dropping email from new/suspect IPs will tremendously reduce your user complaints about spam. It's not really a winnable situation.
upofadown 5y ago
Monopolies gotta monopoly. In the end though they can't lose too much of their incoming email before people will move to email services that are reliable.

Autilio 5y ago

jmconfuzeus 5y ago

Scrolling on this web page is really annoying.

tommek4077 5y ago

To whoever always ends up in spam: stop using cloud providers. Spammers use cloud providers, so you get banned, too. I am having my own mail server for over 15 years, no it is not a lot of maintenance. I had to create some spf records at some point in time and thats it.

jiggunjer 5y ago
I read some ISP will block port 25, while other mail providers block residential IPs. It seems there is no rule that always applies.
- tommek4077 5y ago
  The rule is: no residential ip and no cloud provider.
  - Nextgrid 5y ago
    What's left then? Anything easy to access would've been used for spam in the past. This only really leaves services with a high cost/barrier to entry such as leased lines?
    - tommek4077 5y ago
      Real servers, they don't even need to be expensive. But spammers would not create 12 month contracts. Bonus: as much traffic as you like and REAL hardware. Starts at 6$/month.
amelius 5y ago
One problem is still if you're sending email on behalf of other people. One of those people might do something (maybe even something innocent) that will put your entire service on a blacklist.
- NetOpWibby 5y ago
  I think this has happened to me.

normaler 5y ago

i never had an issue with the big mail servers. All the smaller ones broke me. T-online.de etc.

brongondwana 5y ago
Agree - the really big providers are pretty good because you can get in contact with somebody who gives a shit at the other end and can fix things up. They do so much volume that they can afford somebody who cares about it. The really painful things are either places that subscribe to dodgy blocklists and never check their configuration, or... yeah, places like t-online.de and free.fr.

messo 5y ago

How is IPv6 coverage in the email provider landscape these days? I guess it would be easier to build up a decent IP reputation if it was on a "virgin" IPv6 address used only for this purpose, and not tainted by prior use etc. IPv6-only for email is probably still a pipe dream, I guess...

brongondwana 5y ago
It's pretty bad - nobody is really doing it. On the plus side, since it's all pretty fresh - you can do things like "MUST be DKIM signed from an aligned reverse DNS" on IPv6 and have a chance of it working.
- LeonM 5y ago
  > It's pretty bad - nobody is really doing it. I'm curious how you came to that conclusion. Messo's question above made me curious, so I just did a quick query on the database of Mailhardener. >99% of email coming from Gmail is delivered over IPv6 (from ~10k samples). This appears to be provider specific though. For example: Yahoo seems to mail exclusively over IPv4, Zoho also IPv4. Most exchange based services appear to prefer IPv6. Microsoft 365 also uses IPv6. So your conclusion doesn't really reflect what I am seeing in our database. Obviously, we process mostly DMARC reports, so maybe regular email is more IPv4 biased to support mail services that are not IPv6 capable.
LeonM 5y ago
> I guess it would be easier to build up a decent IP reputation if it was on a "virgin" IPv6 address used only for this purpose, and not tainted by prior use etc. IP addresses (especially IPv6) are ephemeral, email providers know that. Just about every email service has a TTL on IP reputation. Usually, if an email is DKIM signed (and aligned) then domain reputation is used as the key indicator, the sender IP becomes less relevant in that situation. Obviously, each implementation is different. There are no silver bullets here. But what I'm trying to say here is: do not worry about inheriting a 'tainted' IP address. With the limited number of IPv4 addresses available, and the huge volume of spam, just about every publicly available IP would have been 'tainted' by now.

sam_goody 5y ago

I run a mail server [Ubuntu + Postfix + Dovecot + Sieve], and then use a service , (for me - Postmark[1]) to actually send the mail. The integration is extremely easy to setup [2], and solves the deliver-ability issue. Most of these services also take care of the DKIM and DMARC setup, so it is much easier to get things right. And they all offer debugging tools if something isn't sending or returning correctly. On the other hand, they are not in privacy abusing position that Gmail is, and I also don't have to worry that they will close my account with no recourse, as Gmail once did to me. The biggest downside is no email forwarding. There is no way to do email forwarding using a service, since they only send mail from approved senders. (There are Bad Idea solutions, such as rewriting headers with SRS. Don't.) And of course, I need to pay for sending. Mailgun used to have a generous free tier, and Amazon SES is decent at $1/10K mails, but things add up. I chose Postmark for many reasons, but the pricing of different companies is worth looking into if you are sending enough mail. (Sendgrid and Mailchimp Mandril, for example, have a unlimited per deliveree charge, that could conceivably be cheaper for some use cases.) OT, the only reason that I support PoP3 is for Gmail: The only way to "check an external account" to write and send mails in Gmail is PoP3. [1]: postmarkapp.com [2]: https://postmarkapp.com/support/article/832-can-i-configure-postfix-to-send-through-postmark https://postmarkapp.com/support/article/832-can-i-configure-...

narag 5y ago
So in case you don't want to send email, just receive for you, there's not much problem... ???
- z0f 5y ago
  Correct. You can receive emails on any IP, sending is the only time reputation comes into play.

WhyNotHugo 5y ago

Another big issue is silently rejected emails. A client of mine had corporate clients who relied on MS's hosted email solution. Often emails would be accepted from our MTA onto theirs, but never show up in inboxes or spamboxes. They were just silently discarded, even though the IP had pretty good reputation (and had been used to send email for years). Eventually, the problem got so bad they just had to switch to a hosted email solution, since there was no way to work around it. I would have insisted the other company deal with it, but they wouldn't pay my client if the invoices weren't emailed, so we didn't really have a choice.

southerntofu 5y ago
The problem in this case is Microsoft. They've been known to do this for years in order to reinforce their email oligopoly. Unfortunately instances where your business lost money is the only thing Nation State regulators care about, but in this specific instance, you could have sued Microsoft and made a ton of money and maybe paved the way for their stopping to do so entirely from fear of further fines. Maybe it's not too late. You'd be doing all of us a major service.
- tiahura 5y ago
  “in order to reinforce their email oligopoly.” Cite?
  - southerntofu 5y ago
    No specific source for the purpose. That's just classic mafia/oligopoly playbook. A more explicit version would be: "that's a nice email server you got there, would be a shame if your mail were silently dropped; If you just pay for my services that certainly wouldn't happen" Of course, they can't do that with other big players (like Gmail) so they're allowlisted, because they would be afraid of lawsuits and their users would complain about not receiving email from Gmail. By applying this to smaller providers only, they maintain plausible deniability that the problem lies on the other side, a sentiment which is reinforced by maintaining their tech support's lower levels unaware of the scheme (the only ones you can talk to for months after initial complaint).

akshayrajp 5y ago

Does anyone know how to go about building a blog like that? What SSG/theme does the author use? Looks really good.

jcrawfordor 5y ago
It's a short and pretty ugly Python script that reads a directory of text files and plugs them into Jinja2 templates (virtually all of the actual logic is in the templates). I find myself using this pattern pretty often because I am very much not a web developer and find most SSGs to be insufferably complicated for my use cases. When I write new posts, I use a shell script that runs them through aspell, then runs the site generator, does a simple concatenation to build the email version, and calls pandoc to build the secret hidden pdf version.
- akshayrajp 5y ago
  Thank you for describing how you build/develop your blog. Maybe I'll implement some of that for myself.

emptybottle 5y ago

Running mail is complex, and maybe it’s not the best use of your time personally. But please stop dissuading engineers from attempting to run their own mail infrastructure if desired. More people getting familiar with how reliable email systems work is a good thing. With regard to sending mail, a few things go a long way. Check your IPs on dnsbls before sending email from them. Don't expect to send mail successfully from a public cloud address. have at least one backup mail system that isn't listed in any dnsbl. Use separate IP addresses for transactional email, user email and bulk email. Deploy SPF, DKIM and DMARC for your domains.

deltarholamda 5y ago
Totally agree. I think nearly everybody of the HN stripe should run an email server, if only on a vanity domain. There has to be some kind of pushback against centralizing email among Google, Microsoft, et al. Email is fundamental to the open Internet. It's the "killer app," as they used to say in ye olden tymes. It does not belong to 3-4 international corporations, it belongs to us. The article pooh-poohs iRedmail and some others, and while he's not wrong, I can say that iRedmail is about as close as you can get to learning all about the inner workings of a mail delivery system and still be reasonably one-click install. It is opinionated to some degree, but it doesn't lock you in. I've used it for years and have been happy. Unfortunately, delivery problems (as the author noted) meant I have to use Mailgun for their relay service. I think this is a fight we should continue to have. It's worth it.
- nickdothutton 5y ago
  I feel the same way. I wonder if users of hackernews would be interested in a community (semi)Pubnix and email service. Traditional multiuser system with thousands of concurrent users catered for and all the “old” services plus some new ones.
  - lambdaba 5y ago
    Would this be like a modern sdf.org? Sounds cool.

kkjjkgjjgg 5y ago

Reading that article, I had the thought that maybe it is time to abandon mail for all communication and only use it for the required signup processes with web sites. I had actually given up on mail months ago, in the sense of not considering it a reliable channel anymore. But I haven't been strict about enforcing it, and instead live with the risk of missing important messages.

lazyweb 5y ago

A lot of good points. I've been running my own private mail server for almost seven years [1], it's been a very valuable learning experience. [1] https://jschumacher.info/2021/05/running-a-private-mail-server-for-six-years-easy-peasy/ https://jschumacher.info/2021/05/running-a-private-mail-serv...

softwarebeware 5y ago

This pretty much convinced me not to run my own mail server. Thanks! Anyone using Protonmail with a custom domain?

ipodopt 5y ago
I have used both pm and migadu with custom domains. Miguda ends being much better on that front feature, pricing, and ease of use wise. https://www.migadu.com/pricing/ https://www.migadu.com/pricing/ You do lose the layer of encryption protonmail provides. But it is essentially boils down Encryption at Rest and e2e when between two protonmail users. It can be mimicked by pulling down the mail from the servers to your machine and using pgp.
3np 5y ago
It needs to be said that you can get most of the benefits of your own mail-server and still route outgoing SMTP through a provider like mailroute/sendgrid/mailgun/AWS/even google. It’s not an all- or nothing question. Similarly, if you’re worried about the risk of potentially losing incoming email if you have issues, you can keep your existing mail provider choice as fallback in the DNS and fetch+delete mail going there into your primary mailbox every 5 min or so using mbsync.

andris9 5y ago

When I started building https://wildduck.email https://wildduck.email (a completely independent email server implementation) then my goal was to build sonething that is deeply integrated, there would be only a single way of doing things, no file based storage, and management would be done over a REST API. I don’t regret a single decision from back then, the non-unix way has proven to have a lot of upside. PS. It’s not mentioned often but when you send out emails you need to send at least 100 emails per IP per large destination _every day_ (100 to Gmail, 100 to O365 etc). Otherwise you’re not even on the radar and can’t build any reputation. You need to have a constant mail flow going out for the big guys to take you even remotely seriously.