5 ms·

Show HN: Heimdall – Self-managed email alias/forwarding service

130 points by fterh 7y ago

JohnsonY 7y ago
nice job.
andrewkdinh 7y ago
You might consider using a different name, as there’s already a pretty popular dashboard called Heimdall [1] [1] https://github.com/linuxserver/Heimdall https://github.com/linuxserver/Heimdall
- Lammy 7y ago
  My mind went to the Kerberos implementation, despite the slight spelling difference: https://github.com/heimdal/heimdal https://github.com/heimdal/heimdal
- honopu 7y ago
  There’s also a db proxy you can run yourself that sits in front of Postgres and maybe others also named Heimdall. It’s like varnish for your db.
- milankragujevic 7y ago
  There is also the tool for flashing Samsung Android phone firmware on Linux... https://gitlab.com/BenjaminDobell/Heimdall https://gitlab.com/BenjaminDobell/Heimdall
zAy0LfpBZLC8mAC 7y ago
> With Heimdall, you completely own and manage your data and the service. No feature limitations or having to trust a third-party company with your data. > Pre-requisites: You need to own a domain and have an AWS account. For reasonable use cases, you should not exceed AWS's free tier (which is very generous). Erm ... wut?
- justusthane 7y ago
  Unless you physically own a server you have to put your data somewhere, whether that somewhere is Digital Ocean, some shared hosting provider, AWS, or whatever else. You're still more in control of your data than you would be using a closed third-party product. Sure it would be nice to have other options in addition to AWS, but I don't think those two statements are contradictory. Also, I don't know if it is, but the data stored on AWS could be encrypted by the app, in which case you're really not trusting AWS.
  - OJFord 7y ago
    My charitable reading of GP's comment is as a reaction to the fact that the project's been designed in such a way (Serverless :tm:) that AWS is required; a pre-requisite, not an example. From the first quote you might think that it was the deployer's choice where to put it, including on one's own hardware. I don't know, however, what you'd do instead of SES.
  - zAy0LfpBZLC8mAC 7y ago
    > Unless you physically own a server you have to put your data somewhere, whether that somewhere is Digital Ocean, some shared hosting provider, AWS, or whatever else. You're still more in control of your data than you would be using a closed third-party product. The claim was not "you are more in control than with a closed third-party product". The claim was "No [...] having to trust a third-party company with your data.". When you have to use AWS, then you evidently have to trust a third-party company with your data, unless you happen to be AWS. And not only do you have to trust a third party, you even have to trust one particular third party with no alternative if they misbehave somehow. That's pretty close to using a closed third-party product, if you ask me. I mean, really, you are using a closed third-party product--it just happens to be the infrastructure that you build on. > Sure it would be nice to have other options in addition to AWS, but I don't think those two statements are contradictory. So, AWS is either not a third party or could not access your data, no matter how much they wanted to? Or what other alternative do you see to make those statements not contradictory? > Also, I don't know if it is, but the data stored on AWS could be encrypted by the app, in which case you're really not trusting AWS. Wut? Am I just completely misunderstanding what this does? This uses SES, a service by AWS that handles your emails, right? As in: That speaks SMTP for you, and thus sees the plain text of the emails, right? And then, somewhere there is code that handles those emails that runs on machines that AWS has physical access to, right? As in: Code that AWS can trace and modify however they like, right? As in: Code where AWS trivially could extract any possible encryption keys from, right? Unless I am completely misunderstanding this ... what would possibly stop AWS from reading all your emails if they wanted to?
johnebgd 7y ago
I’m confused. Why do you want this? You don’t trust a provider to forward your email? Email isn’t a trusted method of communication anyway.
- sm4rk0 7y ago
  Providers usually don't give you (unlimited number of) aliases.
  - whatsmyusername 7y ago
    If all you want to do is forward all mail for a domain somewhere you can easily do that at most domain registrars. I use this with Monicker.
  - wrboyce 7y ago
    Pretty sure Fastmail do (provide unlimited aliases). They certainly provide wildcard aliases, including domain wildcards.
- fterh 7y ago
  The trust reason is theoretical - in practice, I would trust most decently large services especially for unimportant marketing emails (main use case). My primary motivation in doing this was to learn to use AWS and Serverless framework and also because I really enjoy working on pet projects :) Could you explain why email isn't trusted? It's encrypted (vs SMS) so I'd imagine it's a far more secure way of communicating sensitive information (e.g. bank statements or one time passwords).
kazinator 7y ago
I wrote and use a web service called Tamarind for managing throw-away mail aliases: http://www.kylheku.com/cgit/tamarind/tree/README http://www.kylheku.com/cgit/tamarind/tree/README It integrates into Apache as a CGI program serving up a web UI for managing your aliases. It works by managing the content of an alias file read by your mail server. Authentication of the webUI is via IMAP or SASL. Each throw-away alias is associated with a memo in which you can have text and URL's (that get rendered into links), and a creation time. You can regex search through the aliases, edit the memo fields, rearrange their order and delete them.
- wrboyce 7y ago
  Not sure how this is relevant to OP’s post aside as an attempted hijack. You don’t even address the differences between the product you’re pushing and OP’s. If you want to Show HN something, make your own post; this is just rude.
mmcclure 7y ago
This is a really cool project, so I don't mean to be overly negative, but personally this workflow feels quite a bit more laborious than just having a catch-all email address. Before signing up for a service, I need to email myself to get an address to use for the service? I have all emails for my domain route to me, so when I use a service I just do [service-name]@my-domain.com. If a bad actor gets a hold of it I set up an inbox filter or black hole the email address at the service level. The big advantage of this project seems to be that you can reply, but I've found that a huge proportion of these email aliases are inbound only for me. I'm using GSuite for my personal email but I've been considering Fastmail. Just checked and it looks like they also support sending from those catchall aliases: https://www.fastmail.com/help/receive/alias-catchall.html https://www.fastmail.com/help/receive/alias-catchall.html
- mekster 7y ago
  For groups of people, you can also do, [service-name]@[user-name].my-domain.com to provide everyone the same capability. Email aliases ([user-name]+[service-name]@my-domain.com) isn't the best solution when spammers can remove the alias part (+[service-name]) and you can't know who leaked it.
- caymanjim 7y ago
  You're going to get an astronomical amount of junk mail with a catchall email address. Google is great at filtering spam, but not perfect. If you're running your own mail server, you're going to have a hard time dealing with it, even if you use SpamAssassin and other tools. It's also going to get worse over time, because every address that accepts delivery is going to get added to a database for future spamming. I use Postfix and ViMbAdmin to manage my whitelist via a simple web UI, and I don't find it to be onerous. I don't sign up for new services every day, and it takes about ten seconds to add or delete a service-specific alias.
  - moonlighter 7y ago
    I disagree with the "astronomical amount of junk mail" statement. I've been using FastMail with a catchall email address for years, and get very little spam; most of which is correctly classified as such (I did have to 'train' FastMail for a while with custom spam/no spam folders though).
christefano 7y ago
FYI, the developer has a writeup about the design behind this project: https://medium.com/@fabianterh/how-i-built-heimdall-an-open-source-personal-email-guardian-68e306d172d1 https://medium.com/@fabianterh/how-i-built-heimdall-an-open-... At first I was confused and trying to figure out what AWS services Heimdall uses to work, and this was the section that explained it: Infrastructure I’m using AWS’s Simple Email Service (SES) to send and receive emails, S3 for storage, and Lambda functions for serverless computing. Here’s how it works: All received emails trigger SES to store the email as a file in a S3 bucket, which triggers a Lambda function. Depending on the email, one of several things could happen: 1. The email gets forwarded to your personal email address 2. The email gets forwarded to the original sender (when you reply) 3. A command is invoked by you (e.g. to generate a new alias) 4. Nothing happens (when someone emails an invalid/disabled alias) I chose to use AWS for practical reasons: I’m totally new to cloud computing, and AWS being the most popular cloud computing service means it is easier to find guides and resources online.
- airstrike 7y ago
  > Infrastructure > I’m using AWS’s Simple Email Service (SES) to send and receive emails, S3 for storage, and Lambda functions for serverless computing. Here’s how it works: > All received emails trigger SES to store the email as a file in a S3 bucket, which triggers a Lambda function. Depending on the email, one of several things could happen: > 1. The email gets forwarded to your personal email address > 2. The email gets forwarded to the original sender (when you reply) > 3. A command is invoked by you (e.g. to generate a new alias) > 4. Nothing happens (when someone emails an invalid/disabled alias) > I chose to use AWS for practical reasons: I’m totally new to cloud computing, and AWS being the most popular cloud computing service means it is easier to find guides and resources online.
  - daseiner1 7y ago
    thank you
- fterh 7y ago
  Thanks for linking! I would have linked directly to the blog post but I believe it's against the rules of Show HN, so I chose to include a link in the readme instead!
rodneyg_ 7y ago
I love you. Thanks for this
albertgoeswoof 7y ago
And if you don’t want to host this yourself there’s https://IdBloc.co https://IdBloc.co
thatha7777 7y ago
Self-managed email means using SES? The dream of the 90s exclaims “ouch”. It’s a realistic choice and I am not judging it, but still, ouch.
- fterh 7y ago
  I was born in the 90s so I'm not familiar with the "dream of the 90s" haha - care to explain?
  - mhluongo 7y ago
    We don't run our own infrastructure anymore :(
    - Keverw 7y ago
      Yeah and seems like the big providers don't like self hosted emails. I know someone who runs a hosting company using cPanel and his clients email go to spam a bunch even though not spammy emails, his answer is to just pay extra for Google Apps and that running a email server is too complicated. Not sure how true that is... For my own future projects I plan to run everything in containers, Node, etc so not PHP/MySQL so still need to host the email somewhere, kinda hate the idea of paying a third party and wonder how they would handle shared inboxes(like piping email to a script like you can with a cPanel server). Probably other email solutions though to look into though but haven't looked into it too much yet but kinda hate how we have to rely on large providers it seems for email. Then sounds like people running WordPress or support help desk scripts with contact forms forwarding to their Gmail, etc is a problem too... Those services think your server is the source of spam. But maybe it's worth paying for a third party email server for your staff and also a service with APIs for programmable sending/receiving since you don't want emails to customers like password resets, receipts, etc going to spam. I noticed even when I mark someones emails as not spam they keep going to spam, I don't check my spam daily but sometimes they have to send me a IM to let me know they emailed them... I feel like might be forced paying though instead of self handling email. Kinda feels like giving the mafia some extortion money for protection though in away. Plus email is a bit broken in the first place, open and all is great but opens itself up to abuse to spammers and scammers.
jedberg 7y ago
> Known Limitations > Currently, attachments are not supported. That's kind of a biggie. What happens when someone sends an attachment? Does it bounce? Are they warned? Do I get a notification? Is it silently dropped? From reading the code it seems like it just doesn't include the attachment and then deletes it from S3?
- fterh 7y ago
  Right now, silently dropped. Yeah, this is an issue I'm planning to work on. The reason I chose to release it before supporting attachments is that for my use case for receiving marketing emails (this project is primarily for my personal use, rather than as public software), there are almost never attachments.
  - thedance 7y ago
    This statement is pretty vague. Exactly what is not supported? Virtually all email is in multipart mime format.
whatsmyusername 7y ago
This service is offered by most domain registrars out of the box.
VvR-Ox 7y ago
I like the idea but I do not understand why someone would want something self-managed/hosted and then use AMZN SES to send/receive mail and S3 to save mail/attachments. 1. AMZN has access to your mail (inc. your contacts) so you could just use any other service you do not trust. 2. Why would I process mail just so save it on another machine and not do both on the same server? Probably it has to do with "serverless" (you have to use at least 2 "servers" now, don't you?) but maybe I am just missing the point.

130 points by fterh 7y ago

nice job.

You might consider using a different name, as there’s already a pretty popular dashboard called Heimdall [1] [1] https://github.com/linuxserver/Heimdall https://github.com/linuxserver/Heimdall

Lammy 7y ago
My mind went to the Kerberos implementation, despite the slight spelling difference: https://github.com/heimdal/heimdal https://github.com/heimdal/heimdal
honopu 7y ago
There’s also a db proxy you can run yourself that sits in front of Postgres and maybe others also named Heimdall. It’s like varnish for your db.
milankragujevic 7y ago
There is also the tool for flashing Samsung Android phone firmware on Linux... https://gitlab.com/BenjaminDobell/Heimdall https://gitlab.com/BenjaminDobell/Heimdall

zAy0LfpBZLC8mAC 7y ago

> With Heimdall, you completely own and manage your data and the service. No feature limitations or having to trust a third-party company with your data. > Pre-requisites: You need to own a domain and have an AWS account. For reasonable use cases, you should not exceed AWS's free tier (which is very generous). Erm ... wut?

justusthane 7y ago
Unless you physically own a server you have to put your data somewhere, whether that somewhere is Digital Ocean, some shared hosting provider, AWS, or whatever else. You're still more in control of your data than you would be using a closed third-party product. Sure it would be nice to have other options in addition to AWS, but I don't think those two statements are contradictory. Also, I don't know if it is, but the data stored on AWS could be encrypted by the app, in which case you're really not trusting AWS.
- OJFord 7y ago
  My charitable reading of GP's comment is as a reaction to the fact that the project's been designed in such a way (Serverless :tm:) that AWS is required; a pre-requisite, not an example. From the first quote you might think that it was the deployer's choice where to put it, including on one's own hardware. I don't know, however, what you'd do instead of SES.
- zAy0LfpBZLC8mAC 7y ago
  > Unless you physically own a server you have to put your data somewhere, whether that somewhere is Digital Ocean, some shared hosting provider, AWS, or whatever else. You're still more in control of your data than you would be using a closed third-party product. The claim was not "you are more in control than with a closed third-party product". The claim was "No [...] having to trust a third-party company with your data.". When you have to use AWS, then you evidently have to trust a third-party company with your data, unless you happen to be AWS. And not only do you have to trust a third party, you even have to trust one particular third party with no alternative if they misbehave somehow. That's pretty close to using a closed third-party product, if you ask me. I mean, really, you are using a closed third-party product--it just happens to be the infrastructure that you build on. > Sure it would be nice to have other options in addition to AWS, but I don't think those two statements are contradictory. So, AWS is either not a third party or could not access your data, no matter how much they wanted to? Or what other alternative do you see to make those statements not contradictory? > Also, I don't know if it is, but the data stored on AWS could be encrypted by the app, in which case you're really not trusting AWS. Wut? Am I just completely misunderstanding what this does? This uses SES, a service by AWS that handles your emails, right? As in: That speaks SMTP for you, and thus sees the plain text of the emails, right? And then, somewhere there is code that handles those emails that runs on machines that AWS has physical access to, right? As in: Code that AWS can trace and modify however they like, right? As in: Code where AWS trivially could extract any possible encryption keys from, right? Unless I am completely misunderstanding this ... what would possibly stop AWS from reading all your emails if they wanted to?

johnebgd 7y ago

I’m confused. Why do you want this? You don’t trust a provider to forward your email? Email isn’t a trusted method of communication anyway.

sm4rk0 7y ago
Providers usually don't give you (unlimited number of) aliases.
- whatsmyusername 7y ago
  If all you want to do is forward all mail for a domain somewhere you can easily do that at most domain registrars. I use this with Monicker.
- wrboyce 7y ago
  Pretty sure Fastmail do (provide unlimited aliases). They certainly provide wildcard aliases, including domain wildcards.
fterh 7y ago
The trust reason is theoretical - in practice, I would trust most decently large services especially for unimportant marketing emails (main use case). My primary motivation in doing this was to learn to use AWS and Serverless framework and also because I really enjoy working on pet projects :) Could you explain why email isn't trusted? It's encrypted (vs SMS) so I'd imagine it's a far more secure way of communicating sensitive information (e.g. bank statements or one time passwords).

kazinator 7y ago

I wrote and use a web service called Tamarind for managing throw-away mail aliases: http://www.kylheku.com/cgit/tamarind/tree/README http://www.kylheku.com/cgit/tamarind/tree/README It integrates into Apache as a CGI program serving up a web UI for managing your aliases. It works by managing the content of an alias file read by your mail server. Authentication of the webUI is via IMAP or SASL. Each throw-away alias is associated with a memo in which you can have text and URL's (that get rendered into links), and a creation time. You can regex search through the aliases, edit the memo fields, rearrange their order and delete them.

wrboyce 7y ago
Not sure how this is relevant to OP’s post aside as an attempted hijack. You don’t even address the differences between the product you’re pushing and OP’s. If you want to Show HN something, make your own post; this is just rude.

mmcclure 7y ago

This is a really cool project, so I don't mean to be overly negative, but personally this workflow feels quite a bit more laborious than just having a catch-all email address. Before signing up for a service, I need to email myself to get an address to use for the service? I have all emails for my domain route to me, so when I use a service I just do [service-name]@my-domain.com. If a bad actor gets a hold of it I set up an inbox filter or black hole the email address at the service level. The big advantage of this project seems to be that you can reply, but I've found that a huge proportion of these email aliases are inbound only for me. I'm using GSuite for my personal email but I've been considering Fastmail. Just checked and it looks like they also support sending from those catchall aliases: https://www.fastmail.com/help/receive/alias-catchall.html https://www.fastmail.com/help/receive/alias-catchall.html

mekster 7y ago
For groups of people, you can also do, [service-name]@[user-name].my-domain.com to provide everyone the same capability. Email aliases ([user-name]+[service-name]@my-domain.com) isn't the best solution when spammers can remove the alias part (+[service-name]) and you can't know who leaked it.
caymanjim 7y ago
You're going to get an astronomical amount of junk mail with a catchall email address. Google is great at filtering spam, but not perfect. If you're running your own mail server, you're going to have a hard time dealing with it, even if you use SpamAssassin and other tools. It's also going to get worse over time, because every address that accepts delivery is going to get added to a database for future spamming. I use Postfix and ViMbAdmin to manage my whitelist via a simple web UI, and I don't find it to be onerous. I don't sign up for new services every day, and it takes about ten seconds to add or delete a service-specific alias.
- moonlighter 7y ago
  I disagree with the "astronomical amount of junk mail" statement. I've been using FastMail with a catchall email address for years, and get very little spam; most of which is correctly classified as such (I did have to 'train' FastMail for a while with custom spam/no spam folders though).

christefano 7y ago

FYI, the developer has a writeup about the design behind this project: https://medium.com/@fabianterh/how-i-built-heimdall-an-open-source-personal-email-guardian-68e306d172d1 https://medium.com/@fabianterh/how-i-built-heimdall-an-open-... At first I was confused and trying to figure out what AWS services Heimdall uses to work, and this was the section that explained it: Infrastructure I’m using AWS’s Simple Email Service (SES) to send and receive emails, S3 for storage, and Lambda functions for serverless computing. Here’s how it works: All received emails trigger SES to store the email as a file in a S3 bucket, which triggers a Lambda function. Depending on the email, one of several things could happen: 1. The email gets forwarded to your personal email address 2. The email gets forwarded to the original sender (when you reply) 3. A command is invoked by you (e.g. to generate a new alias) 4. Nothing happens (when someone emails an invalid/disabled alias) I chose to use AWS for practical reasons: I’m totally new to cloud computing, and AWS being the most popular cloud computing service means it is easier to find guides and resources online.

airstrike 7y ago
> Infrastructure > I’m using AWS’s Simple Email Service (SES) to send and receive emails, S3 for storage, and Lambda functions for serverless computing. Here’s how it works: > All received emails trigger SES to store the email as a file in a S3 bucket, which triggers a Lambda function. Depending on the email, one of several things could happen: > 1. The email gets forwarded to your personal email address > 2. The email gets forwarded to the original sender (when you reply) > 3. A command is invoked by you (e.g. to generate a new alias) > 4. Nothing happens (when someone emails an invalid/disabled alias) > I chose to use AWS for practical reasons: I’m totally new to cloud computing, and AWS being the most popular cloud computing service means it is easier to find guides and resources online.
- daseiner1 7y ago
  thank you
fterh 7y ago
Thanks for linking! I would have linked directly to the blog post but I believe it's against the rules of Show HN, so I chose to include a link in the readme instead!

rodneyg_ 7y ago

I love you. Thanks for this

albertgoeswoof 7y ago

And if you don’t want to host this yourself there’s https://IdBloc.co https://IdBloc.co

thatha7777 7y ago

Self-managed email means using SES? The dream of the 90s exclaims “ouch”. It’s a realistic choice and I am not judging it, but still, ouch.

fterh 7y ago
I was born in the 90s so I'm not familiar with the "dream of the 90s" haha - care to explain?
- mhluongo 7y ago
  We don't run our own infrastructure anymore :(
  - Keverw 7y ago
    Yeah and seems like the big providers don't like self hosted emails. I know someone who runs a hosting company using cPanel and his clients email go to spam a bunch even though not spammy emails, his answer is to just pay extra for Google Apps and that running a email server is too complicated. Not sure how true that is... For my own future projects I plan to run everything in containers, Node, etc so not PHP/MySQL so still need to host the email somewhere, kinda hate the idea of paying a third party and wonder how they would handle shared inboxes(like piping email to a script like you can with a cPanel server). Probably other email solutions though to look into though but haven't looked into it too much yet but kinda hate how we have to rely on large providers it seems for email. Then sounds like people running WordPress or support help desk scripts with contact forms forwarding to their Gmail, etc is a problem too... Those services think your server is the source of spam. But maybe it's worth paying for a third party email server for your staff and also a service with APIs for programmable sending/receiving since you don't want emails to customers like password resets, receipts, etc going to spam. I noticed even when I mark someones emails as not spam they keep going to spam, I don't check my spam daily but sometimes they have to send me a IM to let me know they emailed them... I feel like might be forced paying though instead of self handling email. Kinda feels like giving the mafia some extortion money for protection though in away. Plus email is a bit broken in the first place, open and all is great but opens itself up to abuse to spammers and scammers.

jedberg 7y ago

> Known Limitations > Currently, attachments are not supported. That's kind of a biggie. What happens when someone sends an attachment? Does it bounce? Are they warned? Do I get a notification? Is it silently dropped? From reading the code it seems like it just doesn't include the attachment and then deletes it from S3?

fterh 7y ago
Right now, silently dropped. Yeah, this is an issue I'm planning to work on. The reason I chose to release it before supporting attachments is that for my use case for receiving marketing emails (this project is primarily for my personal use, rather than as public software), there are almost never attachments.
- thedance 7y ago
  This statement is pretty vague. Exactly what is not supported? Virtually all email is in multipart mime format.

whatsmyusername 7y ago

This service is offered by most domain registrars out of the box.

VvR-Ox 7y ago

I like the idea but I do not understand why someone would want something self-managed/hosted and then use AMZN SES to send/receive mail and S3 to save mail/attachments. 1. AMZN has access to your mail (inc. your contacts) so you could just use any other service you do not trust. 2. Why would I process mail just so save it on another machine and not do both on the same server? Probably it has to do with "serverless" (you have to use at least 2 "servers" now, don't you?) but maybe I am just missing the point.