6 ms·

Show HN: A fake SMTP server for software integration testing

127 points by aeaa3 2y ago

This is a side project of mine. Use this as your SMTP server in a test environment to guarantee that your users don't receive test emails. Looking for feedback, especially on the security side.

sam_perez 2y ago
Really cool. I couldn't find it documented anywhere, but if you save the unique (guid) url generated, how long is that available for? "Forever"?
- aeaa3 2y ago
  Yes, currently these are never deleted. I will probably update it to delete accounts that have been inactive for a year or something.
netsec1989 2y ago
Wow, this and all the various other options mentioned look very useful. Like others, I have hacked together configurations to do this in the past and would have loved to have these instead. I had no idea on the availability of all the various options, which leads to the question: Where would one find examples of similar network test servers for other protocols/functions? For example: - A SAML IdP - Define accounts and complete a login, allow debug of request/response. - A DNS Server - Define local domains and records, control whether Internet domains are resolvable or just local ‘corporate’ records. - Syslog Server - Catch logs and make them temporarily available. (usually syslogd works for this, but maybe test harnesses have advantages). - SNMP - Trap destination to capture/show alerts
atum47 2y ago
Down for me
TomasEkeli 2y ago
hugged to death? anyway - i got a great 429 back with a suggested try-after -time. it made me smile in appreciation! well done!
- theginger 2y ago
  That's the wrong status code The HTTP 429 Too Many Requests response status code indicates the user has sent too many requests in a given amount of time ("rate limiting"). Unless the server believes you as an individual user were sending too many it should not have been a 429 If the server was unable to handle the volume of requests more generally it should have been a 503 which also supports Retry-After
  - aeaa3 2y ago
    The 429 response is sent when rate-limiting (based on IP address). Either something is wrong with my rate-limiting code or there are many people behind a single IP address. Anyway the limits are increased now.
    - theginger 2y ago
      Possible your application isn't seeing real ips, maybe is seeing the IP of a load balancer or similar.
  - xp84 2y ago
    Maybe one request is one Too Many at this point. ;) Call it a very dynamic rate limit that autoscales all the way down to zero.
  - 1vuio0pswjnm7 2y ago
    IME, it is relatively common to get this response code incorrectly, i.e., not triggered by rate. IME, it can be triggered by sending one and only one HTTP request to a site one has never visited before. For example, a request sent to a certain IP address with a certain set of HTTP headers and header values using a certain HTTP method and HTTP version. Despite what "Too many requests" suggests, people writing/configuring software are (accidentally) sending this error in response to requests based on quality not quantity. In each case where I have received it, I was able to make the error go away by changing something about the request.
duskwuff 2y ago
I prefer self-hosting MailHog: https://github.com/mailhog/MailHog https://github.com/mailhog/MailHog
- keithnz 2y ago
  there's a few of them out there, I use Papercut https://github.com/ChangemakerStudios/Papercut-SMTP https://github.com/ChangemakerStudios/Papercut-SMTP
  - kagevf 2y ago
    I use papercut too.
- sdoering 2y ago
  From the Mailpit README: > Mailpit was originally inspired by MailHog which is no longer maintained and hasn't seen active development or security updates for a few years now.
  - ramon156 2y ago
    Curious as to why it would need updates. Security updates sure, but what really needs to update on an STMP (+ TLS?) solution?
    - e_y_ 2y ago
      Third-party libraries? TLS has definitely had its fair share of implementation and protocol vulnerabilities. Although in mailhog's case it looks like STARTTLS was never implemented. Hence the forks.
    - duskwuff 2y ago
      It's a web application too, not just a SMTP server. The MailHog web app is a little rough around the edges (it's an early AngularJS app) and had some bugs involving HTML mail content, so I can see the need for an updated version.
- 2y ago
abound 2y ago
I recently came across MailSlurper [1], which I think serves a similar use case. [1] https://github.com/mailslurper/mailslurper https://github.com/mailslurper/mailslurper
politelemon 2y ago
Since it seems to be overwhelmed currently, I'd suggest letting users host their own version of this in their own networks.
snikch 2y ago
I have used tools like these for nearly a decade and find them invaluable. My current favourite is Mailpit [1]. The “OG” I would consider Mailcatcher [2] from my Rails days [1] https://github.com/axllent/mailpit https://github.com/axllent/mailpit [2] https://mailcatcher.me/ https://mailcatcher.me/
- dustedcodes 2y ago
  Add https://msgdrop.io https://msgdrop.io to that list :) An API driven approach like Msgdrop is more flexible, as many apps can’t configure a fake SMTP server, especially if you use tools like Mailgun or SendGrid.
- lshevtsov 2y ago
  The "OG" fake SMTP as a service would be Mailtrap [1]. [1] https://mailtrap.io https://mailtrap.io.
- Ocha 2y ago
  Add https://mailsnag.com https://mailsnag.com to the list. You can share email samples with others for the review and it also allows receiving emails for your prod env - send emails to your app and process them in it.
  - remram 2y ago
    Don't they all "allow receiving emails from your prod env"? They all speak SMTP. What am I missing?
    - aeaa3 2y ago
      The idea is that, in a a test environment, we should not actually deliver the mails to the recipient, instead making them available on a web page (or through an API).
aeaa3 2y ago
This is all open source by the way, and can be self-hosted. All the code is at https://github.com/aled/fakemail https://github.com/aled/fakemail.
jallmann 2y ago
Also built something very similar - https://ephemeralpostal.com https://ephemeralpostal.com Comes with a small API so your integration tests can actually check the contents of emails that were expected to be sent out. Did a Show HN a while ago with more details - https://news.ycombinator.com/item?id=40590670 https://news.ycombinator.com/item?id=40590670
mandeepj 2y ago
On the real one, you can set it to -'save emails to filesystem instead of sending out', so a fake SMTP is not really required.
rietta 2y ago
I use dockerized mailcatcher for this. Used it for years with ruby on rails and now use it for everything.
jackthemuss 2y ago
I use MailSlurp
dboreham 2y ago
I have a rule that any time someone ships a server, they're required to also ship a version that supports client testing purposes. Of course nobody has ever observed my rule. Doesn't make it a bad rule...
- endorphine 2y ago
  In what language though?
- soul_grafitti 2y ago
  I wouldn't say it's a 'bad' rule - just ineffective.
SoftTalker 2y ago
If "my users" might receive test emails, that means I'm using real user email addresses in my test data, therfore my test SMTP server should not be a random system on the internet but something I fully control.
- xp84 2y ago
  I think that’s the point. You can self-host it. This is a demo but there’s a link to the source at the bottom of that page.
b112 2y ago
I don't want to burst bubbles, but postfix with few lines of config can redirect all email to DEVs. You really want, whenever possible, to test everything using real tools. Doubly so, using tools you'll use it PROD. However, at least postfix is de-facto standard. apt-get install postfix, postfix-pcre bsd-mailx, config and done. Here's an example redirecting ALL outgoing emails, UNLESS they are to your OK domains. Redirects are sent to an alias in /etc/aliases, which you can point to anything. (Easier for DEVs to modify when required)... /etc/postfix/header_checks (adds a header with original TO) /^(.*)@((?:(?![^\.]+.corp-domain1.com|anotherdomain.ca|localhost).)*)$/ PREPEND DEV-ENV-REDIRECT: ${1}@${2} This MUST be the same as the above regex, so that the TO preservation + redirect are both done in tandem.. /etc/postfix/recipient_canonical_map /^(.*)@((?:(?![^\.]+.corp-domain1.com|anotherdomain.ca|localhost).)*)$/ externalredirect@localhost Then in main.cf # added recipient_canonical_classes = envelope_recipient recipient_canonical_maps = pcre:/etc/postfix/recipient_canonical_map local_header_rewrite_clients = permit_mynetworks # header_checks = pcre:/etc/postfix/header_checks Then in /etc/aliases: # externalredirect: dev # dev's email.. (default unless set) dev: /dev/null Preserving the original TO as another header ensures you can debug if required, whilst preserving 100% the original mail body. Using a second redirect in aliases, allows you do something such as: /etc/aliases: # externalredirect: dev, another_email_address_for_logs # dev's email.. (default unless set) dev: /dev/null So it's super easy for a dev to just change: dev: /dev/null to dev: dev@corp.com Without the need to worry about overall redirect stuff. NOTE that if you don't have a unique email for your company's DEVs to use, this won't work, HOWEVER... you can redirect with more refined controls above. That is, instead of saying "if it's not to a company domain, then redirect this DEV TEST email!", you can "If not to this specific email address, then redirect to this specific email address". The reason I have this setup to redirect of not corp domain, is that the env I have this deployed in is byte per byte 100% identical to PROD deployment, with only very, very, very, minor tweaks. About 20 bytes or so. That way, all tests done in DEV are 100% identical to configs in PROD. You eliminate PROD deploy bugs more aptly this way. And so if local MTAs are postfix in PROD, then you can keep all of your PROD postfix configs, with these minor changes to lock down DEV. And, you can keep the all the config files, all the config, and just have empty header_checks, recipient_canonical_map files. But this means that alert emails that might get send from PROD have genericized domains, so in such envs it's easier to NOT redirect corp dest emails carte blanche, and then send everything else to a redirect dest. That way monitoring / emerg emails get through unvarnished.
tzs 2y ago
Wow...I didn't realize there were so many of these. People have mentioned almost a dozen in the comments so far. I too have one, but it is very barebones. No GUI, no API. I suspect it would fail in many cases that the others handle, but it is fine for my test environment. That environment is basically a bunch of services from work that in production run on separate servers all shoved into one test VM with a firewall that blocks most outgoing connections to keep things from escaping. The firewall reroutes any attempted outgoing port 25 connections to localhost port 2000, which my fake SMTP server listens on. When something connections it creates a timestamped file, sends them a "220 hello" message, and then loops reading what they send. Everything they send is copied to the file. If they send a "quit" command it sends back "221 bye" and disconnects and closed the output file. If they send a "data" command it sends back "354 send the message" and then loops until they send a "." line. When they send that it sends back "250 OK". If they send anything else it just says "250 OK". That ridiculously small subset of SMTP turns out to be fine in my environment. Here it is in case anyone might actually find it useful [1]. Building and running is simple. It's a single Java file, SmtpSink.java. Put that somewhere, "mkdir msgs" there, "javac SmtpSink.java", and then "java SmtpSink". The data for each connection will be in the msgs directory. [1] https://pastebin.com/dqgGZB82 https://pastebin.com/dqgGZB82
lndsh 2y ago
[flagged]
aeaa3 2y ago
A little more information: So, it's called fakemail but there is a real SMTP server in there. Attachments should work fine. Getting the web app to create SMTP accounts was quite tricky, I'm sure there are better ways but I ended up implementing the unix crypt() algorithm in C#. Server is holding up fine so far (there was a rate-limiting bug which brought the site down yesterday). Logs show 37K unique IPs have accessed it since yesterday, and it seems to be using about 1% of the CPU (it's on a free VM in the Oracle Cloud). There is a whole API sitting behind the web page, including proper authentication, but the frontend is very much a MVP. Very few actual emails have been sent to it, so I'd love it if people could actually send stuff. There are a bunch of websites that can be used to send test mails, e.g. https://www.gmass.co/smtp-test https://www.gmass.co/smtp-test https://www.smtper.net/ https://www.smtper.net/ https://smtpserver.com/smtptest https://smtpserver.com/smtptest https://dnschecker.org/smtp-test-tool.php https://dnschecker.org/smtp-test-tool.php
sethammons 2y ago
since folks are sharing tools they use to test, I have one to validate if your email sender can send internationalized emails pünicode.com. Any local part will create a temp cache that you can check to see if your email system can deliver to international local and/or domain parts. EG, send an email to josé@pünicode.com and see it show up at pünicode.com/emails/josé.

127 points by aeaa3 2y ago

This is a side project of mine. Use this as your SMTP server in a test environment to guarantee that your users don't receive test emails. Looking for feedback, especially on the security side.

sam_perez 2y ago

Really cool. I couldn't find it documented anywhere, but if you save the unique (guid) url generated, how long is that available for? "Forever"?

aeaa3 2y ago
Yes, currently these are never deleted. I will probably update it to delete accounts that have been inactive for a year or something.

netsec1989 2y ago

Wow, this and all the various other options mentioned look very useful. Like others, I have hacked together configurations to do this in the past and would have loved to have these instead. I had no idea on the availability of all the various options, which leads to the question: Where would one find examples of similar network test servers for other protocols/functions? For example: - A SAML IdP - Define accounts and complete a login, allow debug of request/response. - A DNS Server - Define local domains and records, control whether Internet domains are resolvable or just local ‘corporate’ records. - Syslog Server - Catch logs and make them temporarily available. (usually syslogd works for this, but maybe test harnesses have advantages). - SNMP - Trap destination to capture/show alerts

atum47 2y ago

Down for me

TomasEkeli 2y ago

hugged to death? anyway - i got a great 429 back with a suggested try-after -time. it made me smile in appreciation! well done!

theginger 2y ago
That's the wrong status code The HTTP 429 Too Many Requests response status code indicates the user has sent too many requests in a given amount of time ("rate limiting"). Unless the server believes you as an individual user were sending too many it should not have been a 429 If the server was unable to handle the volume of requests more generally it should have been a 503 which also supports Retry-After
- aeaa3 2y ago
  The 429 response is sent when rate-limiting (based on IP address). Either something is wrong with my rate-limiting code or there are many people behind a single IP address. Anyway the limits are increased now.
  - theginger 2y ago
    Possible your application isn't seeing real ips, maybe is seeing the IP of a load balancer or similar.
- xp84 2y ago
  Maybe one request is one Too Many at this point. ;) Call it a very dynamic rate limit that autoscales all the way down to zero.
- 1vuio0pswjnm7 2y ago
  IME, it is relatively common to get this response code incorrectly, i.e., not triggered by rate. IME, it can be triggered by sending one and only one HTTP request to a site one has never visited before. For example, a request sent to a certain IP address with a certain set of HTTP headers and header values using a certain HTTP method and HTTP version. Despite what "Too many requests" suggests, people writing/configuring software are (accidentally) sending this error in response to requests based on quality not quantity. In each case where I have received it, I was able to make the error go away by changing something about the request.

duskwuff 2y ago

I prefer self-hosting MailHog: https://github.com/mailhog/MailHog https://github.com/mailhog/MailHog

keithnz 2y ago
there's a few of them out there, I use Papercut https://github.com/ChangemakerStudios/Papercut-SMTP https://github.com/ChangemakerStudios/Papercut-SMTP
- kagevf 2y ago
  I use papercut too.
sdoering 2y ago
From the Mailpit README: > Mailpit was originally inspired by MailHog which is no longer maintained and hasn't seen active development or security updates for a few years now.
- ramon156 2y ago
  Curious as to why it would need updates. Security updates sure, but what really needs to update on an STMP (+ TLS?) solution?
  - e_y_ 2y ago
    Third-party libraries? TLS has definitely had its fair share of implementation and protocol vulnerabilities. Although in mailhog's case it looks like STARTTLS was never implemented. Hence the forks.
  - duskwuff 2y ago
    It's a web application too, not just a SMTP server. The MailHog web app is a little rough around the edges (it's an early AngularJS app) and had some bugs involving HTML mail content, so I can see the need for an updated version.
2y ago

abound 2y ago

I recently came across MailSlurper [1], which I think serves a similar use case. [1] https://github.com/mailslurper/mailslurper https://github.com/mailslurper/mailslurper

politelemon 2y ago

Since it seems to be overwhelmed currently, I'd suggest letting users host their own version of this in their own networks.

snikch 2y ago

I have used tools like these for nearly a decade and find them invaluable. My current favourite is Mailpit [1]. The “OG” I would consider Mailcatcher [2] from my Rails days [1] https://github.com/axllent/mailpit https://github.com/axllent/mailpit [2] https://mailcatcher.me/ https://mailcatcher.me/

dustedcodes 2y ago
Add https://msgdrop.io https://msgdrop.io to that list :) An API driven approach like Msgdrop is more flexible, as many apps can’t configure a fake SMTP server, especially if you use tools like Mailgun or SendGrid.
lshevtsov 2y ago
The "OG" fake SMTP as a service would be Mailtrap [1]. [1] https://mailtrap.io https://mailtrap.io.
Ocha 2y ago
Add https://mailsnag.com https://mailsnag.com to the list. You can share email samples with others for the review and it also allows receiving emails for your prod env - send emails to your app and process them in it.
- remram 2y ago
  Don't they all "allow receiving emails from your prod env"? They all speak SMTP. What am I missing?
  - aeaa3 2y ago
    The idea is that, in a a test environment, we should not actually deliver the mails to the recipient, instead making them available on a web page (or through an API).

aeaa3 2y ago

This is all open source by the way, and can be self-hosted. All the code is at https://github.com/aled/fakemail https://github.com/aled/fakemail.

jallmann 2y ago

Also built something very similar - https://ephemeralpostal.com https://ephemeralpostal.com Comes with a small API so your integration tests can actually check the contents of emails that were expected to be sent out. Did a Show HN a while ago with more details - https://news.ycombinator.com/item?id=40590670 https://news.ycombinator.com/item?id=40590670

mandeepj 2y ago

On the real one, you can set it to -'save emails to filesystem instead of sending out', so a fake SMTP is not really required.

rietta 2y ago

I use dockerized mailcatcher for this. Used it for years with ruby on rails and now use it for everything.

jackthemuss 2y ago

I use MailSlurp

dboreham 2y ago

I have a rule that any time someone ships a server, they're required to also ship a version that supports client testing purposes. Of course nobody has ever observed my rule. Doesn't make it a bad rule...

endorphine 2y ago
In what language though?
soul_grafitti 2y ago
I wouldn't say it's a 'bad' rule - just ineffective.

SoftTalker 2y ago

If "my users" might receive test emails, that means I'm using real user email addresses in my test data, therfore my test SMTP server should not be a random system on the internet but something I fully control.

xp84 2y ago
I think that’s the point. You can self-host it. This is a demo but there’s a link to the source at the bottom of that page.

b112 2y ago

I don't want to burst bubbles, but postfix with few lines of config can redirect all email to DEVs. You really want, whenever possible, to test everything using real tools. Doubly so, using tools you'll use it PROD. However, at least postfix is de-facto standard. apt-get install postfix, postfix-pcre bsd-mailx, config and done. Here's an example redirecting ALL outgoing emails, UNLESS they are to your OK domains. Redirects are sent to an alias in /etc/aliases, which you can point to anything. (Easier for DEVs to modify when required)... /etc/postfix/header_checks (adds a header with original TO) /^(.*)@((?:(?![^\.]+.corp-domain1.com|anotherdomain.ca|localhost).)*)$/ PREPEND DEV-ENV-REDIRECT: ${1}@${2} This MUST be the same as the above regex, so that the TO preservation + redirect are both done in tandem.. /etc/postfix/recipient_canonical_map /^(.*)@((?:(?![^\.]+.corp-domain1.com|anotherdomain.ca|localhost).)*)$/ externalredirect@localhost Then in main.cf # added recipient_canonical_classes = envelope_recipient recipient_canonical_maps = pcre:/etc/postfix/recipient_canonical_map local_header_rewrite_clients = permit_mynetworks # header_checks = pcre:/etc/postfix/header_checks Then in /etc/aliases: # externalredirect: dev # dev's email.. (default unless set) dev: /dev/null Preserving the original TO as another header ensures you can debug if required, whilst preserving 100% the original mail body. Using a second redirect in aliases, allows you do something such as: /etc/aliases: # externalredirect: dev, another_email_address_for_logs # dev's email.. (default unless set) dev: /dev/null So it's super easy for a dev to just change: dev: /dev/null to dev: dev@corp.com Without the need to worry about overall redirect stuff. NOTE that if you don't have a unique email for your company's DEVs to use, this won't work, HOWEVER... you can redirect with more refined controls above. That is, instead of saying "if it's not to a company domain, then redirect this DEV TEST email!", you can "If not to this specific email address, then redirect to this specific email address". The reason I have this setup to redirect of not corp domain, is that the env I have this deployed in is byte per byte 100% identical to PROD deployment, with only very, very, very, minor tweaks. About 20 bytes or so. That way, all tests done in DEV are 100% identical to configs in PROD. You eliminate PROD deploy bugs more aptly this way. And so if local MTAs are postfix in PROD, then you can keep all of your PROD postfix configs, with these minor changes to lock down DEV. And, you can keep the all the config files, all the config, and just have empty header_checks, recipient_canonical_map files. But this means that alert emails that might get send from PROD have genericized domains, so in such envs it's easier to NOT redirect corp dest emails carte blanche, and then send everything else to a redirect dest. That way monitoring / emerg emails get through unvarnished.

tzs 2y ago

Wow...I didn't realize there were so many of these. People have mentioned almost a dozen in the comments so far. I too have one, but it is very barebones. No GUI, no API. I suspect it would fail in many cases that the others handle, but it is fine for my test environment. That environment is basically a bunch of services from work that in production run on separate servers all shoved into one test VM with a firewall that blocks most outgoing connections to keep things from escaping. The firewall reroutes any attempted outgoing port 25 connections to localhost port 2000, which my fake SMTP server listens on. When something connections it creates a timestamped file, sends them a "220 hello" message, and then loops reading what they send. Everything they send is copied to the file. If they send a "quit" command it sends back "221 bye" and disconnects and closed the output file. If they send a "data" command it sends back "354 send the message" and then loops until they send a "." line. When they send that it sends back "250 OK". If they send anything else it just says "250 OK". That ridiculously small subset of SMTP turns out to be fine in my environment. Here it is in case anyone might actually find it useful [1]. Building and running is simple. It's a single Java file, SmtpSink.java. Put that somewhere, "mkdir msgs" there, "javac SmtpSink.java", and then "java SmtpSink". The data for each connection will be in the msgs directory. [1] https://pastebin.com/dqgGZB82 https://pastebin.com/dqgGZB82

lndsh 2y ago

[flagged]

aeaa3 2y ago

A little more information: So, it's called fakemail but there is a real SMTP server in there. Attachments should work fine. Getting the web app to create SMTP accounts was quite tricky, I'm sure there are better ways but I ended up implementing the unix crypt() algorithm in C#. Server is holding up fine so far (there was a rate-limiting bug which brought the site down yesterday). Logs show 37K unique IPs have accessed it since yesterday, and it seems to be using about 1% of the CPU (it's on a free VM in the Oracle Cloud). There is a whole API sitting behind the web page, including proper authentication, but the frontend is very much a MVP. Very few actual emails have been sent to it, so I'd love it if people could actually send stuff. There are a bunch of websites that can be used to send test mails, e.g. https://www.gmass.co/smtp-test https://www.gmass.co/smtp-test https://www.smtper.net/ https://www.smtper.net/ https://smtpserver.com/smtptest https://smtpserver.com/smtptest https://dnschecker.org/smtp-test-tool.php https://dnschecker.org/smtp-test-tool.php

sethammons 2y ago

since folks are sharing tools they use to test, I have one to validate if your email sender can send internationalized emails pünicode.com. Any local part will create a temp cache that you can check to see if your email system can deliver to international local and/or domain parts. EG, send an email to josé@pünicode.com and see it show up at pünicode.com/emails/josé.