4 ms·

Surprised to see this here now after 11 years. Needless to say, this is incredibly out of date. Mailinator now runs across several servers with a real-time web

by zinxq 8y ago

Surprised to see this here now after 11 years. Needless to say, this is incredibly out of date. Mailinator now runs across several servers with a real-time websocket system tied to Redis channels for incoming emails. The multiple server system is more for fault-tolerance than scaling issues. The Websocket-to-redis connection allows incoming emails to be available the instant they arrive (fully) at Mailinator. Instantly - no polling front-end or otherwise. Public email storage is still in-memory with a system that chunks emails into common parts and reuses them. A reference counting garbage collector cleans them up once no email in the system is still using that part. I tried compression a million different ways but nothing beat reusing parts of emails. http://mailinator.blogspot.com/2012/02/how-mailinator-compresses-email-by-90.html http://mailinator.blogspot.com/2012/02/how-mailinator-compre... The 2007 article is a valid picture of things then, but the system has come a long, long way since that time.

newman314 8y ago
Thanks for the service! It's very useful. However, I find that quite a few of the providers have wised up to the domains and more often than not I am now unable to use mailinator address for signups (even with the alternate domains)
- zinxq 8y ago
  Indeed. I thought about a blog post how to configure haproxy to proxy SMTP to mailinator and stay undetectable. But Mailinator's use case has morphed over the years. It's now used as a tool for QA and UAT testing now a great deal. That use case has spawned private domains and API access. Same thought process goes toward services that wish to ban it, if they really want to, that's their business. (You can of course sign-up for a Mailinator account using a Mailinator email)
  - fla 8y ago
    > Public email storage is still in-memory with a system that chunks emails into common parts and reuses them. A reference counting garbage collector cleans them up once no email in the system is still using that part. That sounds really cool. Any chance you'll write a blog post about it ?
    - sova 8y ago
      Looks like he did: http://mailinator.blogspot.com/2012/02/how-mailinator-compresses-email-by-90.html http://mailinator.blogspot.com/2012/02/how-mailinator-compre...
      - fla 8y ago
        Thanks !
anothergoogler 8y ago
Thanks for the writeup. What are you using to handle the WS connections on the server?
- zinxq 8y ago
  import javax.websocket.MessageHandler; Stayed away from frameworks. Mailinator suffers from "do it yourself" syndrome where I wanted write most things myself for the educational value. No Spring, no web frameworks, straight servlets. Custom SMTP server. (It does use Redis pub/sub but I did write my own first, Redis was just too easy not to rely on)
  - augbog 8y ago
    Woohoo Job security though right?
  - rufugee 8y ago
    Reminds me of the Plenty of Fish story in terms of avoiding frameworks and rolling it yourself. In hindsight, would you consider this approach an accelerant or a deterrent? I know I've often found myself thinking about Plenty of Fish when I'm on page 400 of the lastest 1000 page hip framework book :-)
  - bluedino 8y ago
    Is it really written in Java? I love when popular services are written in Java, PHP, or even something crazier like ColdFusion. Just to spite the naysayers.
    - nickpsecurity 8y ago
      That same attitude is why I kept pushing people to do one in Fortran. Then this showed up: https://news.ycombinator.com/item?id=11938405 https://news.ycombinator.com/item?id=11938405 Now, we need a useful app in Unlambda that people actually paid for. Then, you show them the code and tutorials when they're interested in doing extensions. Observe, record, and share their reactions. (villainous laughter) http://www.madore.org/~david/programs/unlambda/#intro http://www.madore.org/~david/programs/unlambda/#intro
flashmob 8y ago
Hi! I run a "copycat" site that perhaps you may be referring to in that post, GuerrillaMail.com (well, actually, I didn't copy it myself, but acquired it from another guy many years back) and also been using Redis to store all incoming mail in memory. RAM is cheap these days and you can find decently priced servers with >= 128GB easy. Haven't moved over to websockets or "chunking" / deduplication yet, but architecture and UI needs an update in that direction to make it more instant. Thanks for some ideas ;-) Actually, like you, I've also hand-rolled most of it myself, replacing the previous guy's architecture. It's been a lot of fun. No frameworks, no bootstrap, just a few dependencies here and there. Started with PHP, but now prefer to use Go for anything new. Also hand-rolled the SMTP server which turned in to a project on its own, https://github.com/flashmob/go-guerrilla https://github.com/flashmob/go-guerrilla
- Dagger2 8y ago
  Is guerrillamail.com supposed to be displaying just a "Welcome to nginx" page? Screenshot on the left: https://nat64check.ipv6-lab.net/v6score/measurement-30830/ https://nat64check.ipv6-lab.net/v6score/measurement-30830/
andris9 8y ago
Very cool! In WildDuck IMAP server I use a bit different deduplication method (I deduplicate decoded attachments, not lines) that does not give 90% reduction but still more than 50%: https://github.com/nodemailer/wildduck/blob/master/README.md#storage https://github.com/nodemailer/wildduck/blob/master/README.md...
jorangreef 8y ago
Have you considered content-dependent chunking for your deduplication? At Ronomon, we use https://github.com/ronomon/deduplication https://github.com/ronomon/deduplication

by zinxq 8y ago

newman314 8y ago

Thanks for the service! It's very useful. However, I find that quite a few of the providers have wised up to the domains and more often than not I am now unable to use mailinator address for signups (even with the alternate domains)

zinxq 8y ago
Indeed. I thought about a blog post how to configure haproxy to proxy SMTP to mailinator and stay undetectable. But Mailinator's use case has morphed over the years. It's now used as a tool for QA and UAT testing now a great deal. That use case has spawned private domains and API access. Same thought process goes toward services that wish to ban it, if they really want to, that's their business. (You can of course sign-up for a Mailinator account using a Mailinator email)
- fla 8y ago
  > Public email storage is still in-memory with a system that chunks emails into common parts and reuses them. A reference counting garbage collector cleans them up once no email in the system is still using that part. That sounds really cool. Any chance you'll write a blog post about it ?
  - sova 8y ago
    Looks like he did: http://mailinator.blogspot.com/2012/02/how-mailinator-compresses-email-by-90.html http://mailinator.blogspot.com/2012/02/how-mailinator-compre...
    - fla 8y ago
      Thanks !

anothergoogler 8y ago

Thanks for the writeup. What are you using to handle the WS connections on the server?

zinxq 8y ago
import javax.websocket.MessageHandler; Stayed away from frameworks. Mailinator suffers from "do it yourself" syndrome where I wanted write most things myself for the educational value. No Spring, no web frameworks, straight servlets. Custom SMTP server. (It does use Redis pub/sub but I did write my own first, Redis was just too easy not to rely on)
- augbog 8y ago
  Woohoo Job security though right?
- rufugee 8y ago
  Reminds me of the Plenty of Fish story in terms of avoiding frameworks and rolling it yourself. In hindsight, would you consider this approach an accelerant or a deterrent? I know I've often found myself thinking about Plenty of Fish when I'm on page 400 of the lastest 1000 page hip framework book :-)
- bluedino 8y ago
  Is it really written in Java? I love when popular services are written in Java, PHP, or even something crazier like ColdFusion. Just to spite the naysayers.
  - nickpsecurity 8y ago
    That same attitude is why I kept pushing people to do one in Fortran. Then this showed up: https://news.ycombinator.com/item?id=11938405 https://news.ycombinator.com/item?id=11938405 Now, we need a useful app in Unlambda that people actually paid for. Then, you show them the code and tutorials when they're interested in doing extensions. Observe, record, and share their reactions. (villainous laughter) http://www.madore.org/~david/programs/unlambda/#intro http://www.madore.org/~david/programs/unlambda/#intro

flashmob 8y ago

Hi! I run a "copycat" site that perhaps you may be referring to in that post, GuerrillaMail.com (well, actually, I didn't copy it myself, but acquired it from another guy many years back) and also been using Redis to store all incoming mail in memory. RAM is cheap these days and you can find decently priced servers with >= 128GB easy. Haven't moved over to websockets or "chunking" / deduplication yet, but architecture and UI needs an update in that direction to make it more instant. Thanks for some ideas ;-) Actually, like you, I've also hand-rolled most of it myself, replacing the previous guy's architecture. It's been a lot of fun. No frameworks, no bootstrap, just a few dependencies here and there. Started with PHP, but now prefer to use Go for anything new. Also hand-rolled the SMTP server which turned in to a project on its own, https://github.com/flashmob/go-guerrilla https://github.com/flashmob/go-guerrilla

Dagger2 8y ago
Is guerrillamail.com supposed to be displaying just a "Welcome to nginx" page? Screenshot on the left: https://nat64check.ipv6-lab.net/v6score/measurement-30830/ https://nat64check.ipv6-lab.net/v6score/measurement-30830/

andris9 8y ago

Very cool! In WildDuck IMAP server I use a bit different deduplication method (I deduplicate decoded attachments, not lines) that does not give 90% reduction but still more than 50%: https://github.com/nodemailer/wildduck/blob/master/README.md#storage https://github.com/nodemailer/wildduck/blob/master/README.md...

jorangreef 8y ago

Have you considered content-dependent chunking for your deduplication? At Ronomon, we use https://github.com/ronomon/deduplication https://github.com/ronomon/deduplication