4 ms·
I helped design and setup 2 large SMTP infrastructures for two different European mobile carriers. Both carriers had more than 1 million customers and processe
by smutticus 16y ago
I helped design and setup 2 large SMTP infrastructures for two different European mobile carriers. Both carriers had more than 1 million customers and processed close to 100 million messages a day. It was about 7 years ago but I doubt things have changed that much.
We made heavy use of Mirapoint. But we also made use in different places of qmail and postfix. At the high end of scaling SMTP routing starts becoming similar to routing IP traffic. You're just not going to beat a dedicated device like Mirapoint with some Unix box you make from scratch. You need dedicated and differentiated mail routers for inbound, outbound and mail storage. You monitor their usage and when they get overloaded you just add more. Since everything is load balanced you can scale close to linearly. We used SAN for storage. Local storage just doesn't work.
Mail hits an inbound SMTP router and that router does an LDAP lookup to find which storage box actually handles hat account. Then it forwards it. That's all it does and it does it fast. Each storage box handles mail for the accounts it stores and IMAP/POP3 access for those accounts as well. The outbound routers just take mail from the storage routers and spool it for outbound.
I could go on but I think you get the idea. The main idea is to put everything behind a load balancer that you can so that you can scale it linearly as much as possible.
- bravura 16y agoLocal storage just doesn't work. Can you explain why?
- Travis 16y agoI suspect it's two things. 1, the need for shared resources (users shouldn't have to connect to server71231.example.com, they should be connecting to mail.example.com. (2) performance (via fiber link or something)
- smutticus 16y agoYou want to give each user a fixed amount of storage. But most users don't use anywhere near their total. So you oversubscribe to save money by pooling everyone's storage in one massive SAN. I imagine Gmail does something similar. I bet if every Gmail user suddenly started using 100% of their email storage Google couldn't cope. But Google knows this won't happen. Over subscription is how you make money in the end. Getting better at figuring out exactly how much you need when you need it directly affects your margins.