7 ms·

Most businesses can probably run on a single server from OVH (see Paul Tyma's Mailinator architecture for inspiration: 1.2 billion emails on a single server, ht

Most businesses can probably run on a single server from OVH (see Paul Tyma's Mailinator architecture for inspiration: 1.2 billion emails on a single server, http://highscalability.com/mailinator-architecture http://highscalability.com/mailinator-architecture), but I suspect a lot of folks in tech want to pad their resume with cloud buzzwords, so they recommend overcomplicated architectures to business that they absolutely wouldn't spend their own money on. So much of tech design, nay, tech thinking, is based on following fashion trends, it's shocking.

jjeaff 7y ago
I think you are right. And I started out doing just that with my business. I could easily run the core of the business on a single server. But I have found that what really starts requiring scalability are all the add-ons. Then we needed caching with redis, then elastic stack for logs, then prometheus for monitoring and on and on. Also, OVH doesn't have much presence in the U.S.
erik_seaberg 7y ago
It's really good machine sympathy for keeping the unfunded service cheap to run, but it's one bad fan bearing away from losing a lot of messages, so I wouldn't say its design assures four nines. And the optimization work to get all the way down to n=1 wouldn't have paid off if Tyma needed to hire someone at market rate to build that and maintain it, because an engineer-month costs hundreds of instance-months.
- parliament32 7y ago
  Honestly physical servers have better uptime than some of amazon's regions.
  - erik_seaberg 7y ago
    YMMV. I've seen an adtech company who purchased hardware to install in colos, and a couple of years later their failure rate was almost 5%/year. I would never bet on four nines from a single commodity box.
dvfjsdhgfv 7y ago
What you describe is both a problem and an opportunity. We live in strange times when people pay through the nose for something that could be easily implemented on dedicated server in much simpler way and with way less risk, all for the fraction of the original amount. But tech comes in circles, and I bet we're going to see some "miraculous discoveries" of how companies saved millions by migrating from AWS to dedicated servers.
- rswskg 7y ago
  My last full time gig did this. They don't really talk about it, but it's a long established tech company in san fran. Saved alot of money doing so.
cwyers 7y ago
The difference between the needs of Mailinator ("Email can reside in RAM because it is temporary (3-4 hours)") and most corporate e-mail services are... large. If you treat e-mail as ephemeral, sure, you can run on a single box somewhere, and in the case of a failure, a stateless recovery will have the service up and running again. Meanwhile, if you're a business, you have actual compliance requirements on how long you have to keep an e-mail around.[1] SarbOx says you have to keep documents related to insider dealings _indefinitely_. Do you, the sysadmin, know which e-mails those are? So you need backups. If you aren't testing your backups, you might as well not have backups. So you need a second server sometimes to test backup restores on. Do you deal with HIPAA protected information? If so, now you have a bunch of compliance requirements about the security of those e-mails and those backups you just made.[2] It turns out that "you can save hundreds of dollars a month on e-mail by introducing an existential risk to your business in the event of a lawsuit" is not a great value proposition for most businesses, and most businesses are better off just going with Office 365 or Google Apps for Business, which have dedicated compliance officers and certifications for all of these issues and a lot that I haven't named.[3][4] Yes, it is possible to overarchitecture things. Yes, there are CIOs and CTOs who want to cargo cult their way to success by imitating successful cloud migrations done by others. But there are a lot of real business problems out there that can be solved by doing things differently than they were done 10 or 20 years ago. 1) https://www.intradyn.com/email-retention-laws/ https://www.intradyn.com/email-retention-laws/ 2) https://www.hipaaguide.net/hipaa-email-compliance-requirements/ https://www.hipaaguide.net/hipaa-email-compliance-requiremen... 3) https://docs.microsoft.com/en-us/office365/servicedescriptions/office-365-platform-service-description/compliance-servicedesc?redirectedfrom=MSDN https://docs.microsoft.com/en-us/office365/servicedescriptio... 4) https://support.google.com/googlecloud/answer/6056694?hl=en https://support.google.com/googlecloud/answer/6056694?hl=en
parliament32 7y ago
We run servers in colos in a bunch of POPs. We've tried to do the "how much would this cost in AWS" a few times now over the years, and it's hilarious how much staggeringly more expensive it is. Even if you do stupid things like assume hardware will depreciate over 2 years (in real life, try 5-10) AWS always ends up being an order of magnitude more expensive. And it's not really business-model specific: we've analyzed compute-heavy/bandwidth-heavy/HA-esque projects and it's always insane how much they get away with charging for services. Maybe if you're in the "under $1k/mo" range it makes sense to microservice in AWS, but even then VPS hosts are so much cheaper and easier to use.

jjeaff 7y ago

I think you are right. And I started out doing just that with my business. I could easily run the core of the business on a single server. But I have found that what really starts requiring scalability are all the add-ons. Then we needed caching with redis, then elastic stack for logs, then prometheus for monitoring and on and on. Also, OVH doesn't have much presence in the U.S.

erik_seaberg 7y ago

It's really good machine sympathy for keeping the unfunded service cheap to run, but it's one bad fan bearing away from losing a lot of messages, so I wouldn't say its design assures four nines. And the optimization work to get all the way down to n=1 wouldn't have paid off if Tyma needed to hire someone at market rate to build that and maintain it, because an engineer-month costs hundreds of instance-months.

parliament32 7y ago
Honestly physical servers have better uptime than some of amazon's regions.
- erik_seaberg 7y ago
  YMMV. I've seen an adtech company who purchased hardware to install in colos, and a couple of years later their failure rate was almost 5%/year. I would never bet on four nines from a single commodity box.

dvfjsdhgfv 7y ago

What you describe is both a problem and an opportunity. We live in strange times when people pay through the nose for something that could be easily implemented on dedicated server in much simpler way and with way less risk, all for the fraction of the original amount. But tech comes in circles, and I bet we're going to see some "miraculous discoveries" of how companies saved millions by migrating from AWS to dedicated servers.

rswskg 7y ago
My last full time gig did this. They don't really talk about it, but it's a long established tech company in san fran. Saved alot of money doing so.

cwyers 7y ago

The difference between the needs of Mailinator ("Email can reside in RAM because it is temporary (3-4 hours)") and most corporate e-mail services are... large. If you treat e-mail as ephemeral, sure, you can run on a single box somewhere, and in the case of a failure, a stateless recovery will have the service up and running again. Meanwhile, if you're a business, you have actual compliance requirements on how long you have to keep an e-mail around.[1] SarbOx says you have to keep documents related to insider dealings _indefinitely_. Do you, the sysadmin, know which e-mails those are? So you need backups. If you aren't testing your backups, you might as well not have backups. So you need a second server sometimes to test backup restores on. Do you deal with HIPAA protected information? If so, now you have a bunch of compliance requirements about the security of those e-mails and those backups you just made.[2] It turns out that "you can save hundreds of dollars a month on e-mail by introducing an existential risk to your business in the event of a lawsuit" is not a great value proposition for most businesses, and most businesses are better off just going with Office 365 or Google Apps for Business, which have dedicated compliance officers and certifications for all of these issues and a lot that I haven't named.[3][4] Yes, it is possible to overarchitecture things. Yes, there are CIOs and CTOs who want to cargo cult their way to success by imitating successful cloud migrations done by others. But there are a lot of real business problems out there that can be solved by doing things differently than they were done 10 or 20 years ago. 1) https://www.intradyn.com/email-retention-laws/ https://www.intradyn.com/email-retention-laws/ 2) https://www.hipaaguide.net/hipaa-email-compliance-requirements/ https://www.hipaaguide.net/hipaa-email-compliance-requiremen... 3) https://docs.microsoft.com/en-us/office365/servicedescriptions/office-365-platform-service-description/compliance-servicedesc?redirectedfrom=MSDN https://docs.microsoft.com/en-us/office365/servicedescriptio... 4) https://support.google.com/googlecloud/answer/6056694?hl=en https://support.google.com/googlecloud/answer/6056694?hl=en

parliament32 7y ago

We run servers in colos in a bunch of POPs. We've tried to do the "how much would this cost in AWS" a few times now over the years, and it's hilarious how much staggeringly more expensive it is. Even if you do stupid things like assume hardware will depreciate over 2 years (in real life, try 5-10) AWS always ends up being an order of magnitude more expensive. And it's not really business-model specific: we've analyzed compute-heavy/bandwidth-heavy/HA-esque projects and it's always insane how much they get away with charging for services. Maybe if you're in the "under $1k/mo" range it makes sense to microservice in AWS, but even then VPS hosts are so much cheaper and easier to use.