5 ms·

Would you rather have to fix your own data center, or wait 4 hours. AWS works 99% of the time,plus it's someone else's problem

by 41209 5y ago

iso1631 5y ago
In my industry, five nines is the starting level. You're proposing something 1,000 times worse.
- sumtechguy 5y ago
  If you have to have 5 9's starting with any cloud service is pretty tough sell at this point. You can build that type of system ontop of them. But it takes a willingness to also build your own and use competing services adding even more complexity. The BYO or competing services bit is so you can keep the lights on even when the cloud eats it, not if. Part of 5 9's is planning for catastrophic failure. Things like 'what if the load balancer/router/backhaul/dns burns out, you have a spare on hand? Then what happens to services where you had them sticky?' Even then what if both the primary and secondary are out? What then? Lots of planning and making sure you know what to do when each of those cases happen. The nice bit though is many services can go down. Yeah it stings a bit (money, reputation, time, etc). But overall it is not that big of deal. But for the places where you can not go down. Tons of planning and tons of backup plans with backup plans, and a different style of producing code.
- Edman274 5y ago
  Besides the fact that they used "99 percent" in a colloquial, informal way ... is that really how calculations like that work, or should work? I mean, if you have 99 percent availability, and 99.999 percent availability, maybe it's valid to say that the former is 1000 times worse than the other, but that could also be true by saying that 99.999 percent availability is 10,000 times worse than 99.9999999 percent availability (where a service only goes down for a third of a second in the span of a decade), and that 99.9999999 percent availability is infinity times worse than something that hasn't had an outage within whatever monitored period (since 100 percent availability is the same as ninety nine dot nine repeating availability)
- stjohnswarts 5y ago
  Then AWS isn't really targeting you as a customer though. Most of the web really isn't that critical and can survive a few hours outage occasionally.
  - greedo 5y ago
    But for those who bought into AWS as a way of improving their performance and stability, "a few hours outage occasionally" can be very expensive. If my company experienced a multi-hour outage "occasionally,", it would be $100K/hour in lost revenue. Your remark reminds me of people who say "Well, Tesla really doesn't mean `autopilot` in the way you think."
emptybottle 5y ago
You don’t need to build your own data center to host outside AWS. You can fully manage your hardware and software fleet while still paying a host to provide data center service, and networking too.
El_RIDO 5y ago
I absolutely prefer to have the option to go into a datacenter in a hurry and actually fix stuff and be in charge, then be stuck with having to wait an indefinite amount of time, twiddle my thumbs, apologize to customers and hope for the best. While I considered myself a decent Windows NT admin, 20 years ago, the reason I went all in on Linux and FLOSS software at the turn of the century was because I dreaded the powerlessness that these proprietary solutions gave me, when they failed. You'd call the vendors, pored through logs, finding obscure, undocumented error codes, etc. With FLOSS and self-hosting youve got all the information at your fingertips. And if you encountered bugs and you can dig into the sources, patch them, re-compile and fix things - and share them with others and feel that you're contributing to our profession. When I got the chance to do cloud projects over the last 5 - 10 years, I always took these opportunities, hoping to ensure that I keep up with the tech. At first I was hopeful to offload the boring ops tasks, take our config management to the next level and automate even more. With every platform I got to work with, AWS, Azure and GCP so far, we kept finding bugs in their APIs, outdated or otherwise incorrect documentation and very unpredictable performance, unless you actually can run stacks at scale to average it out (as in more then just 10 - 20 instances or a larger clustered SaaS of the cloud vendor). Many times we also encountered undocumented limits that required requesting support and waiting for approval by the cloud vendor, to get even their mid-sized resources allocated. It all works very nicely on the free-tier-eligible, smallest instances and services, if as slow and high latency as is to be expected, but as soon as you actually need some decently sized storage, compute or bandwidth, it becomes quickly more expensive than what you can put in two or three datacenters for redundancy yourself, if you look at the yearly costs. So far none of the PoCs I was involved with ever got approved long term. They mostly end up as reference implementations for our customers or show case material for the corporate blog. :-( No thanks, I don't want to go back to feel that powerless as I did on closed systems ever again. Luckily, although many seem to think that cloud is the only option to run at a global scale, you can still provide lots of valuable services on the internet using robust hardware, housed in well connected datacenters.
- 41209 5y ago
  You may be correct, for some very specific use cases. No matter how much corporate clients want to holler, a few hours of downtime isn't going to hurt anyone. I know I'd rather someone else do it, then having to drive 2 hours away to a data center at 4:00 in the morning. But I don't know exactly what you're working on , I definitely can't imagine some use cases where a few hours of down time is just unacceptable. I know I wouldn't want to run a logistics firm with servers that go down all the time.

Would you rather have to fix your own data center, or wait 4 hours. AWS works 99% of the time,plus it's someone else's problem

by 41209 5y ago

Would you rather have to fix your own data center, or wait 4 hours. AWS works 99% of the time,plus it's someone else's problem

iso1631 5y ago

In my industry, five nines is the starting level. You're proposing something 1,000 times worse.

sumtechguy 5y ago
If you have to have 5 9's starting with any cloud service is pretty tough sell at this point. You can build that type of system ontop of them. But it takes a willingness to also build your own and use competing services adding even more complexity. The BYO or competing services bit is so you can keep the lights on even when the cloud eats it, not if. Part of 5 9's is planning for catastrophic failure. Things like 'what if the load balancer/router/backhaul/dns burns out, you have a spare on hand? Then what happens to services where you had them sticky?' Even then what if both the primary and secondary are out? What then? Lots of planning and making sure you know what to do when each of those cases happen. The nice bit though is many services can go down. Yeah it stings a bit (money, reputation, time, etc). But overall it is not that big of deal. But for the places where you can not go down. Tons of planning and tons of backup plans with backup plans, and a different style of producing code.
Edman274 5y ago
Besides the fact that they used "99 percent" in a colloquial, informal way ... is that really how calculations like that work, or should work? I mean, if you have 99 percent availability, and 99.999 percent availability, maybe it's valid to say that the former is 1000 times worse than the other, but that could also be true by saying that 99.999 percent availability is 10,000 times worse than 99.9999999 percent availability (where a service only goes down for a third of a second in the span of a decade), and that 99.9999999 percent availability is infinity times worse than something that hasn't had an outage within whatever monitored period (since 100 percent availability is the same as ninety nine dot nine repeating availability)
stjohnswarts 5y ago
Then AWS isn't really targeting you as a customer though. Most of the web really isn't that critical and can survive a few hours outage occasionally.
- greedo 5y ago
  But for those who bought into AWS as a way of improving their performance and stability, "a few hours outage occasionally" can be very expensive. If my company experienced a multi-hour outage "occasionally,", it would be $100K/hour in lost revenue. Your remark reminds me of people who say "Well, Tesla really doesn't mean `autopilot` in the way you think."

emptybottle 5y ago

You don’t need to build your own data center to host outside AWS. You can fully manage your hardware and software fleet while still paying a host to provide data center service, and networking too.

El_RIDO 5y ago

I absolutely prefer to have the option to go into a datacenter in a hurry and actually fix stuff and be in charge, then be stuck with having to wait an indefinite amount of time, twiddle my thumbs, apologize to customers and hope for the best. While I considered myself a decent Windows NT admin, 20 years ago, the reason I went all in on Linux and FLOSS software at the turn of the century was because I dreaded the powerlessness that these proprietary solutions gave me, when they failed. You'd call the vendors, pored through logs, finding obscure, undocumented error codes, etc. With FLOSS and self-hosting youve got all the information at your fingertips. And if you encountered bugs and you can dig into the sources, patch them, re-compile and fix things - and share them with others and feel that you're contributing to our profession. When I got the chance to do cloud projects over the last 5 - 10 years, I always took these opportunities, hoping to ensure that I keep up with the tech. At first I was hopeful to offload the boring ops tasks, take our config management to the next level and automate even more. With every platform I got to work with, AWS, Azure and GCP so far, we kept finding bugs in their APIs, outdated or otherwise incorrect documentation and very unpredictable performance, unless you actually can run stacks at scale to average it out (as in more then just 10 - 20 instances or a larger clustered SaaS of the cloud vendor). Many times we also encountered undocumented limits that required requesting support and waiting for approval by the cloud vendor, to get even their mid-sized resources allocated. It all works very nicely on the free-tier-eligible, smallest instances and services, if as slow and high latency as is to be expected, but as soon as you actually need some decently sized storage, compute or bandwidth, it becomes quickly more expensive than what you can put in two or three datacenters for redundancy yourself, if you look at the yearly costs. So far none of the PoCs I was involved with ever got approved long term. They mostly end up as reference implementations for our customers or show case material for the corporate blog. :-( No thanks, I don't want to go back to feel that powerless as I did on closed systems ever again. Luckily, although many seem to think that cloud is the only option to run at a global scale, you can still provide lots of valuable services on the internet using robust hardware, housed in well connected datacenters.

41209 5y ago
You may be correct, for some very specific use cases. No matter how much corporate clients want to holler, a few hours of downtime isn't going to hurt anyone. I know I'd rather someone else do it, then having to drive 2 hours away to a data center at 4:00 in the morning. But I don't know exactly what you're working on , I definitely can't imagine some use cases where a few hours of down time is just unacceptable. I know I wouldn't want to run a logistics firm with servers that go down all the time.