3 ms·

I think you should always plan for failures, but modern enterprise hardware is quite reliable. I would even posit that if you stood up a brand new physical serv

I think you should always plan for failures, but modern enterprise hardware is quite reliable. I would even posit that if you stood up a brand new physical server today, it has a good chance of beating AWS uptime (well, not the AWS dashboard numbers) over a one year period.

jupp0r 4y ago
"hardware is quite reliable" is not a valid strategy. Hardware fails with some non-zero probability. You need to have a plan in place what to do if that happens, taking into account service disruption, backups etc. Having a system in place that handles most of this gracefully (like kubernetes) is one way of having such a plan, there are others. Which one works best is dependent on your app, cost of downtime, your team that's tasked with bringing everything back up in the middle if the night, etc. People who leave details like this out when they say "kubernetes is complicated" just haven't seen the complexities of operating a service well.
- fbdab103 4y ago
  My first sentence was to always plan for failures.
  - jupp0r 4y ago
    Yeah sorry it wasn't meant to dispute your argument, just as an addition.
- Nextgrid 4y ago
  Keep in mind that the complexity of your distributed system is a liability itself. Most cloud or large company downtimes aren't due to hardware failure per-se but either operator error or unexpected interactions between components of their distributed system. You might just be trading one source of downtime for another. > the complexities of operating a service well. Keep in mind that in a lot of business applications, downtime isn't the end of the world and might be an accepted and priced-in "cost" of doing business. Operating it as you consider "well" would just cost them more with no benefit.
pixl97 4y ago
It doesn't matter if hardware is 99.99% reliable if you're the .01% that day

3 ms·

I think you should always plan for failures, but modern enterprise hardware is quite reliable. I would even posit that if you stood up a brand new physical serv

by fbdab103 4y ago

jupp0r 4y ago
"hardware is quite reliable" is not a valid strategy. Hardware fails with some non-zero probability. You need to have a plan in place what to do if that happens, taking into account service disruption, backups etc. Having a system in place that handles most of this gracefully (like kubernetes) is one way of having such a plan, there are others. Which one works best is dependent on your app, cost of downtime, your team that's tasked with bringing everything back up in the middle if the night, etc. People who leave details like this out when they say "kubernetes is complicated" just haven't seen the complexities of operating a service well.
- fbdab103 4y ago
  My first sentence was to always plan for failures.
  - jupp0r 4y ago
    Yeah sorry it wasn't meant to dispute your argument, just as an addition.
- Nextgrid 4y ago
  Keep in mind that the complexity of your distributed system is a liability itself. Most cloud or large company downtimes aren't due to hardware failure per-se but either operator error or unexpected interactions between components of their distributed system. You might just be trading one source of downtime for another. > the complexities of operating a service well. Keep in mind that in a lot of business applications, downtime isn't the end of the world and might be an accepted and priced-in "cost" of doing business. Operating it as you consider "well" would just cost them more with no benefit.
pixl97 4y ago
It doesn't matter if hardware is 99.99% reliable if you're the .01% that day