4 ms·

Would you trust cloud providers to never slip up though? I don't think the split is "perfect but pricey" vs "less expensive but shaky". It feels similar to the

Would you trust cloud providers to never slip up though? I don't think the split is "perfect but pricey" vs "less expensive but shaky". It feels similar to the autonomous driving question, where safety of autonomous driving is often compared to perfection instead of human drivers. Would UniSuper engineers have trashed their entire infrastructure the way Google Cloud did? Granted, that's not a common thing to happen, but they got _very_ lucky that day that they had a backup elsewhere and didn't fully commit to trusting Google.

paulryanrogers 1y ago
There is an element of "never fired for buying IBM" involved. If AWS screws up then theoretically they've a bigger bank roll for any settlement or judgement.
jauntywundrkind 1y ago
Nothing's perfect? But theres a lot of orders of magnitude difference in investment in what most companies can and will do versus what a hyperscaler will! And while your company is a single petri dish with your own specific problems, these folks have to see a lot of shit. Sometimes yes it all goes bad even there anyways! The resiliency fails! But given how much they've thought about it & how much they've hedged against 1 in a million 1 in a billion cases that they see semi regularly that your company might never notice until it's too late, I think there's a ton of reasons to trust these generally very well exercises systems with pretty sizable engineering teams & deployments behind them. More so than what most folks can bring to bear. These matters are existential threats for your business. And they are existential threats for big providers, for HyperScalers, even more so. Google very publicly owned up to UniSuper, found out why it happened (a blank form on an internal tool lead to the account having a 1 year time limit with auto-deletion), and publicly announced their three fold plan to make sure at least this specific horribly threat would never happen again: > We deprecated the internal tool that triggered this sequence of events… We scrubbed the system database and manually reviewed all GCVE Private Clouds… We corrected the system behavior that sets GCVE Private Clouds for deletion for such deployment workflows. https://cloud.google.com/blog/products/infrastructure/details-of-google-cloud-gcve-incident/ https://cloud.google.com/blog/products/infrastructure/detail... The mere fact that this was such a notable event should be itself notable. In general, the hyperscalers have been stupendously reliable. And absurdly secure. They have small armies dedicated insuring both. I look forward to this new era where finally we might have computing of our own good enough & broadly known & trusted enough to make doing our own - even with small teams - viable!! But heavens yes, keep off-site backups!!!
- luckylion 1y ago
  The scale becomes not quite as impressive if you consider that those thousands or tens of thousands of engineers also maintain hundreds more services in vastly more complex scenarios because they need to cover many more use cases than yours. Most companies don't need to run everything AWS has to offer, so they wouldn't need to handle all of that complexity. Individual AZs have significantly worse uptime than any DC I've ever worked with. They are very reliable, if you are multi-region - and you hire a small army of consultants.

paulryanrogers 1y ago

There is an element of "never fired for buying IBM" involved. If AWS screws up then theoretically they've a bigger bank roll for any settlement or judgement.

jauntywundrkind 1y ago

Nothing's perfect? But theres a lot of orders of magnitude difference in investment in what most companies can and will do versus what a hyperscaler will! And while your company is a single petri dish with your own specific problems, these folks have to see a lot of shit. Sometimes yes it all goes bad even there anyways! The resiliency fails! But given how much they've thought about it & how much they've hedged against 1 in a million 1 in a billion cases that they see semi regularly that your company might never notice until it's too late, I think there's a ton of reasons to trust these generally very well exercises systems with pretty sizable engineering teams & deployments behind them. More so than what most folks can bring to bear. These matters are existential threats for your business. And they are existential threats for big providers, for HyperScalers, even more so. Google very publicly owned up to UniSuper, found out why it happened (a blank form on an internal tool lead to the account having a 1 year time limit with auto-deletion), and publicly announced their three fold plan to make sure at least this specific horribly threat would never happen again: > We deprecated the internal tool that triggered this sequence of events… We scrubbed the system database and manually reviewed all GCVE Private Clouds… We corrected the system behavior that sets GCVE Private Clouds for deletion for such deployment workflows. https://cloud.google.com/blog/products/infrastructure/details-of-google-cloud-gcve-incident/ https://cloud.google.com/blog/products/infrastructure/detail... The mere fact that this was such a notable event should be itself notable. In general, the hyperscalers have been stupendously reliable. And absurdly secure. They have small armies dedicated insuring both. I look forward to this new era where finally we might have computing of our own good enough & broadly known & trusted enough to make doing our own - even with small teams - viable!! But heavens yes, keep off-site backups!!!

luckylion 1y ago
The scale becomes not quite as impressive if you consider that those thousands or tens of thousands of engineers also maintain hundreds more services in vastly more complex scenarios because they need to cover many more use cases than yours. Most companies don't need to run everything AWS has to offer, so they wouldn't need to handle all of that complexity. Individual AZs have significantly worse uptime than any DC I've ever worked with. They are very reliable, if you are multi-region - and you hire a small army of consultants.