3 ms·

Exactly, not many internal solutions have 11 9s of durability across multiple AZs like S3.

by influx 2y ago

vidarh 2y ago
Mostly because almost nobody needs it, and even fewer need it in that form. And if you need that, then even if AWS can deliver it, you need more than one supplier.
- scarface_74 2y ago
  Because dealing with that much data spread across suppliers is not going to introduce complexity and latency.
  - vidarh 2y ago
    Of course it will introduce complexity and latency, but if durability is so important that you need 11 9's, then you don't have a realistic choice unless all you care about is the appearance of it. Put another way: I was around when WorldCom went bankrupt and their datacentre in half the building we were also in was locked down by administrators. Had a server fail? Tough. You wouldn't get in. They were not Amazon of today sized, but they had $20bn/year revenue. That's one of many failure modes that can take you out if you rely on one provider. I'm uneasy about relying on one provider for far lower durability projects - I usually ensure we at least have backups somewhere else. But if you care enough that you pick AWS due to a need for 11 nines, and you don't include the long list of failure scenarios that include anything from a bankruptcy (yes, it's unlikely), to them kicking you off intentionally or by accident, or you failing to pay a bill, or any number of other scenarios, they might have 11 nines durability, but you don't have a guarantee you'll get access to it. I'm not arguing against using AWS. It's expensive, but it has its uses. I use it at my current employer because our customers want it, and it's not a const-sensitive product. Picking it as part of a strategy to manage risks is entirely valid. But it's certainly not going to save you from complexity unless you cut corners because they get you to ignore risks.
    - scarface_74 2y ago
      Well seeing that if AWS went bankrupt and couldn’t afford their servers, I assure you that the federal government would step in seeing how many government services are hosted both in the commercial, GovCloud and “secret” (an official type) regions not to mention how many financial companies are hosted there. And every cloud provider has their own set of proprietary services that you are going to need to both design around and have staff trained for and constantly be testing for. No “running K8s and open source software” is not the answer. You still need to integrate with thier IAM solution, how they manage their VMs other supporting infrastructure and often if you are large enough, you will have redundant physical connections to their data centers. Also “using Terraform” is not the answer. A TF configuration is not portable at all across vendors.
      - vidarh 2y ago
        > Well seeing that if AWS went bankrupt and couldn’t afford their servers, I assure you that the federal government would step in seeing how many government services are hosted both in the commercial, GovCloud and secret regions not to mention how many financial companies are hosted there. Maybe. Worldcom was eventually bought out too. That does not mean you can assume that there will be no disruption to services, or no loss. And that addressed one of many potential risks. Unless you can guarantee that there are no such risks whatsoever, the point I made stands. That point was simply that if you rely on a third party's durability level, then whatever that level is, the durability of the data from your point of view is lower, because there will always be a risk that your access to the data is removed either permanently or long enough for the eventual recovery to be irrelevant. And so if you actually need and care about that durability level, you'll need multiple providers. Which you probably should at far lower reliability levels anyway. I did not suggest it'd be simpler to use another cloud provider, so I have no idea why you're bringing that up as it's entirely irrelevant to the argument I made.

Exactly, not many internal solutions have 11 9s of durability across multiple AZs like S3.

by influx 2y ago

Exactly, not many internal solutions have 11 9s of durability across multiple AZs like S3.

Mostly because almost nobody needs it, and even fewer need it in that form. And if you need that, then even if AWS can deliver it, you need more than one supplier.

scarface_74 2y ago
Because dealing with that much data spread across suppliers is not going to introduce complexity and latency.
- vidarh 2y ago
  Of course it will introduce complexity and latency, but if durability is so important that you need 11 9's, then you don't have a realistic choice unless all you care about is the appearance of it. Put another way: I was around when WorldCom went bankrupt and their datacentre in half the building we were also in was locked down by administrators. Had a server fail? Tough. You wouldn't get in. They were not Amazon of today sized, but they had $20bn/year revenue. That's one of many failure modes that can take you out if you rely on one provider. I'm uneasy about relying on one provider for far lower durability projects - I usually ensure we at least have backups somewhere else. But if you care enough that you pick AWS due to a need for 11 nines, and you don't include the long list of failure scenarios that include anything from a bankruptcy (yes, it's unlikely), to them kicking you off intentionally or by accident, or you failing to pay a bill, or any number of other scenarios, they might have 11 nines durability, but you don't have a guarantee you'll get access to it. I'm not arguing against using AWS. It's expensive, but it has its uses. I use it at my current employer because our customers want it, and it's not a const-sensitive product. Picking it as part of a strategy to manage risks is entirely valid. But it's certainly not going to save you from complexity unless you cut corners because they get you to ignore risks.
  - scarface_74 2y ago
    Well seeing that if AWS went bankrupt and couldn’t afford their servers, I assure you that the federal government would step in seeing how many government services are hosted both in the commercial, GovCloud and “secret” (an official type) regions not to mention how many financial companies are hosted there. And every cloud provider has their own set of proprietary services that you are going to need to both design around and have staff trained for and constantly be testing for. No “running K8s and open source software” is not the answer. You still need to integrate with thier IAM solution, how they manage their VMs other supporting infrastructure and often if you are large enough, you will have redundant physical connections to their data centers. Also “using Terraform” is not the answer. A TF configuration is not portable at all across vendors.
    - vidarh 2y ago
      > Well seeing that if AWS went bankrupt and couldn’t afford their servers, I assure you that the federal government would step in seeing how many government services are hosted both in the commercial, GovCloud and secret regions not to mention how many financial companies are hosted there. Maybe. Worldcom was eventually bought out too. That does not mean you can assume that there will be no disruption to services, or no loss. And that addressed one of many potential risks. Unless you can guarantee that there are no such risks whatsoever, the point I made stands. That point was simply that if you rely on a third party's durability level, then whatever that level is, the durability of the data from your point of view is lower, because there will always be a risk that your access to the data is removed either permanently or long enough for the eventual recovery to be irrelevant. And so if you actually need and care about that durability level, you'll need multiple providers. Which you probably should at far lower reliability levels anyway. I did not suggest it'd be simpler to use another cloud provider, so I have no idea why you're bringing that up as it's entirely irrelevant to the argument I made.