6 ms·

That is actually surprising to me. Claims like that are pretty common, they make sense and they should be true, so even though I don't really know AWS (/Backbla

by krick 18d ago

That is actually surprising to me. Claims like that are pretty common, they make sense and they should be true, so even though I don't really know AWS (/Backblaze/Azure/whatever) redundancy planning in enough detail, I used to trust them. It's really worrying when they outright say it will be ok, and then a week later it turns out to be not ok.

houssc 18d ago
The caveat is always "if you're using the service correctly" which is not necessarily free. Meaning taking advantage of multiple geo zones, building in redundancy to your stack, etc. Like everything he said is possible if your technology stack living in AWS was designed to survive it. Everyone who has ever had the "we lost your data" email from AWS knows at the end of the day the cloud is just someone else's data center with neat provisioning tools and services.
- jacquesm 18d ago
  The problem is that lots of people seem to be under the impression that they are doing it right because they are using AWS. They don't realize that AWS is a toolbox, not a 'ready made solution for redundancy against all catastrophes you are possibly exposed to'. They use that to their advantage by pricing such solutions at a level that people will either pay through the nose or will be left without recourse when AWS loses their data. It's stupid, but at the same time these beliefs are surprisingly wide spread.
  - jaggederest 18d ago
    If I were a bit more bloody-minded I would launch a service for vibe-coded apps that, under the hood, did everything "the right way" and just charged a flat fee + percent on the underlying. I feel like if this was done correctly it would eat a bunch of the market, but I question how many people are actually willing to pay for "the right way". The last time I had that experience it was with Heroku which was quite a leaky abstraction.
    - xboxnolifes 18d ago
      I think a lot of people have built those services. But doing it right is more expensive, and people end up choosing the $5-$10/mo option over your $30/mo+ that does it right. Multiply those numbers by whatever multiple you want for higher end stuff.
    - zxilly 18d ago
      And the LLM say: "You're absolutely right! We didn't need jaggederest's service. We can build it easily."
      - jaggederest 18d ago
        Well, one of the things I would be very, very focused on is LLM optimization, so that is something that could be mitigated, at least. Especially since I would be building with it, certainly, I would try to engineer it so that it was to Mr. Claude / Mr. Gippity's taste
    - ngc248 18d ago
      1. There is no single "the right way", different app have different "right ways" 2. As for the flat fee, it only works initially when things are simple, as time goes on and your business and usecases you support grows, a flat fee won't work anymore.
      - jaggederest 18d ago
        I mean I would be limiting it to a very specific use case, and the flat fee would be "plus a percentage of the underlying", something like $50/mo + 20% of the AWS spend underlying. And for that, with a simplified use case, well, they can scale up to $20k+ a month if they like, that would be ideal, and if they need a more complicated setup or enterpriseyness, migrate off with my blessing and available-not-required hands on support (again for a reasonable fee, maybe $10k if you want the white glove).
  - Avicebron 18d ago
    Not to be callous but even the 3-2-1 rule is pretty basic, the issue is that people don't apply it. But that's a hiring and strategy thing.
    - SlightlyLeftPad 18d ago
      [flagged]
      - sokoloff 18d ago
        What?! Publicly traded companies are not “legally bound to return growth in share price or dividends.” There are plenty of companies who pay no dividends and have not returned growth in share price. They’re still operating and no one’s coming to throw the execs in jail. On the off chance that there is such a law, please cite it.
        upboundspiral 18d ago
        Not a law perhaps, but an ingrained neoliberal philosophy that is pervasive in certain management cultures in the US definitely.
        dotancohen 18d ago
        Neoliberal philosophy is stable ground for a lawsuit?
        paganel 18d ago
        In a Gramscian understanding of the world it is, yes. Forget Gramsci, right now I'm finishing reading a book by Maurice Gauchet [1] (not a leftist by any means) whos's saying exactly that, i.e. that neo-liberalism is very much defined by its insistence on relying on " le juridique" > Ce dérèglement se manifeste avant tout par la réduction de la démocratie à une logique juridique, le néolibéralisme ayant imposé une conception où le droit, érigé en garant exclusif de la liberté individuelle, relègue le politique à une fonction purement gestionnaire automatically translated as > This dysfunction manifests itself primarily through the reduction of democracy to a legalistic logic, as neoliberalism has imposed a conception in which the law—elevated to the status of exclusive guarantor of individual liberty—relegates the political realm to a purely managerial function. [1] https://en.wikipedia.org/wiki/Marcel_Gauchet https://en.wikipedia.org/wiki/Marcel_Gauchet
  - houssc 18d ago
    Yep, a lot of non-technical leaders believe that 'cloud' is synonymous with 'DR strategy' or even 'backup'. "We won't have to worry about being offline if our server goes down if we move to the cloud!" Some of these people fundamentally don't understand what the cloud is, their assumption is cloud means easy button that solves all your infrastructure and uptime problems.
    - mhitza 18d ago
      You are pointing the fingers at the wrong people. Cloud providers pushed this idea onto executives via their marketing and conferencing channels. Not disimilar to how AI naratives are pushed on executives these last two years.
      - albert_e 18d ago
        But alongside the marketing blitz they also offer certifications for people who are supposed to execute these projects. Even the most foundational certification exam -- which simply tests your recall on what AWS service is for compute versus networking -- teaches and tests you about "Shared Responsibility Model" that draws the line at what the customer is still responsible for when they use cloud bases IaaS / PaaS / SaaS services. If businesses are going to cloud but without engaging / listening to competent people who know these basics -- then the blame needs to be somewhat pointed back at those very business leaders I feel. This is not obscure magical knowledge either that is tightly controlled. Any cloud vendor will freely teach you that. Or even a google search would.
        lelanthran 18d ago
        Who is a CEO going to believe? Amazon, Microsoft and the entire IT infra division or that lone SRE who disagrees with them? I mean, every time cloud comes up on HN we see legions of techies posting strowman arguments about why you should offload everything onto AWS, GCP, Azure, etc.
        avereveard 18d ago
- marcosdumay 18d ago
  No, that quotation on the GP clearly states that AWS has enough redundancy within the same region that they will continue all services running on it if a datacenter is destroyed. It's very clearly not about you being able to set-up redundancy for yourself.
  - cyberax 18d ago
    That's actually true. AWS is designed to survive one datacenter being offline (which happened more than once, btw). When the first DC in ME was hit, AWS continued working normally, with only a few services experiencing issues. But it's not designed to survive TWO datacenters going offline, and in a permanent fashion.
    - marcosdumay 18d ago
      That's fair. The only reference was about one datacenter blowing up. You can't have unlimited redundancy.
    - Dylan16807 18d ago
      Aren't they supposed to have 3 copies of everything? If so, two datacenters going offline at the same time should mean almost zero data loss.
      - dotancohen 18d ago
        No. _You_ are supposed to have three copies of everything - and two of those should be off AWS.
        Dylan16807 18d ago
        Yeah yeah I know about 3-2-1. But when I put data on normal S3, isn't my money supposedly paying for triple redundancy? Look, here's the documentation: https://docs.aws.amazon.com/AmazonS3/latest/userguide/DataDurability.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/DataDu... "S3 Standard, S3 Intelligent-Tiering, S3 Standard-IA, S3 Glacier Instant Retrieval, S3 Glacier Flexible Retrieval, and S3 Glacier Deep Archive redundantly store objects on multiple devices across a minimum of three Availability Zones in an AWS Region. An Availability Zone is one or more discrete data centers with redundant power, networking, and connectivity in an AWS Region." As far as I've heard before that's not with parity and three zones means three copies. But when I search now I see things about 5+4 parity, any insight here?
- testbjjl 18d ago
  Pretty much this, it’s your responsibility to use their tools to make sure your data is managed in such a way that any data destroyed is already elsewhere before the event.
- thayne 18d ago
  > which is not necessarily free Not just in terms of service costs, but in time and complexity. In many cases building out that complexity is complicated and difficult. And sometimes the functionality you need isn't supported in the regions you use.
- mitxela 18d ago
  Amazon never said you had to mirror your data across multiple regions to prevent Amazon-caused data loss.
  - Brigand 18d ago
    It’s not really Amazon-caused
    - mitxela 18d ago
      Yes it is, they didn't have adequate redundancy for this extremely foreseeable event (military site getting hit by missiles).
      - expedition32 18d ago
        War?! In my Middle East? Preposterous.
      - IAmBroom 17d ago
        Negligence is not malicious action. Both are bad, but having a missile hit your shop is never going to mean you are causing violence.
        mitxela 16d ago
        But you are causing your customers to lose data
    - deleted 17d ago
      [deleted]
sandeepkd 18d ago
The devil is always in the details. Somehow I feel that when we offload the responsibility to some one else we get this feeling that the other person/entity would be doing full diligence and whatever else is required to carry out the job perfectly. However in reality most of the times they just do the bare minimum to pass your evaluation criteria to get the job.
- jongjong 18d ago
  This is almost always the case. It's one of the frustrating things about the software industry; because everything is much more complicated than the customer is able to comprehend, a software company can promise anything and the customer can't actually verify. So any software company/project which actually took the time and effort to fully handle the enormous complexity, they can't sell themselves based on that fact because every other company (who didn't invest the effort) is also claiming it and the customer has no mechanism to verify the claims until some major rare event occurs. And most of the effort is required precisely to handle those 1% of rare situations.
- rob74 18d ago
  Especially with Amazon, who are well known for squeezing every last bit of profit from their employees, contractors etc., it doesn't really sound surprising. "Offsite backups?! Sure, you could have had that if you had found the right page in the AWS console and if you would have paid 50% extra!"
  - whizzter 18d ago
    Perhaps, cheapness is always an factor but I'm potentially reading it as the customers perhaps not wanting data to move outside of the country and with AWS only have one datacenter in said country produced this result.
    - sharemywin 17d ago
      but the general assumption is your data isn't going to get lost if you use Amazon. so once the trust is gone it's gone.
deleted 18d ago
[deleted]
deleted 18d ago
[deleted]
brador 18d ago
“It’s not the cloud. It’s just someone else’s computer.” - MM
Melatonic 18d ago
It's probably a dedicated government type thing where the data is housed seperately from normal AWS
general_reveal 18d ago
Well, listen, you do know that even when the largest most ambitious and most sophisticated things are shipped, each owner of each specific part (could be many many owners) basically , to the best of their ability, prayed that nothing particularly bad happens when it’s shipped off. Truly, that’s the best a mortal human can do, pray their part doesn’t break. So then that big thing comes to you. It’s all kind of … held together by a prayer … Trust me I’ve worked at these big places. You wouldn’t believe how much fucking luck and grace from God is allowing you to do anything with your digital life. It’s a mindfuck of a tangled mess out there, eternities worth of written code that only God ensures works together at this point, only to get more hidden with AI.
gregates 18d ago
It really is important to understand the failure modes that the durability model accounts for and what it doesn't. It only accounts for "normal" failures, like an HDD reaching end of life. For example, you mention Backblaze. Backblaze has public posts about their durability model. They claim to use 17:20 Reed-Solomon erasure encoding. That means there are 20 shards of a blob, and you can lose 3 of them and still reconstruct the blob. Think about that for a second. If they store 4 shards in a datacenter, that means that a loss of that one datacenter is sufficient to lose the blob, forever. That entails that blobs are sharded across a minimum of 7 data centers, or the loss of one data center might mean permanent data loss. Which one do you think is true? (In fact it's pretty clear from Backblaze's public posts that they don't shard across data centers at all, only across racks within a data center.) Now, AWS's availability guarantee — not their durability guarantee — entails that they use a less cost-effective erasure coding ratio. S3 is designed so that your blob is available even if a whole AZ goes down, and it's well known that most AWS regions have only 3 AZs. Therefore, if you tolerate the same number of shards lost to HDD failure as Backblaze in your durability model (3), then you might need 17:30 erasure coding to get the same durability and the required availability. That means S3 is storing way more physical bytes than Backblaze — 1.76x the logical size of the blob, instead of Backblaze's 1.18x. That's more expensive, but it also gives you better availability. Which is also why One Zone S3 is cheaper — if you don't care about the availability guarantee, S3 can do what Backblaze does and save 33% on physical bytes, and they pass on 40–50% of those savings to the customer (this is fairer than it sounds — there's more overhead than physical storage bytes). But here's the thing. AWS has more redundancy built in than Backblaze because they make availability guarantees in addition to durability guarantees. BUT the durability model is the same, which is why Backblaze can claim equivalent durability to S3. S3 in fact has better durability — they can survive the permanent loss of an AZ without necessarily losing blobs stored there (with the exception of One Zone blobs), and Backblaze cannot. But that's not actually a factor of the durability model, which is just taking into account normal events like HDD failure. Instead, S3 has durability that's more resilient to AZ loss because of their availability model. It's a side effect that isn't actually part of the durability promise!
- everfrustrated 18d ago
  As far as I'm aware Backblaze stores data only within a single datacenter (for a given region). This likely made sense in their original business model of being "offsite" copy of data. But it very much breaks down for B2 where they're now storing original data. I hope they rethink this model. You do get what you pay for. There's a reason they're cheap.
marcusverus 18d ago
The question is about one AWS facility. The headline refers to facilities. The guy's answer may well be an honest and truthful one.
edgyquant 18d ago
How much data is stored in the average aws data center? Assuming they do regularly back ups, it makes sense that recent data can’t be instantly backed up and so there will always be some data in transit or queued up no?

by krick 18d ago

houssc 18d ago

The caveat is always "if you're using the service correctly" which is not necessarily free. Meaning taking advantage of multiple geo zones, building in redundancy to your stack, etc. Like everything he said is possible if your technology stack living in AWS was designed to survive it. Everyone who has ever had the "we lost your data" email from AWS knows at the end of the day the cloud is just someone else's data center with neat provisioning tools and services.

jacquesm 18d ago
The problem is that lots of people seem to be under the impression that they are doing it right because they are using AWS. They don't realize that AWS is a toolbox, not a 'ready made solution for redundancy against all catastrophes you are possibly exposed to'. They use that to their advantage by pricing such solutions at a level that people will either pay through the nose or will be left without recourse when AWS loses their data. It's stupid, but at the same time these beliefs are surprisingly wide spread.
- jaggederest 18d ago
  If I were a bit more bloody-minded I would launch a service for vibe-coded apps that, under the hood, did everything "the right way" and just charged a flat fee + percent on the underlying. I feel like if this was done correctly it would eat a bunch of the market, but I question how many people are actually willing to pay for "the right way". The last time I had that experience it was with Heroku which was quite a leaky abstraction.
  - xboxnolifes 18d ago
    I think a lot of people have built those services. But doing it right is more expensive, and people end up choosing the $5-$10/mo option over your $30/mo+ that does it right. Multiply those numbers by whatever multiple you want for higher end stuff.
  - zxilly 18d ago
    And the LLM say: "You're absolutely right! We didn't need jaggederest's service. We can build it easily."
    - jaggederest 18d ago
      Well, one of the things I would be very, very focused on is LLM optimization, so that is something that could be mitigated, at least. Especially since I would be building with it, certainly, I would try to engineer it so that it was to Mr. Claude / Mr. Gippity's taste
  - ngc248 18d ago
    1. There is no single "the right way", different app have different "right ways" 2. As for the flat fee, it only works initially when things are simple, as time goes on and your business and usecases you support grows, a flat fee won't work anymore.
    - jaggederest 18d ago
      I mean I would be limiting it to a very specific use case, and the flat fee would be "plus a percentage of the underlying", something like $50/mo + 20% of the AWS spend underlying. And for that, with a simplified use case, well, they can scale up to $20k+ a month if they like, that would be ideal, and if they need a more complicated setup or enterpriseyness, migrate off with my blessing and available-not-required hands on support (again for a reasonable fee, maybe $10k if you want the white glove).
- Avicebron 18d ago
  Not to be callous but even the 3-2-1 rule is pretty basic, the issue is that people don't apply it. But that's a hiring and strategy thing.
  - SlightlyLeftPad 18d ago
    [flagged]
    - sokoloff 18d ago
      What?! Publicly traded companies are not “legally bound to return growth in share price or dividends.” There are plenty of companies who pay no dividends and have not returned growth in share price. They’re still operating and no one’s coming to throw the execs in jail. On the off chance that there is such a law, please cite it.
      - upboundspiral 18d ago
        Not a law perhaps, but an ingrained neoliberal philosophy that is pervasive in certain management cultures in the US definitely.
        dotancohen 18d ago
        Neoliberal philosophy is stable ground for a lawsuit?
        paganel 18d ago
        In a Gramscian understanding of the world it is, yes. Forget Gramsci, right now I'm finishing reading a book by Maurice Gauchet [1] (not a leftist by any means) whos's saying exactly that, i.e. that neo-liberalism is very much defined by its insistence on relying on " le juridique" > Ce dérèglement se manifeste avant tout par la réduction de la démocratie à une logique juridique, le néolibéralisme ayant imposé une conception où le droit, érigé en garant exclusif de la liberté individuelle, relègue le politique à une fonction purement gestionnaire automatically translated as > This dysfunction manifests itself primarily through the reduction of democracy to a legalistic logic, as neoliberalism has imposed a conception in which the law—elevated to the status of exclusive guarantor of individual liberty—relegates the political realm to a purely managerial function. [1] https://en.wikipedia.org/wiki/Marcel_Gauchet https://en.wikipedia.org/wiki/Marcel_Gauchet
- houssc 18d ago
  Yep, a lot of non-technical leaders believe that 'cloud' is synonymous with 'DR strategy' or even 'backup'. "We won't have to worry about being offline if our server goes down if we move to the cloud!" Some of these people fundamentally don't understand what the cloud is, their assumption is cloud means easy button that solves all your infrastructure and uptime problems.
  - mhitza 18d ago
    You are pointing the fingers at the wrong people. Cloud providers pushed this idea onto executives via their marketing and conferencing channels. Not disimilar to how AI naratives are pushed on executives these last two years.
    - albert_e 18d ago
      But alongside the marketing blitz they also offer certifications for people who are supposed to execute these projects. Even the most foundational certification exam -- which simply tests your recall on what AWS service is for compute versus networking -- teaches and tests you about "Shared Responsibility Model" that draws the line at what the customer is still responsible for when they use cloud bases IaaS / PaaS / SaaS services. If businesses are going to cloud but without engaging / listening to competent people who know these basics -- then the blame needs to be somewhat pointed back at those very business leaders I feel. This is not obscure magical knowledge either that is tightly controlled. Any cloud vendor will freely teach you that. Or even a google search would.
      - lelanthran 18d ago
        Who is a CEO going to believe? Amazon, Microsoft and the entire IT infra division or that lone SRE who disagrees with them? I mean, every time cloud comes up on HN we see legions of techies posting strowman arguments about why you should offload everything onto AWS, GCP, Azure, etc.
        avereveard 18d ago
marcosdumay 18d ago
No, that quotation on the GP clearly states that AWS has enough redundancy within the same region that they will continue all services running on it if a datacenter is destroyed. It's very clearly not about you being able to set-up redundancy for yourself.
- cyberax 18d ago
  That's actually true. AWS is designed to survive one datacenter being offline (which happened more than once, btw). When the first DC in ME was hit, AWS continued working normally, with only a few services experiencing issues. But it's not designed to survive TWO datacenters going offline, and in a permanent fashion.
  - marcosdumay 18d ago
    That's fair. The only reference was about one datacenter blowing up. You can't have unlimited redundancy.
  - Dylan16807 18d ago
    Aren't they supposed to have 3 copies of everything? If so, two datacenters going offline at the same time should mean almost zero data loss.
    - dotancohen 18d ago
      No. _You_ are supposed to have three copies of everything - and two of those should be off AWS.
      - Dylan16807 18d ago
        Yeah yeah I know about 3-2-1. But when I put data on normal S3, isn't my money supposedly paying for triple redundancy? Look, here's the documentation: https://docs.aws.amazon.com/AmazonS3/latest/userguide/DataDurability.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/DataDu... "S3 Standard, S3 Intelligent-Tiering, S3 Standard-IA, S3 Glacier Instant Retrieval, S3 Glacier Flexible Retrieval, and S3 Glacier Deep Archive redundantly store objects on multiple devices across a minimum of three Availability Zones in an AWS Region. An Availability Zone is one or more discrete data centers with redundant power, networking, and connectivity in an AWS Region." As far as I've heard before that's not with parity and three zones means three copies. But when I search now I see things about 5+4 parity, any insight here?
testbjjl 18d ago
Pretty much this, it’s your responsibility to use their tools to make sure your data is managed in such a way that any data destroyed is already elsewhere before the event.
thayne 18d ago
> which is not necessarily free Not just in terms of service costs, but in time and complexity. In many cases building out that complexity is complicated and difficult. And sometimes the functionality you need isn't supported in the regions you use.
mitxela 18d ago
Amazon never said you had to mirror your data across multiple regions to prevent Amazon-caused data loss.
- Brigand 18d ago
  It’s not really Amazon-caused
  - mitxela 18d ago
    Yes it is, they didn't have adequate redundancy for this extremely foreseeable event (military site getting hit by missiles).
    - expedition32 18d ago
      War?! In my Middle East? Preposterous.
    - IAmBroom 17d ago
      Negligence is not malicious action. Both are bad, but having a missile hit your shop is never going to mean you are causing violence.
      - mitxela 16d ago
        But you are causing your customers to lose data
  - deleted 17d ago
    [deleted]

sandeepkd 18d ago

The devil is always in the details. Somehow I feel that when we offload the responsibility to some one else we get this feeling that the other person/entity would be doing full diligence and whatever else is required to carry out the job perfectly. However in reality most of the times they just do the bare minimum to pass your evaluation criteria to get the job.

jongjong 18d ago
This is almost always the case. It's one of the frustrating things about the software industry; because everything is much more complicated than the customer is able to comprehend, a software company can promise anything and the customer can't actually verify. So any software company/project which actually took the time and effort to fully handle the enormous complexity, they can't sell themselves based on that fact because every other company (who didn't invest the effort) is also claiming it and the customer has no mechanism to verify the claims until some major rare event occurs. And most of the effort is required precisely to handle those 1% of rare situations.
rob74 18d ago
Especially with Amazon, who are well known for squeezing every last bit of profit from their employees, contractors etc., it doesn't really sound surprising. "Offsite backups?! Sure, you could have had that if you had found the right page in the AWS console and if you would have paid 50% extra!"
- whizzter 18d ago
  Perhaps, cheapness is always an factor but I'm potentially reading it as the customers perhaps not wanting data to move outside of the country and with AWS only have one datacenter in said country produced this result.
  - sharemywin 17d ago
    but the general assumption is your data isn't going to get lost if you use Amazon. so once the trust is gone it's gone.

deleted 18d ago

[deleted]

deleted 18d ago

[deleted]

brador 18d ago

“It’s not the cloud. It’s just someone else’s computer.” - MM

Melatonic 18d ago

It's probably a dedicated government type thing where the data is housed seperately from normal AWS

general_reveal 18d ago

Well, listen, you do know that even when the largest most ambitious and most sophisticated things are shipped, each owner of each specific part (could be many many owners) basically , to the best of their ability, prayed that nothing particularly bad happens when it’s shipped off. Truly, that’s the best a mortal human can do, pray their part doesn’t break. So then that big thing comes to you. It’s all kind of … held together by a prayer … Trust me I’ve worked at these big places. You wouldn’t believe how much fucking luck and grace from God is allowing you to do anything with your digital life. It’s a mindfuck of a tangled mess out there, eternities worth of written code that only God ensures works together at this point, only to get more hidden with AI.

gregates 18d ago

It really is important to understand the failure modes that the durability model accounts for and what it doesn't. It only accounts for "normal" failures, like an HDD reaching end of life. For example, you mention Backblaze. Backblaze has public posts about their durability model. They claim to use 17:20 Reed-Solomon erasure encoding. That means there are 20 shards of a blob, and you can lose 3 of them and still reconstruct the blob. Think about that for a second. If they store 4 shards in a datacenter, that means that a loss of that one datacenter is sufficient to lose the blob, forever. That entails that blobs are sharded across a minimum of 7 data centers, or the loss of one data center might mean permanent data loss. Which one do you think is true? (In fact it's pretty clear from Backblaze's public posts that they don't shard across data centers at all, only across racks within a data center.) Now, AWS's availability guarantee — not their durability guarantee — entails that they use a less cost-effective erasure coding ratio. S3 is designed so that your blob is available even if a whole AZ goes down, and it's well known that most AWS regions have only 3 AZs. Therefore, if you tolerate the same number of shards lost to HDD failure as Backblaze in your durability model (3), then you might need 17:30 erasure coding to get the same durability and the required availability. That means S3 is storing way more physical bytes than Backblaze — 1.76x the logical size of the blob, instead of Backblaze's 1.18x. That's more expensive, but it also gives you better availability. Which is also why One Zone S3 is cheaper — if you don't care about the availability guarantee, S3 can do what Backblaze does and save 33% on physical bytes, and they pass on 40–50% of those savings to the customer (this is fairer than it sounds — there's more overhead than physical storage bytes). But here's the thing. AWS has more redundancy built in than Backblaze because they make availability guarantees in addition to durability guarantees. BUT the durability model is the same, which is why Backblaze can claim equivalent durability to S3. S3 in fact has better durability — they can survive the permanent loss of an AZ without necessarily losing blobs stored there (with the exception of One Zone blobs), and Backblaze cannot. But that's not actually a factor of the durability model, which is just taking into account normal events like HDD failure. Instead, S3 has durability that's more resilient to AZ loss because of their availability model. It's a side effect that isn't actually part of the durability promise!

everfrustrated 18d ago
As far as I'm aware Backblaze stores data only within a single datacenter (for a given region). This likely made sense in their original business model of being "offsite" copy of data. But it very much breaks down for B2 where they're now storing original data. I hope they rethink this model. You do get what you pay for. There's a reason they're cheap.

marcusverus 18d ago

The question is about one AWS facility. The headline refers to facilities. The guy's answer may well be an honest and truthful one.

edgyquant 18d ago

How much data is stored in the average aws data center? Assuming they do regularly back ups, it makes sense that recent data can’t be instantly backed up and so there will always be some data in transit or queued up no?