4 ms·

Turns out "Cloud" is someone else's computer, with the same possible issues. Saying that, I would be very surprised if the same ever happened on S3. Not saying

Turns out "Cloud" is someone else's computer, with the same possible issues. Saying that, I would be very surprised if the same ever happened on S3. Not saying it won't, but if the specs are to be believe, seems pretty rock solid. On a separate note, really hope the industry either makes cloud compute cheaper or starts using better priced competitors/on premisis soon, the big 3 clouds are crazy expensive IMO.

jacquesm 3y ago
Interestingly: S3 loses based on their own figures some data every year. They store 100 trillion objects and they say they are 99.999999999% reliable on an annual basis. So every year they lose some objects, but the chance that these are your objects is vanishingly small.
- layoric 3y ago
  Ahh interesting, I thought this was a calculated reliability rate, not an actualized one.
  - jacquesm 3y ago
    You'd hope that with such a massive statistical base that the two would approach each other?
    - klysm 3y ago
      Well if they’ve never lost an object then how would you go about doing that.
      - tux3 3y ago
        Statistics can do that. "If we were only X% durable, what are the odds we would have made it so far with no data loss?" Then pick a threshold p value you like and that X% will be a reasonably confident estimate If other words if you think you have 99.999999999 but you really have 99.9, Nature will correct you rather quickly. If reality persistently lets you get away with it, you can counterfactually deduce you must have more nines.
        vlovich123 3y ago
        You’d have to randomly sample the set of all objects to do that. But I’m not aware of a database technique that lets you pick randomly. So I’m not sure how you’d validate the objects exist in practice. Checking 100 trillion objects seems expensive and time consuming. Doing it in flight while objects are being accessed is free but also heavily biased (ie those are files you’d be more likely to repair in the first place anyway)
        klysm 3y ago
        I don’t understand how to calculate those odds though. Like is it a continuous rate of event probability?
        tux3 3y ago
        There's different sources of failures, and when you start claiming 11 nines of durability, you really have to go and think about all the different possible causes of failure. So it's not just a flat continuous rate, it's a sum of many different things. 11 nines is such a big number, you start modelling natural catastrophes, floods, solar flares, very unlikely discrete events as well.
- thrtythreeforty 3y ago
  Nitpick: they actually make a distinction between availability and durability. Availability means "you can get your data." Durability means "we still have your data." Only durability has 11 9's.
  - layoric 3y ago
    Right, but we are talking about data loss, so I think we are all talking about durability.
  - jacquesm 3y ago
    Yes, I should have used durability. Thank you for the correction.
- medler 3y ago
  Be careful with projecting these numbers forward. As far as I can tell, there’s no SLA for durability, only for 99.9% uptime (you can get 10% off your bill if this is violated)
  - gabeio 3y ago
    > Be careful with projecting these numbers forward. As far as I can tell, there’s no SLA for durability, only for 99.9% uptime (you can get 10% off your bill if this is violated) > Designed to provide 99.999999999% durability and 99.99% availability of objects over a given year. https://docs.aws.amazon.com/AmazonS3/latest/userguide/DataDurability.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/DataDu... This isn’t in their SLA but they do post it.
- EarthLaunch 3y ago
  With those figures, that would be 1000 objects per year.
deleted 3y ago
[deleted]
ShadowBanThis01 3y ago
DropBox lost a shitload of my files, which were due to a client. They never figured out how, and the files reappeared a week later. In the meantime, I had to sign up for Google Drive and deliver the files late... looking unprofessional. Don't rely on "the cloud" unless you absolutely have to.

jacquesm 3y ago

Interestingly: S3 loses based on their own figures some data every year. They store 100 trillion objects and they say they are 99.999999999% reliable on an annual basis. So every year they lose some objects, but the chance that these are your objects is vanishingly small.

layoric 3y ago
Ahh interesting, I thought this was a calculated reliability rate, not an actualized one.
- jacquesm 3y ago
  You'd hope that with such a massive statistical base that the two would approach each other?
  - klysm 3y ago
    Well if they’ve never lost an object then how would you go about doing that.
    - tux3 3y ago
      Statistics can do that. "If we were only X% durable, what are the odds we would have made it so far with no data loss?" Then pick a threshold p value you like and that X% will be a reasonably confident estimate If other words if you think you have 99.999999999 but you really have 99.9, Nature will correct you rather quickly. If reality persistently lets you get away with it, you can counterfactually deduce you must have more nines.
      - vlovich123 3y ago
        You’d have to randomly sample the set of all objects to do that. But I’m not aware of a database technique that lets you pick randomly. So I’m not sure how you’d validate the objects exist in practice. Checking 100 trillion objects seems expensive and time consuming. Doing it in flight while objects are being accessed is free but also heavily biased (ie those are files you’d be more likely to repair in the first place anyway)
      - klysm 3y ago
        I don’t understand how to calculate those odds though. Like is it a continuous rate of event probability?
        tux3 3y ago
        There's different sources of failures, and when you start claiming 11 nines of durability, you really have to go and think about all the different possible causes of failure. So it's not just a flat continuous rate, it's a sum of many different things. 11 nines is such a big number, you start modelling natural catastrophes, floods, solar flares, very unlikely discrete events as well.
thrtythreeforty 3y ago
Nitpick: they actually make a distinction between availability and durability. Availability means "you can get your data." Durability means "we still have your data." Only durability has 11 9's.
- layoric 3y ago
  Right, but we are talking about data loss, so I think we are all talking about durability.
- jacquesm 3y ago
  Yes, I should have used durability. Thank you for the correction.
medler 3y ago
Be careful with projecting these numbers forward. As far as I can tell, there’s no SLA for durability, only for 99.9% uptime (you can get 10% off your bill if this is violated)
- gabeio 3y ago
  > Be careful with projecting these numbers forward. As far as I can tell, there’s no SLA for durability, only for 99.9% uptime (you can get 10% off your bill if this is violated) > Designed to provide 99.999999999% durability and 99.99% availability of objects over a given year. https://docs.aws.amazon.com/AmazonS3/latest/userguide/DataDurability.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/DataDu... This isn’t in their SLA but they do post it.
EarthLaunch 3y ago
With those figures, that would be 1000 objects per year.

deleted 3y ago

[deleted]

ShadowBanThis01 3y ago

DropBox lost a shitload of my files, which were due to a client. They never figured out how, and the files reappeared a week later. In the meantime, I had to sign up for Google Drive and deliver the files late... looking unprofessional. Don't rely on "the cloud" unless you absolutely have to.