3 ms·

I was wondering if the state of the world in 2020 might have dramatically changed their business / throughput / access patterns in a meaningful enough way to ca

I was wondering if the state of the world in 2020 might have dramatically changed their business / throughput / access patterns in a meaningful enough way to cause this dip. I'm not sure if they have a measure of the disk utilization or read/write load along with the failure rate.

brianwski 6y ago
Disclaimer: I work at Backblaze, but mostly on the client that runs on desktops and laptops. > I'm not sure if Backblaze has a measure of the disk utilization or read/write load along with the failure rate. We publish the complete hard drive SMART stats for anybody to attempt these analysis. Most of us in Backblaze engineering get giddy with excitement when a new article comes out that looks at correlating SMART stats and failures. :-) For example, this article circulated widely at Backblaze a few days ago: https://datto.engineering/post/predicting-hard-drive-failure-with-machine-learning https://datto.engineering/post/predicting-hard-drive-failure...
- codezero 6y ago
  I anticipate these reports every year and have strong trust in the data - I want to make that clear - Backblaze has done a massive service to the entire industry by collecting and aggregating this kind of data. I'm really super curious about the dip in errors over the past year :)
  - rootusrootus 6y ago
    Whether intentional or not, it's also great word-of-mouth advertising. My preexisting experience with Backblaze's hard drive stats reporting definitely worked positively in their favor when I was looking for a new backup service.
    - brianwski 6y ago
      > Whether intentional or not, it's also great word-of-mouth advertising. Oh, this has really worked out for Backblaze and we know it. The first time we published our drive failure rates (I think January of 2014?) a few people said, "Uh oh, now Backblaze will get sued by the drive manufacturers." And we cringed and waited. :-) But the lawsuit never came, in fact there were NO repercussions, only increased visibility. People who have never heard of our company before find the data interesting, and then they ask "hey, what does this company do to own this many drives?" And a few (like yourself) sign up for the service. Existing customers seem to stick with us for a long time, and even recommend us to other friends and family from time to time. So one tech person who stumbles across these stats might ACTUALLY bring us 3 or 4 more customers over the next 5 years. That's real money to us. Not only have the drive manufacturers not sued us, they are actually NICE to us beyond the scale of our actual drive purchases! In one amusing example, our drive stats were used in a lawsuit as evidence. To be clear Backblaze was not the plaintiff or the defendant in the court case, we had no skin in the game at all and didn't want to be involved, but our drive data (and internal emails) were subpoenaed to be entered into evidence. Before we were served, the drive manufacturer called us and apologized for the inconvenience and made it clear they had no beef with us. Yes, a multi-national company that makes BILLIONS of dollars per year called a 40 person company (at the time) that could barely make payroll each month to apologize for the inconvenience. :-) We thought it was very considerate of them, and a little amusing. I'm proud to be the one that "signed" the papers indicating Backblaze had been "served".
      - rootusrootus 6y ago
        Clicked on your profile and noticed that you are a fellow Beaver. Go Beavs! ;-). I got my CS degree from OSU about 10 years after you graduated.
        brianwski 6y ago
        > noticed that you are a fellow Beaver. Go Beavs! ;-) Ha! This is a silly novelty "Beaver Baseball Cap" I was given during my internship in 1988 at Hewlett-Packard in Corvallis: https://i.imgur.com/G0rPHGP.jpg https://i.imgur.com/G0rPHGP.jpg For 32 years it has sat on my computer monitor or on a shelf nearby. Nobody really asks about it anymore. I got pretty lucky on timing at OSU. The year before I was there they taught beginning programming on punch cards on a mainframe computer called a "CDC Cyber". In my freshman year I took Pascal on the brand new 1984 Macintoshes in the Computer Science department, while the engineering students still learned Fortran on the mainframe my freshman year. In 1992 I got a job as a Software Engineer at Apple in Cupertino, and you can trace that straight back to my blind luck of starting my programming education at OSU on the Mac in the exact correct year. Well, I'd rather be lucky than good. And I've ALWAYS been lucky in my career.
- matmatmatmat 6y ago
  Hi Brian, just a note of thanks to you and Backblaze for publishing these data. I always refer to them before a purchase and they're really helpful.
- willis936 6y ago
  There are other interesting factors to look for as well. Temperature, moisture, electrical noise on the power rails, infrasound, etc.
- rachelwenzel 6y ago
  Wow- crazy to see people at Backblaze actually read that article!! Thank you guys for putting out so much data and writing so much about your findings- it was HUGE in helping me come to conclusions about what's realistic to assume from SMART stats. Y'all are doing some really really cool stuff.
  - atYevP 6y ago
    Yev here -> of course we read it! It's a data-rich article about data failure! And the kudos is mutual - loved the post! :D

brianwski 6y ago

Disclaimer: I work at Backblaze, but mostly on the client that runs on desktops and laptops. > I'm not sure if Backblaze has a measure of the disk utilization or read/write load along with the failure rate. We publish the complete hard drive SMART stats for anybody to attempt these analysis. Most of us in Backblaze engineering get giddy with excitement when a new article comes out that looks at correlating SMART stats and failures. :-) For example, this article circulated widely at Backblaze a few days ago: https://datto.engineering/post/predicting-hard-drive-failure-with-machine-learning https://datto.engineering/post/predicting-hard-drive-failure...

codezero 6y ago
I anticipate these reports every year and have strong trust in the data - I want to make that clear - Backblaze has done a massive service to the entire industry by collecting and aggregating this kind of data. I'm really super curious about the dip in errors over the past year :)
- rootusrootus 6y ago
  Whether intentional or not, it's also great word-of-mouth advertising. My preexisting experience with Backblaze's hard drive stats reporting definitely worked positively in their favor when I was looking for a new backup service.
  - brianwski 6y ago
    > Whether intentional or not, it's also great word-of-mouth advertising. Oh, this has really worked out for Backblaze and we know it. The first time we published our drive failure rates (I think January of 2014?) a few people said, "Uh oh, now Backblaze will get sued by the drive manufacturers." And we cringed and waited. :-) But the lawsuit never came, in fact there were NO repercussions, only increased visibility. People who have never heard of our company before find the data interesting, and then they ask "hey, what does this company do to own this many drives?" And a few (like yourself) sign up for the service. Existing customers seem to stick with us for a long time, and even recommend us to other friends and family from time to time. So one tech person who stumbles across these stats might ACTUALLY bring us 3 or 4 more customers over the next 5 years. That's real money to us. Not only have the drive manufacturers not sued us, they are actually NICE to us beyond the scale of our actual drive purchases! In one amusing example, our drive stats were used in a lawsuit as evidence. To be clear Backblaze was not the plaintiff or the defendant in the court case, we had no skin in the game at all and didn't want to be involved, but our drive data (and internal emails) were subpoenaed to be entered into evidence. Before we were served, the drive manufacturer called us and apologized for the inconvenience and made it clear they had no beef with us. Yes, a multi-national company that makes BILLIONS of dollars per year called a 40 person company (at the time) that could barely make payroll each month to apologize for the inconvenience. :-) We thought it was very considerate of them, and a little amusing. I'm proud to be the one that "signed" the papers indicating Backblaze had been "served".
    - rootusrootus 6y ago
      Clicked on your profile and noticed that you are a fellow Beaver. Go Beavs! ;-). I got my CS degree from OSU about 10 years after you graduated.
      - brianwski 6y ago
        > noticed that you are a fellow Beaver. Go Beavs! ;-) Ha! This is a silly novelty "Beaver Baseball Cap" I was given during my internship in 1988 at Hewlett-Packard in Corvallis: https://i.imgur.com/G0rPHGP.jpg https://i.imgur.com/G0rPHGP.jpg For 32 years it has sat on my computer monitor or on a shelf nearby. Nobody really asks about it anymore. I got pretty lucky on timing at OSU. The year before I was there they taught beginning programming on punch cards on a mainframe computer called a "CDC Cyber". In my freshman year I took Pascal on the brand new 1984 Macintoshes in the Computer Science department, while the engineering students still learned Fortran on the mainframe my freshman year. In 1992 I got a job as a Software Engineer at Apple in Cupertino, and you can trace that straight back to my blind luck of starting my programming education at OSU on the Mac in the exact correct year. Well, I'd rather be lucky than good. And I've ALWAYS been lucky in my career.
matmatmatmat 6y ago
Hi Brian, just a note of thanks to you and Backblaze for publishing these data. I always refer to them before a purchase and they're really helpful.
willis936 6y ago
There are other interesting factors to look for as well. Temperature, moisture, electrical noise on the power rails, infrasound, etc.
rachelwenzel 6y ago
Wow- crazy to see people at Backblaze actually read that article!! Thank you guys for putting out so much data and writing so much about your findings- it was HUGE in helping me come to conclusions about what's realistic to assume from SMART stats. Y'all are doing some really really cool stuff.
- atYevP 6y ago
  Yev here -> of course we read it! It's a data-rich article about data failure! And the kudos is mutual - loved the post! :D