24 ms·
Hard Drive Stats for 2019
- metalliqaz 7y agoI use Backblaze's massive infrastructure to store pictures of my keyboard.
- rosstex 7y agoSemi-bummed my school partnered with another backup company, cause I'd love to support BackBlaze.
- dleslie 7y agoSigned up for this a week ago. 45 days remaining to upload. Hurray for Canadian internet.
- UI_at_80x24 7y agoThis may not apply to you, but atleast 2 of the UnderDogs in the Canadian ISP world (MNSi, & TekSavvy) have been rolling out Gigabit fiber. I've got a 1Gb fiber pipe for 1/10th the cost that Cogeco was charging.
- mrguyorama 7y agoI have ~10mbps upload here in the US, and my backup was looking to take about a month for about 3ish TB of data. One thing that helped is that with default settings of only 1 backup thread, the Windows client was unable to saturate my upload bandwidth. Upping it to 4-6 threads allowed it to keep enough data moving to actually saturate my upload bandwidth and brought my backup down to like a week.
- mherrmann 7y agoDoes anyone here have experiences with BackBlaze's B2 service for hosting files? I'm considering switching to it from S3 because it is much cheaper. (I need to transfer 2-3TB / month, usually in 2-3 bursts of worldwide distribution).
- the_svd_doctor 7y agoI use it for personal backups with rclone. Works great.
- atYevP 7y agoYev from Backblaze here -> We're definitely more affordable and our integrations (https://www.backblaze.com/b2/integrations.html https://www.backblaze.com/b2/integrations.html) make it easy to get your data to us. We even have partnerships with companies who can help transfer data from S3 into Backblaze B2!
- mherrmann 7y agoThank you Yev. I'm wondering about the bandwidth, especially internationally. Do you have any numbers on that? Say split by Europe/US/Other.
- mherrmann 7y ago(I need to quickly ship a 50mb file to 50,000 clients worldwide.)
- jermaustin1 7y agoHey Michael, I host RAW photos I want to share inside B2 (48mb each), and then put CloudFlare in front of it using their tutorial [1]. It gets edge caching, and achieves 200-500mbps. Its great, and I have absolutely no complaints. 1: https://help.backblaze.com/hc/en-us/articles/217666928-Using-Backblaze-B2-with-the-Cloudflare-CDN https://help.backblaze.com/hc/en-us/articles/217666928-Using...
- ronnier 7y agoI have two ST12000VN0007 (VN) Seagate drives. The report shows the ST12000NM0007 (NM) has a 3.32% failure rate. I wonder how closely related the VN and NM models are.
- war1025 7y agoIf you look, that drive model is also the most highly used by far. I think it's just a matter of the larger sample size / use time.
- JohnJamesRambo 7y agoSurely it doesn’t matter when you have 10,000s of drives? Aren’t you already at a large enough sample size? If it isn’t, what is the point of them publishing this every year? I don’t know the math of the matter though.
- war1025 7y agoYea I don't know. I'm not big on statistics either. I just noticed that the drives that did the worst were the ones that had the most usage overall.
- labawi 7y agoThat would probably be price ~ failure rate correlation.
- generalpass 7y ago> Surely it doesn’t matter when you have 10,000s of drives? Aren’t you already at a large enough sample size? If it isn’t, what is the point of them publishing this every year? I don’t know the math of the matter though. I think drive age matters? I'm not clear if they cycle drives out at a certain age or just run them until they fail. Also, if a drive is low enough in cost, then the additional cost of replacing an incremental 1% may be lower than the cost of acquisition of a more reliable drive.
- 7y ago
- UI_at_80x24 7y agoI live for these reports. Always insightful and professional. Thank-you SO MUCH for publishing this data.
- atYevP 7y agoYev here -> You're welcome! The conversation's always fun :D
- donmcronald 7y agoI barely had time to skim it, but I'm not sure I like how the ST12000NM0008 shows up in the table. I find it really hard to reason about what the real failure rate could end up being on those drives. For example, you've got about 45 days average on each drive, so the failure rate is multiplied by roughly 8 to extrapolate the annualized failure rate. Doesn't that over state the estimated rate of failure since drives will tend to fail more often at the start of their life? I only guesstimated out of the table and didn't have time to look at the actual data, so it's possible I misread something.
- alberth 7y agoWhy do people use Amazon S3 when Backblaze B2 is 1/4 the cost of S3 and also includes a CDN for free. You also get way faster access speeds with Backblaze vs Amazon since they tier their IO speeds. https://www.backblaze.com/b2/cloud-storage.html https://www.backblaze.com/b2/cloud-storage.html
- smeyer 7y agoI think a big reason is that people are using the rest of the amazon ecosystem. If your costs aren't primarily storage, you might be willing to pay a premium to use something that integrates nicely with other services you're using. Here's an article[0] that does some other comparisons between providers and mentions things like upload speed and security features. [0] https://www.cloudwards.net/azure-vs-amazon-s3-vs-google-vs-backblaze-b2/ https://www.cloudwards.net/azure-vs-amazon-s3-vs-google-vs-b...
- 3fe9a03ccd14ca5 7y agoIAM access control, ease of use, reliability, speed.
- mythz 7y agoWhere's the info about the free CDN?
- nielsole 7y agoCheck Cloudflare bandwidth alliance
- mythz 7y agoThis bandwidth alliance? https://www.cloudflare.com/bandwidth-alliance/ https://www.cloudflare.com/bandwidth-alliance/ Cloudfare is a featured integration that only mentions that transfer fees are free not that CDN hosting is free: https://www.backblaze.com/b2/solutions/content-delivery.html https://www.backblaze.com/b2/solutions/content-delivery.html Cloudfare does have a free CDN tier "For individuals with a personal website and anyone who wants to explore Cloudflare." but it's not the same as B2 including a CDN for free, even Azure is apart of the bandwidth alliance.
- generalpass 7y agoI have wondered about system downtime or time operating in a degraded state. My understanding is other than mirrored, RAID configurations may take a long time to rebuild on the larger drives and this is a contributing factor to why the highest sales volume of drives has been 'stuck' at 4TB (thus the lower $/GB price).
- linsomniac 7y agoThey don't use traditional RAID setups there. My understanding is they use a proprietary data encoding and distribution, which is more accepting of individual drive failures and reduces rebuild times. I believe I've heard they use something more like erasure coding rather than RAID-5.
- ddorian43 7y agohttps://www.backblaze.com/blog/reed-solomon/ https://www.backblaze.com/blog/reed-solomon/ There are many open source libraries.
- throwaway17_17 7y agoDoes anyone have any opinions and experience using backblaze as a personal only cloud storage and offsite backup for smaller amounts of data (under 30 TB)
- deleted 7y ago[deleted]
- clSTophEjUdRanu 7y ago+1 I have the same question and would like to read replies.
- dwohnitmok 7y agoI use Backblaze's B2 service for both backup (via restic) and archival storage (via git-annex). I only maintain a distinction between the two in case I ever want to move to another service, and also because git-annex and restic have different strengths that make them more or less suitable for unchanging archives and often changing backups respectively. Between the two I have about 1 TB stored with them. I have yet needed to do a full restore, but I do partial restores from time to time to double-check my backup procedures and every time it's done what I wanted. My monthly costs are usually a bit under $5. Note I essentially never use B2's API directly, and only use it as a backend through wrappers others have written, so I have no real experience with how good its API is. One of the few times I did try the API, I remember at one point I think I was getting Java exceptions back in the error messages, which was mildly concerning from a hygiene perspective and made for rather terrible error messages, but no sensitive data was being emitted. I also think that's been fixed. The bottom line is that B2 has worked fine for me and at a good price point.
- newscracker 7y ago> for smaller amounts of data (under 30 TB) Did you mean to say 30 GB or 30 TB? Calling 30 TB as "smaller amount" seems weird to me in 2020, especially for personal data. Perhaps it would be the norm in a couple of decades. :) FWIW, I have way under 1 TB of personal data to backup to different locations, and I consider that to be relatively large.
- sillyquiet 7y agoNo really related remarks about this handy study, but anybody else still in real awe about how spoiled we are with regards to the sizes and speeds of HDs nowadays? I mean the smallest capacity drive on their chart is 4 Terabytes.
- newscracker 7y agoSince HDDs have, for the most part, been relegated to being external drives on laptops, I'm still looking forward to SSDs becoming way cheaper and reaching current HDD prices per GB. Internal storage on laptops has reduced or stayed the same while our datasets have grown exponentially over the years (with photos and videos). Since SSDs also perform much better when there's always a good amount of free space (for wear leveling and maintenance), it's all the more painful to live with lower capacity SSDs on laptops.
- klodolph 7y agoSizes, yes, speeds, no. 600 MB/s of data transferred, and only for linear accesses.
- zaat 7y agoNot feeling spoiled at all, not at all. Especially not with 2 to 3 percent of failure rate. The failure rate I experienced in my workstation makes me worry about not having raid 1 or 10. HDs for 9 TB in raid 10 are not that cheap. But the bigger issue is that the warranty terms for HDs nowadays is down to 2 or 3 years, so this investment is short living. It also tell you something about the manufacturers reliability estimation of their products.
- AnIdiotOnTheNet 7y agoCan't say I agree with that sentiment. The fact that I can quite reasonably have a 30TB usable RAID5 NAS array makes me feel pretty spoiled. Then again, I'm old enough that my first HDD was 10MB.
- chousuke 7y ago
- Siecje 7y agoI have about 10 TB of video files. I use BackBlaze for Windows but I would like the files to be available on other computers and my phone in my local network. What can I use to do this and still keep offsite backups?
- mrguyorama 7y agoI think their more premium plans offer sharing
- sandoooo 7y agoDropbox?
- deleted 7y ago[deleted]
- robertoandred 7y agoI love Backblaze, but their log package in my Library folder has grown to something like 10 gigs. Wish there was a way around that.
- newscracker 7y agoSlightly off topic: is anyone using B2 (which seems cheaper if you have more than one computer for a certain amount of data) for personal data backups with strong client side encryption across multiple platforms (Linux, Mac, Windows)? If so, how do you handle it?
- w33ble 7y agoI do this, though not from Windows, just Mac and Linux. I use restic, which has B2 support smd handles all the encryption. it also does diffing for backups. There's a Windows build, so I assume it would work for you there as well. You can view and download builds at https://github.com/restic/restic/releases/ https://github.com/restic/restic/releases/ I don't automate this though, I just use it for occasional backups. Not sure what the automation story around restic is.
- hashhar 7y agoYes. I use restic same as the sibling comment. Have >8TB of data from multiple machines with a lot of deduplication (source is somewhere around 10 to 12TB).
- S3raph 7y agoI sync all my device files to a local Freenas server which runs duplicacy in a jail and sync's it every night to backblaze B2. I looked at duplicity, restic, attic, borg and in the end settled for duplicacy. Pay attention to the duplicacy license, for somebody it could be a problem.
- orhanhh 7y agoI use Arq on two macs and it works very well with B2.
- zippergz 7y agoI use and like Arq also, but the OP asked for something that covers Linux, which I believe Arq does not.
- seanlane 7y ago
- ksec 7y agoLooking at those Data, It seems they will soon reach 1000 PB / 1EB. The top 5 Annualised hard drive failure rate are all from Seagate. All Drive from Hitachi and Toshiba has AFR lowered than 1%. So basically dont buy Seagate.
- arminiusreturns 7y agoIt's pretty much been this way for a few years, with only a few model lines of Seagate being the outlier. As always, thanks to the BackBlaze team for publishing these numbers.
- LeifCarrotson 7y agoMy math says they're already over 1000 PB/1 EB: 1,089,318 = 4 * 2852 + 4 * 12746 + 8 * 1000 + 12 * 1560 + 12 * 10859 + 4 * 19211 + 6 * 886 + 8 * 9809 + 8 * 14447 + 10 * 1200 + 12 * 37004 + 12 * 7215 + 4 * 99 + 14 * 3619 Don't think I made a typo there, but please check my work. Even counting as 1024 TB = 1 PB and 1024 PB = 1 EB, that leaves 1,048,576 TB = 1 EB and they're over that threshold. The February 5, 2018 "500 Petabytes and Counting" blog post should soon be eclipsed by a 1 EB post - though it appears they're counting actual data stored, not capacity. Nonetheless, with some redundancy, extra capacity, and overhead, we'll likely see that number soon.
- AdamGibbins 7y ago> So basically dont buy Seagate. Or do, because they're cheaper than the competition and modern systems can handle failures.
- antonyh 7y agoTrue with enough redundancy it's fine, and if they have special terms with SG such as free replacements and heavy discounts then it's little wonder they use so many. Myself though, for SoHo use, I'm willing to pay more for less stress because I don't have the sheer volume of devices, and the time to replace is time spent doing something useful instead of shuffling HDDs and rebuilding RAID arrays. A 5% saving on a handful of drives is not worth it, but a 40% saving on thousands makes them competitive.
- LeifCarrotson 7y agoInteresting how the numbers carry over year-to-year in https://www.backblaze.com/blog/wp-content/uploads/2020/02/Blog_3_year_Drive_Stats_Chart.png https://www.backblaze.com/blog/wp-content/uploads/2020/02/Bl... Some models are dwindling. Some are being tested. Others (like the Seagate and HGST 12 TB) are increasing. Only thing that's really perplexing is why they keep buying more and more of the high-failure-rate Seagate 12 TB drives. It must be more than 3% cheaper to buy (and service!) a Seagate with a 3% chance of failure than to buy an equivalent HGST with a 0.4% chance of failure. I guess when you have 120,000 drives, easy hot-swap enclosures, and software to handle it all that makes good sense! But as an individual consumer, even with a Backblaze backup, it's definitely worth my time to spend a bit more on a drive that's far more reliable than to save a few dollars on a Seagate.
- alanfranz 7y agoAt least in Europe, HGST is much more expensive than Seagate. Almost double the price, usually.
- leokennis 7y agoVery anecdotal evidence, but 3 of the 3 Seagate drives I ever used (all external 2,5” USB 3 HDD’s, in Seagate’s own enclosures) failed within 2 years, under very modest workloads (just used to store video files for my tv to play). Meanwhile all WD’s have been rock solid.
- magnat 7y agoDoes anyone remember what is their definition of "drive failure"? Is it SMART "failure imminent" report, single uncorrectable read error or complete data loss for a whole disk? I recall reading about it in one of their previous report, but can't find it again. EDIT: nevermind, found it. "Backblaze counts a drive as failed when it is removed from a Storage Pod and replaced because it has 1) totally stopped working, or 2) because it has shown evidence of failing soon. A drive is considered to have stopped working when the drive appears physically dead (e.g. won’t power up), doesn’t respond to console commands or the RAID system tells us that the drive can’t be read or written." https://www.backblaze.com/blog/hard-drive-smart-stats/ https://www.backblaze.com/blog/hard-drive-smart-stats/
- jmnicolas 7y agoAny particular reason they don't use Western Digital drives ?
- volkl48 7y agoI will point out that HGST is owned by Western Digital and all their products are being rebranded to WD.
- forgot-my-pw 7y agoOther than HGST is owned by Western Digital, they also said: There were no Western Digital branded drives in the data center in 2019, but as WDC rebrands the newer large-capacity HGST drives, we’ll adjust our numbers accordingly.
- gesman 7y agoSo what does these mean: smart_177_raw smart_177_normalized smart_233_raw smart_235_normalized ???
- thenewwazoo 7y agoThey're S.M.A.R.T. attributes: http://www.cropel.com/library/smart-attribute-list.aspx http://www.cropel.com/library/smart-attribute-list.aspx
- gesman 7y agoThank you!
- S3raph 7y agoI'm a very happy customer, but please do something about your mobile app (android) it's really horrible.
- leokennis 7y agoI agree the mobile app (on iOS in my case) is at best an afterthought, and most likely not even a high ranking afterthought. However, out of curiosity...what would you imagine a better Backblaze mobile app would do?
- S3raph 7y agoFor sure it is/should not be high priority, but releasing such an app in 2020 for sure does not reflect the great skills of the backblaze team. At least show me some basic stats, account settings and invoices. You can only download files from your buckets and that's it.. really?
- exabrial 7y agoI still can't believe BackBlaze gives this data away for free. Seems like something they should be selling to other cloud providers
- Mirioron 7y agoMaybe they consider this report to be an ad for their services? The name recognition this report gives them is probably quite valuable.
- icelancer 7y agoIt also pressures HDD companies to make better products and appear higher on these lists, which is good for Backblaze.
- kevinchen 7y agoI’m sure Amazon, Google, Facebook, etc collect their own stats on drive failure. It would be almost negligent if they were just guessing in the dark every time they buy drives. Main difference is probably Backblaze is small enough to publish these stats without hurting their supplier relationships. (pure speculation)
- nolok 7y agoTo quote them : Transparency breeds trust. We’re in the business of asking customers to trust us with their data. It seems reasonable to demonstrate why we’re worthy of your trust.
- donatj 7y agoI have made all my hard drive purchasing decisions based almost entirely on these reports for the last couple years and have not been disappointed with the results.
- voldacar 7y agoSeagate always seems to have much higher faliure rates compared to HGST/WD/Toshiba etc Does anyone here know the exact reason why? I assume there are enough people on this site who have worked for them or a competitor :)
- rasz 7y ago30% profit margin, why change anything?
- anonsivalley652 7y agoAh yes, the reliable BackBlaze folks. That they've out-Googled Google in a niche using mostly commodity infrastructure and kept their business alive for so long is a testament to their ingenuity (I wonder how their operating costs compare with AWS Glacier which has a theoretical advantage of unpowered disks.). And the releasing of this proprietary operational business data is a testament to their coolness factor. It's a timely article as I'm looking at HC530's (WUH721414ALE6L4 / WUH721414ALN6L4 (wiredzone carries it)) for a home FreeNAS box: - any relatively-modern enterprise 4U 3.5" storage box with Xeon 4 cores or so - quieter, high-volume fan mod - RAM: 64-128 GiB, beyond that isn't useful unless deduping - NIC: X710-T4L 4x 10GbE copper NIC - ZIL: mirrored pair of high-endurance, write-intensive, reliable SSD like Optane 900p/905p 280-480GB - L2ARC: striped pair of read-intensive/larger SSDs like the Gigabyte Aorus Gen4 1 TB This will fit nicely as my home NAS for a water-cooled dual EPYC virtualized server/workstation build underway. I managed to get a single water block with (3) G1/4 connections that will cool both CPUs and the VRM chokes/converters. If anyone has better suggestions, please chime in.
- tbrock 7y agoWhy does someone need something like this? I ran a home symbology NAS at one point but it wasn’t worth the trouble. Let others run and maintain those hot, loud, power hungry disks.
- anonsivalley652 7y agoThen you already made the mistakes of: - conflating trouble for you with trouble for me, which it clearly isn't - not owning your own data - paying more to store it - paying to access it - ability to keep things that aren't worth storing on paid clouds but aren't all that much when kept on cheap drives Furthermore, there are additional network costs such as AWS network charges AND home ISP data limits. And there are other uses, such as: - backing-up VMs - backing-up computers - caching package and source code repos - backing-up CCTV footage - and whatever else comes along
- zaarn 7y agoSpeed. Streaming files (movies, tv shows, youtube channels, linux ISO's, large file collections) is a lot faster; I can reliably hit 100MB/s on my home connection, my DSL caps out at 20MB/s if I do nothing else.