12 ms·
Hard Drive Cost per Gigabyte
- gwern 9y agoThere's a clear regime shift in the slope pre/post Thai floods. It's not a surprise that the floods set back Kryder's law by years, but does anyone have any insight in why it would affect the rate of progress and switch hard drives to a much slower rate of improvement? Surely the floods wouldn't still affect investment or R&D? Or is the timing merely a coincidence and reflect consumer demand being saturated, switches to SSD etc?
- briffle 9y agoThere was also some consolidation at that same time. For example, Western Digital purchased Hitachi in 2012. At around the same time, Fujitsu, Lacie, and Samsung also were acquired. http://www.itcandor.com/hard-disk-suppliers/ http://www.itcandor.com/hard-disk-suppliers/
- rb808 9y agoYeah I had to buy a new HD recently and was a little shocked that there are only really 2 manufacturers + Toshiba. No wonder prices aren't great.
- jaclaz 9y agoYep, the Wikipedia page: https://en.wikipedia.org/wiki/List_of_defunct_hard_disk_manufacturers https://en.wikipedia.org/wiki/List_of_defunct_hard_disk_manu... has an excellent "consolidation diagram": https://en.wikipedia.org/wiki/List_of_defunct_hard_disk_manufacturers#/media/File:Diagram_of_Hard_Disk_Drive_Manufacturer_Consolidation.svg https://en.wikipedia.org/wiki/List_of_defunct_hard_disk_manu...
- fnj 9y agoWhy do you phrase it "2 manufacturers + Toshiba"? Why not just "3 manufacturers"?
- rb808 9y agoAgreed its a little weird, but if you look at best sellers the first Toshiba is on the 3rd page. https://www.amazon.com/Best-Sellers-Electronics-Internal-Hard-Drives/zgbs/electronics/1254762011 https://www.amazon.com/Best-Sellers-Electronics-Internal-Har...
- antisthenes 9y agoAFAIK Toshiba is the smallest one of the 3, it is not their primary business, and there's a high likelihood of their HDD business being sold off to one of the 2 big ones in the near future.
- chx 9y agoLacie, as far as I am aware, never manufactured their own hard drives. Look here for example in 2006 http://forums.macresource.com/read.php?1,85231,85262 http://forums.macresource.com/read.php?1,85231,85262 > Thank you for your email. We use a few different manufacturers, so we would not be able to tell you which drives go to which item (except the 500GB drives which are Hitachi). In general we do use Western Digital, Seagate, and we did use Maxtor before they were bought by Western Digital. For DVDs, we use Samsung. Wikipedia says , much earlier: > As a subsidiary of Quantum, La Cie was licensed as the exclusive manufacturer of Apple-branded external SCSI hard drives, using Quantum hard disks.
- Theodores 9y agoThis confirms what I remember. A 3D modeller I worked with was deeply upset when Maxtor took over Quantum as the 2Gb SCSI disks that were carried around at the time were LaCie, apparently with Quantum disks which were allegedly the best for replaying video frames. At the time I did not know LaCie was owned by Quantum. Back then there wasn't the internet to quickly Google brands so you only had marketing information to go on.
- chx 9y agoo_O Maxtor purchased Quantum in October 2000. This is the same month when Adwords left beta. What the heck are you talk of? By 2000 home broadband is beginnning to spread, dial up Internet is everyday in most of the so called developed world.
- giobox 9y agoI'd always assumed La Cie's storage business model the past however many decades was to take stock drives from other manufacturers and put them in nicely designed enclosures. The premium on their stuff has always seemed to correlate to the nicer materials and design on the cases, not the drives themselves. I guess occasionally they also seem to tap the early adopter market when a new storage interconnect comes out (FireWire 800, Thunderbolt etc). They often seem to be one of the first out of the gate with support when Apple add a new port.
- codemac 9y agoA LOT happened in the ~2011 -> ~2013 timeframe, including the largest consolidation in hard drive manufacturing history (outside of maybe ~1992). Toshiba acquired Fujitsu's HDD business in 2010. Seagate acquired Samsung's HDD business in 2011 (remember they acquired Maxtor in 2006 as well). Western Digital acquired the HGST group in 2012 (makers of the best damned 2TiB drive I've ever seen in my life) You're left with 3 hard drive manufacturers, of which two are major SSD vendors for enterprise. This reduction to 3 vendors has a lot more to do with pricing than almost anything related to consumers's HDD purchases (who's growth since 2010 is something like 8%, it'd be better to invest in S&P 500 then run an entire consumer hard drive business)
- pmoriarty 9y agoHow many people were buying Fujitsu, Samsung and HGST drives before the acquisitions? My impression was that most people were buying Seagate and WD drives for a long time, and the rest were bit players. So I'm not sure how much the aquisition actually changed in the market. It effectively wasn't very competitive to begin with. Much like the CPU market, where Intel and AMD are the only two big players. Or the desktop operating system market, divided mainly between Microsoft and Apple.
- jrnichols 9y ago> How many people were buying Fujitsu, Samsung and HGST drives before the acquisitions? People? Not sure. But I was seeing a ton of Fujitsu and HGST drives in Dell and Apple stuff.
- codemac 9y ago> How many people were buying Fujitsu, Samsung and HGST drives before the acquisitions? Many many many. But the purchasers weren't consumers or even businesses, but other storage vendors higher up the value chain (see: emc, netapp, etc).
- Analemma_ 9y agoI think it's a "perfect storm" of all of the above. All around the same time, there were the floods, consolidation in the market reducing competition, and the large consumer shift to SSDs (and probably consumers "topping out" and deciding 4 TB drives are enough). IMO the last effect is the largest. In any technology, the continuous cost decreases we're used to rely critically on a constant stream of consumer dollars, especially as the low-hanging fruit is plucked and you have to work harder to get further improvements. HDD companies know the writing is on the wall and that huge investments in R&D are going to take longer to pay back as the consumer market dries up.
- frik 9y agoIs it just the flood? Or is it cloud providers and big data corps were buying all HDDs they could get - and the consumer market got the remaining rest
- magila 9y agoWhat happened at about the same time as the Thai floods is that areal density hit a brick wall. The timing is just a coincidence. The reason that 4TB remains the cheapest capacity is that it turns out 4TB is the most you can store using a conventional four platter drive, which is the most efficient configuration to manufacture. AFAIK all 6TB drives have five platters which of course costs more to make. 8TB drives are either six platters in a helium filled case or five platters with shingled recording, the former being expensive and the latter having significant performance implications. Fortunately for Backblaze, their use case is relatively well suited for singled drives which is what makes 8TB drives economical for them.
- deleted 9y ago[deleted]
- slyrus 9y agoAnd SSD costs still 10x per GB... Hoping that comes down.
- tombrossman 9y agoAgreed, I'm having to constantly manage space and move media files to external drives to keep using a SSD for my primary drive. I bought 30GB, 128GB, and 256GB drives as prices came down and I had hoped 1TB drives would be under $200 by now. Not even close. What is worse, I built a new desktop and planned to use a M.2 SSD for even higher speeds. That's a long way off now and will probably have to wait for the next build. Truly a first-world problem but damn, where are my cheap SSD's?
- tertius 9y agoThey're here. Samsung 960 EVO 500GB at $250. That's insane my friend. I guess it depends how long you've been in the game. I remember my 40MB drive being huge after we upgraded from a 12MB drive. It was truly hard to find enough to fill it up with.
- agumonkey 9y agoOnce tiny tiny sad thing, they shifted the entry models since. I used to find 40$ average but good brand (intel) ssd; now it's impossible.
- mark-r 9y agoThat's exactly the opposite of the way I expected things to turn out. Hard drives with their moving parts have a floor on the price, just because you can't get all those parts any cheaper; the only way to get the $/TB lower is to keep making them bigger. SSDs have no such problem, so I expected on the low end we'd see SSDs undercutting hard drives by a lot. Where are the $20 SSDs?
- tertius 9y ago
- xyzxyz998 9y agoI want to hear from people who are: 1) Coders, not designers 2) Use machine only for production, nothing "fun" Do you ever feel the need for going above 256 G? I have 3 operating systems installed- Win10, Win7, Arch, About 20GB of media, Office 2015, etc on my primary production machine but I'm still using less than 120 GB. I'm asking because I'm wondering if any developer has a reason to not go for sub-100$ SSD ~ 256 GB and instead opt for 4TB of HDD?
- pmoriarty 9y agoI've filled well over 500 gigs with VM images. Now, whether I actually need 500+ gigs of VM images is another question. I could definitely clean up and pare them down to just what I need, which would no doubt be much less than 500 gigs. But time is money, and it's just cheaper to buy more disk space than spend my time being an overpaid garbage collector.
- koyote 9y agoPictures take up the majority here. I do not keep them on my primary (SSD) harddrive though. Then there's also video, which is getting more and more popular (gopro, drones) with filesizes not really going down much due to things like FullHd and now 4k. Edit: I did not register your 'nothing fun' point properly. I guess you mean a pure working machine? Then no, I do not see a reason to have a hard drive above say 512gb. Do keep in mind that code-bases grow quickly though, I have about 25gb of source code on my main work machine.
- codemac 9y agoOnly because consumer filesystems suck. I'd love to have a completely versioned filesystem, at all times, regardless of how big it got.
- jl6 9y agoI am working on such a thing. Would you be interested in testing it?
- 9y ago
- Fej 9y agoMany people are saying here that the slowing of the rate of change is due to business consolidation... I would argue that it is probably due to physics instead. The traditional way of storing more data on a platter, shrinking the size of each bit, has just about run out. Each bit is now stored on a handful of atoms. The drive manufacturers are trying new things to get around this problem (e.g. shingling) but once each bit is down to a few atoms you can't really go any smaller. Further improvements in disk technology will probably have to be a bit more radical.
- StillBored 9y agoExcept oddly enough, you can frequently buy a drive wrapped in a USB enclosure for 10-15% less than the bare drive... Plus, the sleep modes on a lot of "consumer" drives significantly slow them down when used in a desktop environment (as well as creating second long pauses in RAID arrays with more than 5-6 drives), and the option to disable that "feature" is slowly disappearing on drives without the "enterprise" tax. Market segmentation shenanigans like this would never fly in a healthy competitive market are now very common.
- Fej 9y agoThat's true. Honestly I'm just hoping that HGST keeps their reputation as reliable HDDs even though they've been bought by WD. Backblaze always found that they fail less often.
- timerol 9y agoI would be interested in seeing the log of the prices on the vertical axis. The curve seems to fit the exponential decline in prices that I expect to see. I think the difference in regimes would be more clear cut that way. Just picking the end points and the middle, I see: (11 cents, 2009), (5 cents, 2013), (2.8 cents, 2017). So it went from slightly more than halving in cost in the first 4 years (55% reduction) to slightly less than halving in cost in the second 4 years (44% reduction). There's definitely a bit of a slowdown there, but not the huge difference that the article claims. Of course, the first half includes the Thailand flood, and the second half doesn't. Googling for some historical data leads me to http://www.mkomo.com/cost-per-gigabyte http://www.mkomo.com/cost-per-gigabyte, which claims that (from 1980-2010), the cost of a gigabyte has been halving in about 14 months, which is closer to the decay pre-Thailand flood.
- akrasuski1 9y agoThat is exactly what I thought - it seems to follow the exponential decay. A quick and dirty graph with logarithmic scale on price: https://imgur.com/a/90a4p https://imgur.com/a/90a4p Pretty much a straight line, except for the Thailand hiccup.
- synack 9y agoI wrote a thing to pull the prices of all the hard drives on Amazon every 12 hours and sort them by price per GB. https://diskprices.com/ https://diskprices.com/
- mark-r 9y agoI hope that's not the sort of thing that gets you banned from Amazon forever. That would be kind of harsh.
- synack 9y agoAs my understanding of the terms of service goes, it's allowed as long as I don't compare the prices to other sites.
- SquareWheel 9y agoIt's not an abuse of the API because prices are not being compared against products on other stores. However he is abusing their affiliate policy by not including an associates program disclaimer on the page.
- synack 9y agoThere's a disclaimer at the bottom.
- SquareWheel 9y agoIt's not disclosing that you make use of affiliates. Read section 5 of the associates program agreement. https://affiliate-program.amazon.com/help/operating/agreement https://affiliate-program.amazon.com/help/operating/agreemen...
- jdc0589 9y agolooks like the one thats the cheapest on your site @ $0.022 (Seagate Expansion 8TB Desktop External Hard Drive USB 3.0 (STEB8000100) ) is actually on sale and works out to $0.0186 right now. I may jump on that quickly....
- galantbutterfly 9y agoThe lowering costs of 8tb HDD's combined with it being Amazon Prime day, means you can get one of these bad boys for only $150. I just ordered 2. https://www.amazon.com/Seagate-Expansion-Desktop-External-STEB8000100/dp/B01HAPGEIE/ref=sr_1_3/130-8064997-0146330?s=electronics&ie=UTF8&qid=1499794547&sr=1-3&keywords=8tb+external+hard+drive https://www.amazon.com/Seagate-Expansion-Desktop-External-ST...
- elorant 9y agoJudging by the price points it seems that they're not buying enterprise level HDDs. There is no way you could buy an enterprise HDD today at $0.02 per GB. I bought a 4TB Seagate Constellation a couple of months ago and the price was more like $0.06 per GB.
- breakingcups 9y agoThey mention it in their article, as well as many of their other articles. They use "consumer" drives. For a more in-depth explanation, see https://www.backblaze.com/blog/enterprise-drive-reliability/ https://www.backblaze.com/blog/enterprise-drive-reliability/
- brianwski 9y agoCopy and pasted from my response in a different comment: > BackBlaze was typically only using consumer drives Disclaimer: I work at Backblaze. If anybody is curious, in our blog posts we list the exact serial numbers of all of the drives. See here for an example: https://www.backblaze.com/blog/wp-content/uploads/2017/05/blog-hd-stats-q1-2017-table.jpg https://www.backblaze.com/blog/wp-content/uploads/2017/05/bl... We have been at this for a while, and the distinction between "consumer drive" and "enterprise drive" is not what most people think it is. But in the end it simply doesn't matter -> the differences between failure rates of individual MODELS of drives in one line is larger than any difference in failure rates of "consumer" vs "enterprise". TL;DR - Backblaze uses both "consumer" and "enterprise" drives. But check the individual drive serial number, that is more important than arbitrary marketing groupings.
- tytso 9y agoThe key flaw in this analysis is that the author is only using consumer grade drives from Amazon. And yes, I think what is going on is that consumers don't really need much more than 4TB. Heck, most consumers have trouble filling 1 or 2TB. It's for the same reason that drive managed SMR drives haven't really been a big thing, in the consumer market. Who cares about 15-20% more bytes when you can't use all of the bytes in a 4TB CMR drive? Companies at the "hyper-scale cloud companies" (e.g., Facebook, Amazon, Microsoft, Google) can actually use the extra bytes, and they are probably the ones that will be able to use the 10TB, 12TB, 14TB, 16TB etc. drive that are on the vendors' roadmap[1]. [1] http://www.pcworld.com/article/3162084/storage/seagates-roadmap-includes-14tb-16tb-hard-drives-within-18-months.html http://www.pcworld.com/article/3162084/storage/seagates-road... It may be that Amazon might not be interested in stocking these drives because they won't sell enough of them to consumers. And if some of the discussions proposed in Google's "Disks for Data Centers"[2] paper come to fruition, such as adding a second disk actuator arm, or changing the height of the disk drives, there might be a further diversion between drives optimized for the consumer market and drives optimized for the "hyper-scale cloud market". (Note: the intent of Google publishing this paper is to hopefully allow discussions amongst the community of hyper-scale cloud storage customers to work with the HDD vendors for something that helps the whole high capacity storage industry.) [2] https://research.google.com/pubs/pub44830.html https://research.google.com/pubs/pub44830.html
- soared 9y agoI think that is the point, rather than the flaw, of this article.
- gwern 9y ago> The key flaw in this analysis is that the author is only using consumer grade drives from Amazon ? This is Backblaze, a commercial storage company. They buy in bulk in thousands of drives, not from Amazon, and the prices are what they paid. As they say, "The price you might get at Costco or BestBuy, or on Amazon will most likely be higher." It's the trend and comparisons which matter, not so much the absolute price.
- 9y ago
- SubiculumCode 9y agoI noticed and was irritated with the 4 Tb anomaly when I was setting up a RAID6 trying to achieve 30 Tb of usable space after mirroring and a hot spare.
- SubiculumCode 9y agoEnterprise class,btw
- deleted 9y ago[deleted]
- MR4D 9y agoWhen can we finally start measuring cost per TB ? The need to constantly multiply numbers by 1,000 is rather annoying. Yes, we can all do it, but still.
- deleted 9y ago[deleted]