28 ms·

It's not just "consumer drive" versus "enterprise drive". It's also "consumer drive" versus "hyper-scale cloud storage drive". Google used to use consumer dr

by tytso 9y ago

It's not just "consumer drive" versus "enterprise drive". It's also "consumer drive" versus "hyper-scale cloud storage drive". Google used to use consumer drives (and the latter category didn't used to exist) but the world has moved on, and if you are looking for the lowest storage TCO (again, please see Google's "Disks for Data Centers" paper), I'm going to suggest it's a lot more than just using 4T drives. Hint: you're assuming the only cost is just the raw spindle. You also need to take into account the cost of power, cooling, some portion of the server that the disk is attached to, the cost of the tray, etc., etc., etc. This is all in the "Disks for Data Center" paper, so I'm not giving away any corporate secrets. Disclaimer: I work at Google and I am a co-author (but only in a very small way) of the "Disks for Data Centers" paper. We have also been at this for a while...

brianwski 9y ago
> Hint: you're assuming the only cost is just the raw spindle. Not me, not sure who you are responding to? I was just correcting the record that Backblaze ONLY used "consumer" drives when some of our drive serial numbers are "enterprise" drives, I wasn't commenting on total cost of ownership. Backblaze has our own spreadsheet of total cost of ownership, and we are mostly deploying 8 TByte drives right now because the increased density means renting fewer cabinets and paying less for electricity. I'll just make the point that the spreadsheet takes the inputs of WHICH DATACENTER because different locations pay different amounts for electricity. > I am a co-author of the "Disks for Data Centers" paper. I read it when it came out, it is filled with good ideas, concepts, possibilities, and suggestions. Very worth reading. However, it is all about spinning drives, which may not be the lowest cost and highest performance in "X" number of years. There is a lot of disagreement around this point (even within Backblaze) but I think "X" is "really close" like within two years. Even if the individual SSD costs more per GByte, like you brought up, the savings in electricity (our number one datacenter cost) are really good which might make up for it soon (again, depending on the region they are deployed in). But for at least the next two years (and possibly five?), I think we all know we're deploying spinning drives. :-)
- tytso 9y ago
  Well, the Backblaze blog post only talks about the raw spindle costs, and strongly implies that 4TB drives were the lowest cost option and ignores the existence of any disks larger than 8TB. Which is true only if you are talking about consumer drives and only concerned about raw spindle costs. Otherwise, it's a highly misleading post. The electricity cost and the fractional share of the disk tray and server to attach the HDD are mostly constant no matter whether you use a 4TB, 8TB, 10TB, or 14TB disk. So if you are only worried about renting fewer cabinets and paying less for electricity, then if 8TB drives make sense, 14TB drives would make even more sense, no? Of course, you have to worry about more than just bytes; you also have to worry about IOPS. So it starts being about whether you can use nearly all of the IOPS in every drive. (This is why having a "hot pool" and a "cold pool" of disks really don't make that much sense. You'll leaving money on the table by wasting the potential usable IOPS in your cold pool. Instead you can get a much more TCO positive solution by mixing your hot and cold data so you case use most of the IOPS and most of the bytes for all of your drives.) But if you think SSD's are so "cheap", why not use 14TB drives plus SSD's? Perhaps because using SSD's for storage (as opposed to cache) really isn't that cheap as people think. Sure, lots of IOPS, but unless the bytes you need to read are the right ones to be stored in the SSD, the cost per byte for SSD's is quite astronomical, even taking electricity, server attachment costs, etc., into account. As a result, I believe using large numbers of spinning drives will make sense for a lot longer than you think.

by tytso 9y ago

brianwski 9y ago

> Hint: you're assuming the only cost is just the raw spindle. Not me, not sure who you are responding to? I was just correcting the record that Backblaze ONLY used "consumer" drives when some of our drive serial numbers are "enterprise" drives, I wasn't commenting on total cost of ownership. Backblaze has our own spreadsheet of total cost of ownership, and we are mostly deploying 8 TByte drives right now because the increased density means renting fewer cabinets and paying less for electricity. I'll just make the point that the spreadsheet takes the inputs of WHICH DATACENTER because different locations pay different amounts for electricity. > I am a co-author of the "Disks for Data Centers" paper. I read it when it came out, it is filled with good ideas, concepts, possibilities, and suggestions. Very worth reading. However, it is all about spinning drives, which may not be the lowest cost and highest performance in "X" number of years. There is a lot of disagreement around this point (even within Backblaze) but I think "X" is "really close" like within two years. Even if the individual SSD costs more per GByte, like you brought up, the savings in electricity (our number one datacenter cost) are really good which might make up for it soon (again, depending on the region they are deployed in). But for at least the next two years (and possibly five?), I think we all know we're deploying spinning drives. :-)

tytso 9y ago
Well, the Backblaze blog post only talks about the raw spindle costs, and strongly implies that 4TB drives were the lowest cost option and ignores the existence of any disks larger than 8TB. Which is true only if you are talking about consumer drives and only concerned about raw spindle costs. Otherwise, it's a highly misleading post. The electricity cost and the fractional share of the disk tray and server to attach the HDD are mostly constant no matter whether you use a 4TB, 8TB, 10TB, or 14TB disk. So if you are only worried about renting fewer cabinets and paying less for electricity, then if 8TB drives make sense, 14TB drives would make even more sense, no? Of course, you have to worry about more than just bytes; you also have to worry about IOPS. So it starts being about whether you can use nearly all of the IOPS in every drive. (This is why having a "hot pool" and a "cold pool" of disks really don't make that much sense. You'll leaving money on the table by wasting the potential usable IOPS in your cold pool. Instead you can get a much more TCO positive solution by mixing your hot and cold data so you case use most of the IOPS and most of the bytes for all of your drives.) But if you think SSD's are so "cheap", why not use 14TB drives plus SSD's? Perhaps because using SSD's for storage (as opposed to cache) really isn't that cheap as people think. Sure, lots of IOPS, but unless the bytes you need to read are the right ones to be stored in the SSD, the cost per byte for SSD's is quite astronomical, even taking electricity, server attachment costs, etc., into account. As a result, I believe using large numbers of spinning drives will make sense for a lot longer than you think.