6 ms·
Most HDDs are going to Server Market. And from that point of view the capacity increase has been very very slow. Edit: It seems lots of our computing component
by ksec 6y ago
Most HDDs are going to Server Market. And from that point of view the capacity increase has been very very slow.
Edit: It seems lots of our computing components has reached plateau or soon within this decade. From HDD, DRAM, NAND, Chip Processes, etc. That is not saying they wont improve, but their Unit Cost aren't dropping any more.
- rubiquity 6y agoAnd for the server market there's a sweet spot for each workload for how big you want your spinning disks to be. These disks aren't getting any faster and rebuilding a 22 TB HDD is going to take a while and that imposes serious durability risks.
- ody4242 6y agoIt's not just the rebuild,some storage software (CEPH for example) also validate the data on the disks from time to time, and since the iops is quite limited on spindles, it takes more and more of that iops bucket just to verify the data that you have on the disks.
- bayindirh 6y agoZFS also periodically re-silvers the disks to keep the FS in top shape. IIRC ZFS tries to re-silver disks when the traffic is low but, it's not always possible. I bet that the disks is 16+ TB range will be used for colder tiers of the storage. Also, they should be useful in the OST of the LustreFS since the random read storm hits the MDT more severely.
- karamanolev 6y agoA friendly minor correction - a resilver is only when a drive is replaced, the periodic checksum check is a scrub in ZFS parlance.
- magicalhippo 6y agoAnd the scrub has to be initiated somehow, typically cron job, it's not automatic in ZFS. Though NAS distributions like TrueNAS/FreeNAS sets this up for you by default.
- bayindirh 6y ago> And the scrub has to be initiated somehow, typically cron job, it's not automatic in ZFS. The devices were Oracle/Sun ZFS appliances (I tortured a 7320, we liked it & bought a full-out 7420) so, maybe it was set up to scrub/resilver automatically in some cases. As I've aforementioned, we're more of a Lustre shop and retired the systems some years ago.
- bayindirh 6y ago> A friendly minor correction... Thank you. I'm no expert in ZFS TBH (We use Lustre much more) but, IIRC, when I was benchmarking the then new Sun Oracle ZFS 7320, I remember it was resilvering the disks after especially torturous loads, at night. Maybe it was specific to the appliances (Our behemoth 7420 did the same) or, something was wrong. I remember Oracle/Sun guys jokingly asking me whether I succeeded to make it resilver the disks and, hearing it did indeed resilver the disks a dozen times visibly upset them. They've only said that "Pack it up, we need to go". Fun times, it was.
- Fnoord 6y agoHmm, if you had to resilver the disks a dozen times, I would assume that is indication of a hardware failure somewhere in the device (perhaps RAM?).
- rubiquity 6y agoYes that’s a good call out. I was being lazy but there are quite a few different reasons to need to scan the entire disk.
- formerly_proven 6y agoSince all heads are mounted to the same arm, only one head can lock onto a track at a given time. What if each head had independent micro-actuators so that all heads can be locked to their track of the same radius, with the data being distributed across all heads. Wouldn't this improve throughput n-fold? Edit: Seems like modern hard drives already have micro-actuators for each head to overcome precision and bandwidth issues with the main arm actuator, but it seems like none of those enable a range of motion that is sufficient to lock multiple heads simultaneously.
- droffel 6y agoIf you've got slow HDDs, the usual solution is to RAID them together. At that point your limiter starts becoming how fast you can slurp data down the line. RAID-0 would sort of emulate what you're talking about here.
- R0b0t1 6y agoThat still trades off density for speed, having independent arms would be a huge benefit.
- rbanffy 6y agoWould it still be cheaper than a similar SSD? Would it be cheaper than an SSD-augmented HD?
- droffel 6y ago> That still trades off density for speed It does not. RAID-0 has no duplication, it acts effectively like your independent arms. No density tradeoff, just speed.
- R0b0t1 6y agoThere is a tradeoff: With RAID0 your parallelization for read or write is on the scale of 24TB chunks, not within 24TB chunks.
- 6y ago
- chungy 6y agoI think a big problem too is that SSD prices haven't gone down fast enough. Basically the only reason that spinning rust is still a thing, is because SSDs are far more expensive.
- jeffbee 6y agoApples and oranges, right? You need ten thousand hard drives to match the IOPS of one SSD and even with the 10000 disks your service latency will still be three orders of magnitude worse. The other advantage of an SSD is in terms of bytes per volume, in case rack density matters to you.
- votepaunchy 6y agoYou need both SSD and HDD, the latter for archival storage and data replication. And that only works at data center scale.
- taneliv 6y agoPutting 24TB parts in a Backblaze Storage Pod 6.0 would presumably allow 1440TB in a 4U rack mount server.[1] In practice, how would you reach the same density with SSDs? (I haven't looked into it, just curious if you know that SSD options more dense than that exist, and whether they are equally openly documented.) [1] https://www.backblaze.com/blog/open-source-data-storage-server/ https://www.backblaze.com/blog/open-source-data-storage-serv...
- ddulaney 6y agoThere aren’t many full open-source solutions available, but you can buy 100TB 3.5” SSDs for data center use.[1] At Storage Pod densities, that’s 6000TB in 4U. I’m not sure if some other factor comes in that limits density, but that’s a first-order estimate. [1] https://nimbusdata.com/products/exadrive/pricing/ https://nimbusdata.com/products/exadrive/pricing/
- rbanffy 6y ago