7 ms·
14TB Hard Drives Now Available
- dmitrygr 9y agoDoes Linux kernel have the code to manage these SMR drives as they need to be? (Seems like HGST purposefully left that to the OS to do - imho a smart choice)
- koverstreet 9y agoIf someone was looking to jump into working on bcachefs, adding SMR support would be an excellent project.
- valarauca1 9y agoYes https://lwn.net/Articles/637035/ https://lwn.net/Articles/637035/
- loeg 9y agoNote that the conclusion of that article is that developers are thinking about it, not that it is solved in Linux. Linux may detect SMR drives and query them for zone information, but I don't believe ext4 or any other Linux filesystem works particularly well on them. I would love to be shown otherwise, though. > Reinecke wondered if the host-managed SMR drives would actually sell. Petersen piled on, noting that the flash-device makers had made lots of requests for extra code to support their devices, but that eventually all of those requests disappeared when those types of devices didn't sell. Reinecke's conclusion was that it may not make a lot sense to try to make an existing filesystem work for host-managed SMR drives. Edit: wmf points out that F2FS works reasonably well[0], until you have to do a garbage collection pass. [0]: http://events.linuxfoundation.org/sites/events/files/slides/lemoal-Linux-SMR-vault-2017.pdf http://events.linuxfoundation.org/sites/events/files/slides/...
- wmf 9y agoHere's the recent-ish status: http://events.linuxfoundation.org/sites/events/files/slides/lemoal-Linux-SMR-vault-2017.pdf http://events.linuxfoundation.org/sites/events/files/slides/... Looks like F2FS should work as of Linux 4.10.
- loeg 9y agoSee slide 21 — F2FS garbage collection after file/directory deletion is very expensive compared to classical filesystems on conventional disks.
- mjw1007 9y ago4.13 (released this September) added the dm-zoned device-mapper target, which presents an SMR drive as if it was a normal block device. I suppose what it does is similar to what drive-managed SMR drives do internally.
- white-flame 9y agoI bought an 8TB shingled drive. While the initial hundreds of MB of backup files wrote at reasonable speeds, it quickly dropped down to 4-5MB/sec sustained average for the rest, until it was idle for hours to catch up. Host management probably can help, but I'm not touching another SMR drive again.
- ioquatix 9y agoI guess if you use these for a log-style database, write only backup (e.g. a replacement for tapes with better random read), you'll be happy. But for a general purpose drive they suck.
- pavs 9y agoTrue. My rule of thumb, anything over 4TB, 7200rpm or not - I use only for long backup. Not on production app that needs good IO speed.
- white-flame 9y agoHigher capacity drives often have higher platter data rates, because of the increased density (or more platters & heads) at the same rotational speed. They should have better I/O throughput than 4TB drives... as long as they're not SMR. :-P A SMR drive that takes multiple days to transfer 500GB to it isn't useful as a maintained backup either.
- phil21 9y agoSMR drives really aren't intended at all for single drive applications. Get 512 of them in a giant pool (especially if it's SMR aware) and the throughput issues start to be less of a concern for certain applications. Anything write intensive of course means you immediately look elsewhere.
- loeg 9y agoThis is kind of a non-sequitur. SMR drives have vastly worse performance than non-SMR drives. Neither RPM nor capacity has anything to do with SMR or non-SMR.
- loeg 9y agoSMR drives can be thought of as tape libraries. You have a bunch of large, independent tapes that can all be written sequentially. But you can't do random writes to the tape. (SMR drives also have a lower density, non-SMR "scratch region" that can be used for metadata, or to build up to a larger I/O piecemeal. My metaphor is breaking down here.) They're not generally useful for the kinds of applications harddrives are classically used for (classic filesystems). They're mostly useful for applications that already use tape, with the benefit of much quicker random reads. Log filesystems may be able to take advantage of them. There will necessarily be I/O overhead to garbage collect deleted files out of SMR regions. (Because you have to rewrite the entire SMR region to compact.) Is anyone using or planning to use SMR drives in production? If you're able to share, I'd be curious to learn about your use case and how you plan to make efficient use of the disk.
- jedberg 9y agoSounds like they would be perfect for Cassandra, or anything else that uses SSTables.
- manigandham 9y agoWhy? SSTables/LSM storage for file data is completely separate (and much higher level) from physical storage tech like SMR. These drives are very slow compared to even standard hard drives and probably cant even keep up with writes, let alone compaction.
- fulafel 9y agoSMR drives have been shipping since 2014 (eg Seagate Archive HDD), and people are using classic filesystems with them to replace media storage, backups, and other largish file apps. It's much slower than traditional spinning HDDs at small write IO's, once you blow the write cache layer, but not tape-level terrible. You wouldn't use one for your root FS. But then again you wouldn't use a spinning HDD there anyway. There are ideas about "host-managed SMR" too where the HDD device presents the low level SMR device, and would need customized filesystems and drivers. But the idea isn't very attractive in the marketplace.
- aembleton 9y ago"2.5 million hours MTBF" How do they calculate that? That works out as 285 years! Half of these drives will last 285 years of constant operation. I find that hard to believe.
- piaste 9y agoI assume you take the first batch of N drives, run them for K weeks, see how many fail, and extrapolate. The methodology ought to be published, though.
- deleted 9y ago[deleted]
- jsmthrowaway 9y agoIt is. There’s a JEDEC standard for SSDs and probably one for spinners. Different companies use different write loads to find it. You’re pretty bang on, actually, but some vendors test for years.
- jsmthrowaway 9y agoIt’s predicted statistical failure rate in aggregate for the drive model. Two drives running for 24 hours is 48 hours for MTBF purposes, so scale MTBF by the number of drives. 1,000 drives would expect to have one failure per 104 days. 10,000 drives would expect to have one failure about every week and a half. It’s not “half these drives will run 2.5 million hours.” I agree, the “mean” is confusing until you know that. It also doesn’t predict low quantities well, including 1, for obvious reasons. The larger the quantity gets, the better the prediction tends to get IME. Backblaze has published work on this. Failure rate is also independent of lifetime. Drives have other measurements for lifetime.
- mozumder 9y agoWhat are acceptable low-number aggregate? 10? 100?
- 9y ago
- tempestn 9y agoIf you want a reasonably massive HDD for (relatively) cheap that's usable in general purpose applications, the Seagate Ironwolf is starting to look pretty good. Their 10TB drive has been hovering around the same $/TB as an average 4TB lately.
- mentos 9y agoMy hesitation to a drive this large is that you stand to lose that much more in a failure?
- dsr_ 9y agoYou are expected to deploy these in very large RAIDs or similar protection schemes. They are terrible as single disks, or even in small arrays. Think of them as an alternative to tapes.
- Dylan16807 9y agoI'd find it pretty catastrophic to lose one terabyte of data. The important part is having backups, not drive size.
- dghughes 9y agoI'm curious why helium would be used instead of a vacuum. I can imagine a vacuum would be difficult to maintain but would it be any worse than preventing helium from leaking out? Helium is rare and critical for research and MRI machines it seems wasteful to use it in hard drives.
- ykler 9y agoHard drives need a gas because the heads use it to float over the platter. And helium is hardly that rare that it is an issue; it is used in balloons for children.
- lamynator 9y agoHelium used for balloons is low grade helium that has already been recovered from another process. Medical grade helium is virtually pure helium and that is in short supply. Question whether this HDD needs pure or low grade.
- hedora 9y agoThe heads actually fly over the platter, airplane style. Vacuum would make them crash. The molecules in the atmosphere are too big for the tolerances they need for the head to track properly, so they use Helium. Note this is a much less wasteful use case than children’s balloons. Also, modern MRI machines and other research equipment wouldn’t be feasible without massive amounts of storage to back them (though these drives are probably a poor fit for those use cases, since they are soft-realtime, and shingled drives can stall for a long time in the worst case).
- known 9y agoAnd 14TB RAM in future