6 ms·
Once the latency requirement goes past ... i dunno, a few seconds? "Insh'allah" time frames; tape comes in and stomps all over the competition with its space /
by h2odragon 5y ago
Once the latency requirement goes past ... i dunno, a few seconds? "Insh'allah" time frames; tape comes in and stomps all over the competition with its space / cost tradeoff. Dismountable media and a mechanism to swap tapes (even a meatsicle if need be) just has too many advantages to compete with.
Gets me thinking tho: Why stop at 5.25in drives? Let's go back to washing machine sized units or even bigger. Let's stack platters so dense and heavy that we can use the spindle as a flywheel UPS system too.
Or how about making a truly gigantic tape spool? Perhaps kevlar threads impregnated with iron could serve as "tape" and we could spin kilometers of that onto a spool.
- bluedino 5y agoQuantum's marketing for their BigFoot 5.25" drives was interesting. They argued that a 3600 RPM drive was as fast as 5400 RPM 3.5" drive because you're spinning a larger diameter platter. Technically true. They also claimed that since you could fit more data on a track, since it was larger, you would do less seeking, and that's what makes mechanical hard drives slow. This didn't stop BigFoot drives from being slower than 3.5" drives that were 3 years older and 1/4 of the capacity.
- h2odragon 5y agoI loved my bigfoot drives. Yes they were slow, but they was cheap and ran cool and (excepting certain lines) proved amazingly durable. As in, "ran as the mail spool disk for a decade" for a system sending ~5k emails a day. That one i think i still have on a shelf here somewhere with a "well done thou faithful servant" postit note
- bluedino 5y agoIn my time doing tech support and repair, they were probably the most replaced drive to failure that I ever experienced. At least until the Deathstars came out.
- h2odragon 5y agoI think i was buying refurbs then. as i recall the issue was with the controller boards. i know a bunch got used as cheap system/boot disks on alphas at CGI render farms, too: some of my stock came from surplus "Titanic" render nodes. I recall they had special cases for them; 24 pack pink foam crates. I had at least two of the crates...
- bluedino 5y agoCompaq must have been a big buyer of them at the time - they used them in quite a few Presario models (home models, cost cutting I'm sure)
- GekkePrutser 5y agoYes I remember those foam crates. 10% of a new crate of Bigfoots were DOA right out of the box. But I guess the ones that lasted more than a week were much more likely to last years.
- wongarsu 5y agoTape is great for large-scale setups, with the cost all being in the drives and media being dirt cheap. But what if I just want a backup of my personal or small business's data to put into a safe. Even regular hard drives are far more economical than spending a couple thousand on a previous-generation LTO drive, or many hundred on an even older model.
- ComputerGuru 5y agoI was going through that same dilemma but they’re really not perfectly comparable options. Drives are more sensitive to storage and handling damage than a tape cartridge is.
- selfhoster11 5y agoAn LTO4 drive is capable of writing 800GB raw capacity cartridges, and it can be had for ~$120.
- wongarsu 5y agoLTO4 cartridges go for about $10-20 each. At about $15 they break even with hard drives in price per GB. If you were willing to go with used hard drives the break even point is closer to $10 per cartridge. It's useful if you get a good deal or if you like the unique advantages of tape (smaller medium with decent shelf life), but otherwise the price advantage seems dubious.
- formerly_proven 5y agoTape has an inherent density advantage over hard drives because tape stores data in the volume - tape is a 3D storage technology. Hard drives are limited to recording on a handful of slices through a cylinder inside them; tapes record on the peeled surface of a cylinder roughly the same size. One tape cartridge contains more than 30 times as much surface area as the highest-density hard drives for recording data.
- colejohnson66 5y agoThe trade off to all that capacity is latency. But for archival purposes, it doesn’t matter if it takes 2 seconds or 2 minutes. As long as the writes are quick enough to keep up with the data.
- monocasa 5y agoAnd tape has (historically at least; I haven't check in for a few years) been faster at purely streaming loads than spinning rust. Pre SSDs, there'd be systems that streamed to tape to keep up with input data, then would offline load that into spinning rust databases to crank numbers where seeking dominated.
- ComputerGuru 5y agoYup. A well-known issue with tape is that machines have a hard time supplying it with data fast enough to keep up with its linear travel speed when creating the backups.
- selfhoster11 5y agoI believe that LTO can actually regulate the speed of the tape
- ComputerGuru 5y agoIt can, within reason. Also source machines (and more importantly, source media) are much faster in the first place this side of 2010.
- notacoward 5y agoTape also has some inherent disadvantages. Perhaps surprisingly, the lack of near-instantaneous seek to the data you want isn't one of them. The larger issue at scale is that it's not disk. It's a separate kind of device, with its own power and (often very stringent) environmental requirements, and its own software. Also big silos aren't cheap, both because of inherent cost and for lack of significant competition. Hyperscalers hate that type of heterogeneity, so they'll only go down that road if they feel they have no other choice. Yes, media are cheap, but the extra operational complexity can obliterate that advantage. A lot of this doesn't apply as you go smaller, but also the drive cost starts to dominate (can't be amortized over many cheap tapes). There's a pretty narrow band where tape is likely to have a real advantage over super-high-capacity disks.
- m0lecules 5y agoWhy can't tape be self-contained like HDDs? Aside from the fact that most people using tape for archival storage don't want to pay extra for the read/write heads, SATA interface, etc., there is no reason why you couldn't package all these things into a self-contained tape unit with a small flash disk acting as a small cache and directory listing. You could definitely package such a thing for consumers, for example, but most workloads there aren't a great fit for the medium. Basically the only thing that makes sense is using it for archival and backups.
- dragontamer 5y ago> Why can't tape be self-contained like HDDs? Because that gets rid of the main advantage of tape. Which is that the tape-media has no read/write head and is therefore much much much cheaper to mass produce. --------- In practice, people buy tape-libraries entirely. Like a 3U unit with 50-tape slots + a few drives to read/write to those tapes, and then hook them up to the network. https://www.quantum.com/en/products/tape-storage/ https://www.quantum.com/en/products/tape-storage/ From this perspective, you buy as many tapes as you want storage (aiming for 500TB? Buy like 40 LTO7 tapes for your library. Aiming for 1000TB? Buy 80 LTO7 tapes for your library, assuming compression of course). From there, you just read/write to the library, and have the underlying software handle the details, like any other NAS.
- 323 5y agoI have a need to store a few hundred TB of data, and tape is extremely expensive for this, since it has a $5k+ upfront cost. So I just buy external $200 USB hard drives. There is a market for something lower latency, higher capacity, but pay as you go (no upfront cost). Frankly I wouldn't mind some sort of giant disk drive, vinyl record size, if that makes sense and is lower cost/TB.
- h2odragon 5y agoYes. There was a time when the tape drive wasn't so expensive; the densities weren't there either but that market climbed the wrong side of the curve in a couple of ways. The modern tape drives are miracles of precision and the tapes are these cool little cartridges but why? The meat and machines handling tape carts liked the of 3/4in format just fine. With modern electronics for the write heads and slightly less ruinously expensive loading and running hardware we could still get decent data density, even if we have to hunt for the tracks each tape load.
- jmwilson 5y agoIt didn't used to be this way. Tape was once affordable for individuals; ca. 2000 Travan was marketed to consumers and DDS wasn't too much of a stretch for prosumers. But they're all gone now and only LTO is still standing. Given the small, cornered market segment that tape now serves, there is no incentive to bring the price down. I'd like the option of being able to manage my own backups locally instead of using cloud storage, but the economics of Glacier remain too good until the storage size is several times what I could foresee needing.
- selfhoster11 5y agoMy tape setup cost less than a single external hard drive. Older tape techs come down in price in time.
- wazoox 5y agoMy company has built something for that. Basically you have a nextcloud instance that you fill up with your data, then you ask for an LTO tape to be written with it. Repeat ad infinitum. The tapes are yours; when you buy a tape it comes with 3 free operations (read, write, checksum control). If you want your tapes back, we'll mail them to you, else we keep them in storage for you. As the tapes are written using LTFS, you can easily read them back anywhere with the proper drive. You only pay a fee for the cloud storage; it also gives you access to your archive's database (what file is on which tape, etc).
- memco 5y agoIt’s true tapes are okay when time is not of the essence, but there’s currently a lot of pain dealing with tapes when it comes to loading reading writing and verifying everything. You need some way to make sure the data on the tape matches the original after writing. Then you need some way to verify the copy you get back from the tape matches. You need to have multiple copies because sometimes whole tapes go bad. You also have a limited number of times you can try to read or write the tape without too much wear so you have to have a plan for how to read or write those things efficiently. This is stuff that can be mostly automated but it’s surprising to me that mostly this is done manually and the tools that do exist are often bespoke in one way or another. A nice thing about a hard drive is that most of that logic is baked into the firmware of the drive so you are less dependent on having a good system for verification and record keeping (though you still need it). Tapes can currently take minutes to load and hours to read or write if you need to copy the whole thing. I’m sure if someone was going to make those dice of drives they’d figure out the economics and logistics but it would be a monumental effort both technically and physically to make it work well.
- dragontamer 5y ago> You need some way to make sure the data on the tape matches the original after writing. Do you? Virtually every filesystem has some kind of error detection, maybe even error-correction built in. That seems like a solved problem to me. CD-R solved this by writing Reed-Solomon codes every couple of bytes so that if any error occurred, you could just fix them on the fly. (As such: you could have scratches erase all sorts of data, but still read the data back just fine) I have to imagine that tapes have a similar kind of error-correction going on, using whatever is popular these days (LDPC?). Once you have error correction and error detection, you just read/write as usual. ------- If Tapes don't have that sort of correction/detection built in, you can build it out at the software level (like Backblaze used to do)... or maybe like Parchive (https://en.wikipedia.org/wiki/Parchive https://en.wikipedia.org/wiki/Parchive).
- snerbles 5y ago> or maybe like Parchive par2 even has options for specifying level of redundancy. I've had good experience in recovering large corrupted files from an external drive - since then, I've incorporated it into the automated backups of my personal infrastructure. https://github.com/Parchive/par2cmdline https://github.com/Parchive/par2cmdline
- dragontamer 5y ago> Gets me thinking tho: Why stop at 5.25in drives? Let's go back to washing machine sized units or even bigger. Let's stack platters so dense and heavy that we can use the spindle as a flywheel UPS system too. Because modern 20TB hard drives already take like a full week to read from beginning to end (aka: time for a RAID6 rebuild), which is too long as it is. The problem with hard drives is that they need "more read heads per byte" than the current technology. You can solve this by making more hard drives (ex: RAIDing together smaller drives, say 4x5TBs), so you have more read/write heads going over your data faster. There's multi-actuator hard drives coming up (2 independent heads per drive), which will report themselves to the OS as basically 2-hard drives in one case. I think that's where the storage market needs to go. ------------ Tape is the king of density: having the most bytes of storage on the fewest number of read/write heads possible. But Hard Drives encroach upon that space, especially with like 20TBs on one read/write head. > Or how about making a truly gigantic tape spool? Perhaps kevlar threads impregnated with iron could serve as "tape" and we could spin kilometers of that onto a spool. There's no advantage to that. A stack of 300 LTO-tapes will practically use up the same amount of space as one-tape that's 300x longer. (Besides, LTO-8 tapes are 960 meters long, roughly 1km. The idea of pulling on a thing that's 300km long and hoping for it to not rip itself apart is... pretty crazy. There's definitely some physical constraints / practicality with regards to just material science: shear/stress kind of calculations) The "jukebox" concept... really a tape-libraries (robot that picks tapes out of storage compartments and shoves them into the drive) is the ultimate solution to density.
- hakfoo 5y agoI never understood why they didn't make a hard disc that operated its heads in parallel and distributed data accordingly. Still one assembly, but using all the resources it has. You'd have (for example) a four-platter, eight head drive, and instead of storing a given byte serially at cylinder 123, head 0, sector 5, bytes 0-7, you'd store it at cylinder 123, sector 5, byte 0, and light up heads 0-7 all at once. Now, this might have been hard with early logical drive designs that tracked the physical geometry and expected a more serial format, but that was long ago hidden beneath LBA translation. Maybe there's a power or crosstalk issue with activating 8 heads at once.
- jasonhansel 5y agoI'm curious as to whether there would be a market for a new, more modern form of tape-based storage, one that incorporated ideas from modern HDD design, had a lower barrier to entry, and/or was based on open-source hardware outside of an existing proprietary ecosystem.
- Aerroon 5y agoTape, at least right now, seems to have a pretty steep entry barrier though. Tape drives cost thousands of dollars and each tape itself seems to be fairly expensive too. About $50 for an LTO-7 tape that's 6 TB. That's $8-9 per TB just for the tape. HDDs seem to be around $20-25 per TB. Need to 300 TB of capacity? 50 tapes at $50 totals $2,500 + a few thousand dollars for the tape drive. Or you could buy about 12x 18TB external drives for about $450 each totaling $5,400. About the same.
- selfhoster11 5y agoFor home use, you can probably get an LTO4 setup going for less than $250 (including some used tapes). LTO-5 will remain write compatible with your tapes and LTO6 will remain read compatible, so there's a forward migration path that will keep things cheaper.
- Lramseyer 5y ago> Why stop at 5.25in drives? Let's go back to washing machine sized units or even bigger. Let's stack platters so dense and heavy that we can use the spindle as a flywheel UPS system too. Current HDD tracks are around 50-60nm wide, and their limitation is not magnetic grain density (those are around 10-12nm for non-HAMR/MAMR substrates and even smaller for HAMR/MAMR substrates.) The limitation is the long (relatively speaking) actuator arm trying to precisely stay on a very narrow data track. There's actuator flexure, disk flutter, and aerodynamic noise, all of which increase with increasing platter radius. This increases your minimum track width, and minimum disk thickness/minimum disk spacing and end up defeating the purpose of bigger disks in the first place. Also it's important to keep in mind that in order to manufacture a drive, the entire drive is written to and read back multiple times, which takes an increasingly long time, creating a huge push for some steps to be combined. That all being said, I'm not entirely sure where the sweet spot in terms of data density per unit volume. It might be slightly more efficient with 5.25 drives. But I can assure you that the record sized or washing machine sized HDDs of olden days are not practical anymore ...though they would be dope if they did exist!