32 ms·
Petabyte tape cartridges are coming
- dragontamer 6y agoAssuming you can read/write at 512 MB/s, it would take 24-days to read/write a full Petabyte. Despite the long time to read/write, any situation where capacity is king (aka: any Tape-drive solution today), probably benefits from something like this. This isn't a solution where you backup / restore from per se, but instead provide dozens of backups to (and probably only restore once from). Every day, you backup up to 42TBs of data (512MBps going full tilt for all 24-hours). Then, when your backups fail, or someone asks for data from 3 weeks ago, you rewind the tape and grab that data. Crazy that this would all fit on one tape potentially.
- bombcar 6y agoLTO speed has stopped doubling with LTO-4 - it's up to 400 MB/s uncompressed - which by the time LTO-15 is out it might be around 1GB/s or more. Still a long time to write.
- yters 6y agoCould that be mediated with many read heads at different locations?
- dragontamer 6y agoTapes only have one head. Otherwise, they couldn't wind or rewind. It'd probably be mediated with a thicker tape with a thicker head: the thicker the tape, the more "data per inch" passes through the head, which allows bandwidth to increase. That would change the LTO-tape dimensions however, so that probably isn't an option. Fortunately, by shrinking down the size of data (more data per square mm), the bandwidth necessarily increases. So maybe this technology will have a faster read/write speed than today's tech.
- klodolph 6y agoI don’t see a reason why you couldn’t have multiple heads. As it stands, in order to write a complete LTO tape, you make multiple passes across the tape. A second write head would allow you to make fewer passes across the tape to fill it. The data on an LTO tape is written in a serpentine manner, or boustrophedon if you prefer, with overlapping series of tracks separated by the servo tracks used to position the head.
- JustSomeNobody 6y agoWhat would happen when one head can't write out data fast enough (for some reason) and the tape has to slow down and back up. Now, instead of one head, you have multiple heads that have to resync and get going again. I dunno, if multiple heads were worth it, I think it would be common.
- klodolph 6y ago> What would happen when one head can't write out data fast enough (for some reason) and the tape has to slow down and back up. The tape does not rewind in this scenario. It is fairly common for the system to not be able to keep up with the full write speed of the tape. Just as a reminder, LTO7 has a write speed of 300 Mb/s, and filling a 300 Mb/s pipe for five and a half hours is no small task. There are two ways the system deals with it. First, the drive will slow down the tape to match the data rate. Second, the drive can mark a section of tape as failed and then rewrite that data to the next part of the tape. Instead of rewinding and writing over a section of tape, you’re using up more tape this way. Tapes, like hard drives and SSDs, come with a certain amount of extra capacity to allow for this as well as other write errors. > I dunno, if multiple heads were worth it, I think it would be common. I’m sure that it’s “not worth it”, yes, absolutely. But I don’t see major technical hurdles here. One technical hurdle would be figuring out how to, say, fill an 800 Mb/s pipe for six hours straight. That’s hard. It gets harder when you realize that a single tape library might have as many as 64 or 80 tape drives. 64 drives, multiplied by 400 Mb/s, is 26 Gb/s.
- 6y ago
- frongpik 6y agoTbh, I think that in the near future we'll figure how to flip one bit worth of information in a single atom, perhaps with a high precision magnetic field, and use crystals to store amounts of data we don't have prefixes for yet.
- minitoar 6y agoDoubtful this is useful because it’s just not enough mass to be stable.
- smaddox 6y agoNot anywhere near room temperature, anyways. Plus it's not particularly helpful to store data in a single atom when it takes a much larger device to read the data. NAND flash cells are already about as small as they can be, but now they're being stacked on top of each other for better density. This is a much more promising avenue. Holographic storage also leverages the third dimension for huge increase in density compared to traditional optical drives.
- frongpik 6y agoWhat do you mean? I thought atoms in crystals are well localized and only oscillate around their average grid position.
- chefkoch 6y agoTape libraries often have multiple drives and it's possible to write parallel to them.
- sly010 6y agoI decided to invest in an LTE drive to periodically archive my ZFS based NAS. This relatively old drive support 50-150 MB/s with a small internal buffer. My biggest problem ended up being that the default tar util on ZFS on spinning rust cannot even saturate the 50MB/s consistently, so the drive had so stop and rewind (called shoe-shining). It would take ages to backup parts of my filesystem with lot of small files (i.e. git repos). I ended up writing a custom software with an adaptive scheduler to avoid sudden speed changes. That is to say as far as I am concerned my ancient LTO6 drive is too fast...
- throw0101a 6y agoIt's been best practice for a while in the 'enterprise' realm to do disk-to-disk backup first so that the streams are all sequential and then copy to tape. Enterprise-y software can also generally multiplex multiple clients to a single tape.
- kstrauser 6y agoI used Amanda Backup[0] for years on a Unix farm, and that's exactly how it works. Client apps on all of the machines being backed up ran tar and streamed the output to a backup server where it was written to disk. Then the completed files were streamed out to the tape. [0] http://www.amanda.org http://www.amanda.org
- TedDoesntTalk 6y agoI don’t understand how this scales. If you have 1 PT to backup to tape, your first have to write it to disks on a backup server? So you have to buy 1PT worth of backup disks first, is that correct?
- kstrauser 6y agoYou wouldn't usually backup the whole thing first. That 1 PB is probably a bunch of smaller projects, so you'd create a tar of the first project, then flush it to tape. Tar up the second project, then flush it to tape. It's not so bad if you're writing 23 tarballs out to tape and pausing between each one. What you really want to avoid is where the tape drive is pausing many times during a single archive, perhaps because tar can't keep the drive fed while it's traversing a big directory of teensy files.
- heimatau 6y agoWe're getting closer to writing out the Akashic records. Interesting recursion of our present moments that it becomes observable.
- adrianmonk 6y ago> Assuming you can read/write at 512 MB/s I realize that's just an assumption for the sake of discussion (and thank you for labeling it as one), but I think it's a little on the low side. Wikipedia says (https://en.wikipedia.org/wiki/Linear_Tape-Open#Generations https://en.wikipedia.org/wiki/Linear_Tape-Open#Generations) that LTO-9 can write at 400 MB/s native and LTO-10 is going to be 1100 MB/s. Since this new Fujifilm format is denser, I would expect it to be even higher speed. Although speed and density don't have a simple linear relationship, speed generally increases with density. Generating enough data to keep the tape moving without a buffer getting empty is an important consideration, but I think this product is mostly aimed at people who have enough data to back up that they can fill a tape in a few hours. (Otherwise, why spend the money on cutting edge technology?) So they probably have lots and lots of machines to back up, and you can stream backups in parallel, writing to a staging area and/or multiplexing onto the tape. Also, there's some risk if you leave a tape in the drive and append to it over the course of several days. While it's still in the drive, it's not quite an offline backup. The drive could malfunction and chew up the tape, or some software could accidentally rewind and start overwriting.
- at_a_remove 6y agoThe generations so far has been roughly once every 2.5 years. Stretch it out to three years. Nor has each generation provided a doubling. Let's suggest a factor of 1 and two-thirds, to be on the safe side. We could then expect LTO-10 in 2023, LTO-11 in 2026, LTO-12 in 2029, LTO-13 in 2032, and LTO-14 in 2035, with a final uncompressed capacity of about 234 TB. Note that I have been conservative for both the generations and the increases in capacity. If you squeezed in an extra generation and assumed a full doubling each time, you would end up at 1,152 TB, or the 1 PB predicted by the article. This looks to be ... right on schedule. Now if the cost of the drives would only come down ...
- samstave 6y agoHonest question, is it the data-capacity-scale of LTO tape to be the driving factor instead of using a lot of micro SSD for example? I haven't used LTO tapes in a long while so I am not current on modern backup strategies... but is tape still king in backups?
- jandrese 6y agoMicroSD cards are nowhere near reliable enough to stripe a petabyte of data across them. At current prices it would also cost around $120,000 + the cost of the readers and the robot to swap them around.
- Beached 6y agotape is still king of long term or cold backups. most people also keep hot backup on hdd or ssd in addition. but there are usually off prem long term cold storage backups along with these and those are very often tape. these tapes are for like FUUULLL dr and not often used but kept as insurance against the hot short term backups getting hosed
- at_a_remove 6y agoJust geometrically, tape will be king for ... quite some time. Imagine a HDD platter. You can only write to the surface of it. That's a small surface area, when you think about it. Now imagine the surface area of an unspooled tape ... Of course, SSDs are a different beast but they are nowhere near as dense. The spooling and unspooling for the geometric advantage is a tradeoff between time and space. Therefore, tape will continue to reign supreme for colder backup up until such time as some kind of holographic, volume-penetrating technique reaches the sheer density of a wound tape, adjusted by some kind of time factor for the read-write of our hypothetical holocube.
- des429 6y agoJust curious — who's using these today? Is it mainly for cases where you need to be able to easily physically transfer data?
- dolni 6y agoBig companies that have a sizable on-prem data center and need a cost effective way to store backups off site. At least that was the case at a place I worked ~5 years ago.
- tomerico 6y agoAn example is the large hadron colider, where each experiment produces an enormous amount of data from all sensors.
- atty 6y agoThat’s not just the LHC - it’s pretty common across high energy physics, nuclear physics, and astronomy, at least. It’s a pretty great fit for the technology. You keep the most recent data on hard drives for analysis, and as new data comes online, the old data goes to tape. You don’t want to get rid of the old data, of course, because you need it for analysis verification and when people come up with new analysis ideas, but the older data naturally gets accessed less and less as time goes on, making tape storage a natural choice.
- deleted 6y ago[deleted]
- deleted 6y ago[deleted]
- hobojones 6y agoI've been told that NOAA uses tape to stream time series weather data into long-running simulations.
- fulafel 6y agoIt's probably more expensive to use disk for what you'd get from a tape robot, considering the worse reliability and shelf life of disks and near-absence of hard disk loading robots on the market.
- 1MachineElf 6y agoWhat would be the best way to handle a tape drive like this? Imagine investing in a 1PB tape, and then accidentally dropping it on the floor. Even just setting it down on a table too hard might brick it, or mean it needs to go back to the manufacturer for a expen$$$$$ive repair. Gravity seems like an unreasonably large risk for such a valuable object. Maybe the ideal environment for these is a zero-gravity one? Perhaps the future of "big data" is somewhere out in space?
- eurekin 6y agoOr forgetting to remove from pocket during an MRI scan
- atlbeer 6y agoGoogle: "tape library robot"
- RandallBrown 6y agoAre tapes particularly fragile? At least any more fragile than most other media? I remember being pretty rough with audio and video cassettes, but then again those aren't digital so some damage probably wasn't noticeable.
- Sharlin 6y agoThese are not meant to be handled by human hands.
- madpata 6y agoAre you called Linus Sebastian?
- KaiserPro 6y agothese tapes are going to be fairly cheap (<$400) by 2035. But more importantly, if its an LTO-style tape, it'll be fairly solid. even without the case, you can drop them and not worry too much. Lord knows I've dropped loads in my time.
- fl0wenol 6y agoIf you google this technical white paper from HP (a major LTO vendor) you'll see that the format (in all iterations) is designed to be pretty durable: 4AA4_3781EEE.pdf In particular, they test dropping tapes hundreds of times from 30" AFF onto concrete in different orientations, making sure they'll still work properly. Not every vendor does this, but they all like to be perceived as archival quality and durable. I wouldn't be too worried.
- Jkvngt 6y agoRemember 3D holographic data cubes? I was promised, in the mid-1990s, that we'd have them by 2000 and that they would have "up to" 100 times the storage capacity of a DVD.
- busterarm 6y agoAlso microfiber-based data storage media (first paper on this was in 1998) which require new lasers that we still don't have yet.
- Jkvngt 6y agoI guess I'm just going to have to see it in a store to believe it.
- mnw21cam 6y agoYeah, so we currently have micro-SD cards with a 1TB capacity, which is 100 times the storage capacity of a DVD. I see your holographic data cube and raise you a fingernail.
- Jkvngt 6y agoSo you're saying this tape claim will end up being some new form of CD?
- google234123 6y agoThat's not at all what he is saying.
- swiley 6y agoI guess it's still glass with incredibly tiny details etched into it, just not quite as fantastic looking.
- vram22 6y ago>I see your holographic data cube and raise you a fingernail. Reminds me of this comment: https://news.ycombinator.com/item?id=17647545 https://news.ycombinator.com/item?id=17647545
- nayuki 6y agoWhy can't we have higher density optical disc formats? Is BDXL at 128 GB really the end of technological innovation?
- aidenn0 6y agoPrevious optical disc formats reached critical volume from content companies. Now that it's possible, they would much rather get the recurring revenue of you renting access to their content.
- ghaff 6y agoThey're happy to sell you Blu-Ray discs if that's what you want. Consumers, which is where most of the market is, don't really need higher capacity for anything. And even a 10x increase for BDXL probably doesn't really get you to a capacity that's all that useful for most types of archiving.
- djrogers 6y ago> And even a 10x increase for BDXL probably doesn't really get you to a capacity that's all that useful for most types of archiving Sure it would - it would get over the 1Tb point, which would mean most users could back up their devices to a single disc again.
- ghaff 6y agoI expect the economics wouldn't work out for most consumers. (And even if they did, how many would actually make regular backups?) And I'd point out that they can backup, even in an automated way, multiple terabytes to about a $100 disk drive today.
- ClumsyPilot 6y agoThats not very usefull for an offline backup, Optical is meant to past decades on a shelf in a way HDDs do not. They can be put in a letter, etc.
- peter_d_sherman 6y agoFascinating! My curiosity is, could functioning transistors somehow be created on either a BaFe or SrFe substrate by a magnetic or other electromagnetic beam of some sort? Perhaps one of those is the material for doing something like that... Anyway, amazing storage capacity!
- russellbeattie 6y ago14 years from now? Call me a luddite, but I'm going to hold off even thinking about this until then. Maybe I'll get one for my retirement party a couple years later...
- asien 6y agoSame reaction. That’s a hell lot of time honestly, I don’t what the world will look like in 15 years. That said 1PTB is really massive amount of data
- ksec 6y agoWould there be a market for a smaller pro-sumer version Tape Drive Backup? May be the size of Mini Disc Or Zip Disc at around 5TB. What are the current strategy for consumer offline backup. Simply copying to another HDD?
- adrianmonk 6y agoThere used to be, but that market died. Back in the 90s, some formats were relatively popular with power users and small businesses: QIC (https://en.wikipedia.org/wiki/Quarter-inch_cartridge https://en.wikipedia.org/wiki/Quarter-inch_cartridge) Travan (https://en.wikipedia.org/wiki/Travan https://en.wikipedia.org/wiki/Travan) Data8 (https://en.wikipedia.org/wiki/Data8 https://en.wikipedia.org/wiki/Data8) I think what killed them is economics. The per-gigabyte cost is lower because tape media is cheaper than hard drives, but the initial investment is much higher because tape drives are expensive. The reason tape drives are expensive probably has to do with manufacturing volume but I'm sure also it's because they are mechanically complex. You need to wind tape onto a spool, you need to keep it at the right tension, and there's the loading/unloading of the tape cartridge. And tape drives have wear and tear (tape physically dragging across the head), and when the drive breaks, it's expensive to repair, which is big drawback for an individual.
- rusk 6y agoBeing able to write your own optical media killed off tapes as consumer devices. Now that optical drives are on the way out there might be a place for tapes again ...
- andrewflnr 6y agoWhen would they be better than flash storage, though?
- jamiek88 6y agoFor longevity I suppose. Don’t you need to power on flash storage occasionally?
- deleted 6y ago[deleted]
- gorgoiler 6y agoI cut my teeth on tape — Travans, IBM3590, IBM3592... Cold storage is a wonderful thing. A mechanical bridge from the ethereal to the real world with a true disconnect between the medium and the mechanism. I worry about the longevity of our data but some people are doing good things. Act now! Support your local internet archive!
- tzs 6y agoHere's what I want. A box that has a tape drive and a hard disk. For every disk write my computer does, I want it to send to the box a copy of the data, what disk it was written to, and what sector number. I want the box to store that information on its hard disk. When it accumulates enough that writing it to tape would not be too inefficient [1], it should write it out to tape and free the space on its hard disk. I monitored my SSD writes for a few years, to see how long it would be before I had to worry about hitting their write limits. I found that I was writing under 3 TB/year on both my home and work machines. A box like that, even with a modest size current tape cartridge, would be enough to handle all of my computers for several years before filling up the first cartridge. [1] I don't know if this is still the case, but it used to be that when tape drives stopped and started, you'd get a gap that wasted some tape.
- Keyframe 6y agoSo, backblaze / dropbox.. but in a physical box?
- jjeaff 6y agoPresumably much less expensive since cloud storage providers are mostly instant access data stored on HDDs and network connected to the cloud. Plus, I'm sure writing to tape is slow. But it has to be faster than most people's paltry upload speed (at least in the US).
- PurpleFoxy 6y agoTapes might be cheaper than HDDs but the tape drives are obscenely expensive to the point where it only makes sense to buy one if it would replace at least 100 hard drives.
- nerpderp82 6y agoWhat if we used the 1Gbps lines we have to our houses to sync data to a neighborhood level tape back system. We could just swing by the minimart and pickup some beer and our weekly backup on a datacart.
- ratiolat 6y agoKids these days... Spectrum protect/Tivoli storage manager/adsm is where the backup is. Since 1989.
- tomc1985 6y agoIs any of this ever going to be accessible to a consumer market? Or is it always going to cost gobs of money?
- athms 6y agoNo. The consumer and SOHO tape market died around 2000 as companies sold off their business units and/or went bankrupt. Trickle down in the LTO range is buying a refurbished drive that is several generations behind and you better have SCSI or SAS in your computer.
- kristopolous 6y ago2035 is a long way off. I've seen estimates of 1PB on ssd coming in 2023. I honestly don't see the numbers working out for tape if this is the trajectory