5 ms·
Assuming you can read/write at 512 MB/s, it would take 24-days to read/write a full Petabyte. Despite the long time to read/write, any situation where capacity
by dragontamer 6y ago
Assuming you can read/write at 512 MB/s, it would take 24-days to read/write a full Petabyte.
Despite the long time to read/write, any situation where capacity is king (aka: any Tape-drive solution today), probably benefits from something like this.
This isn't a solution where you backup / restore from per se, but instead provide dozens of backups to (and probably only restore once from). Every day, you backup up to 42TBs of data (512MBps going full tilt for all 24-hours).
Then, when your backups fail, or someone asks for data from 3 weeks ago, you rewind the tape and grab that data. Crazy that this would all fit on one tape potentially.
- bombcar 6y agoLTO speed has stopped doubling with LTO-4 - it's up to 400 MB/s uncompressed - which by the time LTO-15 is out it might be around 1GB/s or more. Still a long time to write.
- yters 6y agoCould that be mediated with many read heads at different locations?
- dragontamer 6y agoTapes only have one head. Otherwise, they couldn't wind or rewind. It'd probably be mediated with a thicker tape with a thicker head: the thicker the tape, the more "data per inch" passes through the head, which allows bandwidth to increase. That would change the LTO-tape dimensions however, so that probably isn't an option. Fortunately, by shrinking down the size of data (more data per square mm), the bandwidth necessarily increases. So maybe this technology will have a faster read/write speed than today's tech.
- klodolph 6y agoI don’t see a reason why you couldn’t have multiple heads. As it stands, in order to write a complete LTO tape, you make multiple passes across the tape. A second write head would allow you to make fewer passes across the tape to fill it. The data on an LTO tape is written in a serpentine manner, or boustrophedon if you prefer, with overlapping series of tracks separated by the servo tracks used to position the head.
- JustSomeNobody 6y agoWhat would happen when one head can't write out data fast enough (for some reason) and the tape has to slow down and back up. Now, instead of one head, you have multiple heads that have to resync and get going again. I dunno, if multiple heads were worth it, I think it would be common.
- klodolph 6y ago> What would happen when one head can't write out data fast enough (for some reason) and the tape has to slow down and back up. The tape does not rewind in this scenario. It is fairly common for the system to not be able to keep up with the full write speed of the tape. Just as a reminder, LTO7 has a write speed of 300 Mb/s, and filling a 300 Mb/s pipe for five and a half hours is no small task. There are two ways the system deals with it. First, the drive will slow down the tape to match the data rate. Second, the drive can mark a section of tape as failed and then rewrite that data to the next part of the tape. Instead of rewinding and writing over a section of tape, you’re using up more tape this way. Tapes, like hard drives and SSDs, come with a certain amount of extra capacity to allow for this as well as other write errors. > I dunno, if multiple heads were worth it, I think it would be common. I’m sure that it’s “not worth it”, yes, absolutely. But I don’t see major technical hurdles here. One technical hurdle would be figuring out how to, say, fill an 800 Mb/s pipe for six hours straight. That’s hard. It gets harder when you realize that a single tape library might have as many as 64 or 80 tape drives. 64 drives, multiplied by 400 Mb/s, is 26 Gb/s.
- dsr_ 6y agoIf you have a 64 drive tape robot, you probably have 65 computers: one controlling the load/unload mechanism, servicing 64 servers each dealing with one drive. (Or perhaps 16 servers that each handle 4, or 8x8.) Each of them has its own high speed drive for cache and independent NIC.
- klodolph 6y agoThat's part of the solution, yes, but it's difficult to get the whole system set up this way and achieve something near maximum throughput. My understanding is that most people who use tape don't get anywhere near maximum throughput over the long term. Naive use of drives is still quite common... things like backing up with tar, or dumping a database straight to tape, and it's easy to imagine buffer starvation scenarios. Returning to the idea of increasing throughput on the drives by adding heads, I'm just doubtful that it would be the best way to improve overall system performance. For alternatives, you can either get engineers to optimize usage of existing drives or buy more drives.
- frongpik 6y agoTbh, I think that in the near future we'll figure how to flip one bit worth of information in a single atom, perhaps with a high precision magnetic field, and use crystals to store amounts of data we don't have prefixes for yet.
- minitoar 6y agoDoubtful this is useful because it’s just not enough mass to be stable.
- smaddox 6y agoNot anywhere near room temperature, anyways. Plus it's not particularly helpful to store data in a single atom when it takes a much larger device to read the data. NAND flash cells are already about as small as they can be, but now they're being stacked on top of each other for better density. This is a much more promising avenue. Holographic storage also leverages the third dimension for huge increase in density compared to traditional optical drives.
- frongpik 6y agoWhat do you mean? I thought atoms in crystals are well localized and only oscillate around their average grid position.
- chefkoch 6y agoTape libraries often have multiple drives and it's possible to write parallel to them.
- sly010 6y agoI decided to invest in an LTE drive to periodically archive my ZFS based NAS. This relatively old drive support 50-150 MB/s with a small internal buffer. My biggest problem ended up being that the default tar util on ZFS on spinning rust cannot even saturate the 50MB/s consistently, so the drive had so stop and rewind (called shoe-shining). It would take ages to backup parts of my filesystem with lot of small files (i.e. git repos). I ended up writing a custom software with an adaptive scheduler to avoid sudden speed changes. That is to say as far as I am concerned my ancient LTO6 drive is too fast...
- throw0101a 6y agoIt's been best practice for a while in the 'enterprise' realm to do disk-to-disk backup first so that the streams are all sequential and then copy to tape. Enterprise-y software can also generally multiplex multiple clients to a single tape.
- kstrauser 6y agoI used Amanda Backup[0] for years on a Unix farm, and that's exactly how it works. Client apps on all of the machines being backed up ran tar and streamed the output to a backup server where it was written to disk. Then the completed files were streamed out to the tape. [0] http://www.amanda.org http://www.amanda.org
- TedDoesntTalk 6y agoI don’t understand how this scales. If you have 1 PT to backup to tape, your first have to write it to disks on a backup server? So you have to buy 1PT worth of backup disks first, is that correct?
- kstrauser 6y agoYou wouldn't usually backup the whole thing first. That 1 PB is probably a bunch of smaller projects, so you'd create a tar of the first project, then flush it to tape. Tar up the second project, then flush it to tape. It's not so bad if you're writing 23 tarballs out to tape and pausing between each one. What you really want to avoid is where the tape drive is pausing many times during a single archive, perhaps because tar can't keep the drive fed while it's traversing a big directory of teensy files.
- heimatau 6y agoWe're getting closer to writing out the Akashic records. Interesting recursion of our present moments that it becomes observable.
- adrianmonk 6y ago> Assuming you can read/write at 512 MB/s I realize that's just an assumption for the sake of discussion (and thank you for labeling it as one), but I think it's a little on the low side. Wikipedia says (https://en.wikipedia.org/wiki/Linear_Tape-Open#Generations https://en.wikipedia.org/wiki/Linear_Tape-Open#Generations) that LTO-9 can write at 400 MB/s native and LTO-10 is going to be 1100 MB/s. Since this new Fujifilm format is denser, I would expect it to be even higher speed. Although speed and density don't have a simple linear relationship, speed generally increases with density. Generating enough data to keep the tape moving without a buffer getting empty is an important consideration, but I think this product is mostly aimed at people who have enough data to back up that they can fill a tape in a few hours. (Otherwise, why spend the money on cutting edge technology?) So they probably have lots and lots of machines to back up, and you can stream backups in parallel, writing to a staging area and/or multiplexing onto the tape. Also, there's some risk if you leave a tape in the drive and append to it over the course of several days. While it's still in the drive, it's not quite an offline backup. The drive could malfunction and chew up the tape, or some software could accidentally rewind and start overwriting.