4 ms·

Tape has two really important features that HDDs don't: o Linear speed o transportability Assuming you can feed the data to a tape drive, it will run at top

Tape has two really important features that HDDs don't: o Linear speed o transportability Assuming you can feed the data to a tape drive, it will run at top speed until the tape is full. When you have a tape robot with 25+ drives, this means you can easily keep up with a 40/100gig pipe LTO tapes really are portable. They have been designed to stand up to dropping. The abuse our tapes would get when being transported by the understandably grumpy runners was considerable. We could always get the data out of them. Portable HDDs, not so much.

Dylan16807 6y ago
> Linear speed 360 megabytes per second for LTO-8. You can consistently match/beat that linear speed with two hard drives, each costing a tenth as much as a tape drive.
- dTal 6y ago
  I feel this comparison is a little glib, considering that when the hard drives fill up you need to buy two new ones, while the tape drive only needs a new tape.
  - skoskie 6y ago
    Fair, but the upfront price of a tape drive is $3-5k. The cost benefit comes down to the equilibrium and how much storage you need.
    - sukilot 6y ago
      Can I take a drive to a copy shop to convert it to tape for $50? (If not, why not?)
      - est31 6y ago
        Pretty sure that there are colocation places out there that have this, but for copy shops I think it wouldn't work because in general, people have too few data. Most people are perfectly served by the HDDs/SSDs that ship with their computers, and the minority that's not served often buys one or two external HDDs of the newest generation. That's enough for the largest part of the population. Fewest private people have data on the order of dozens of TB where HDDs become so expensive that you are motivated to use the user unfriendly tapes. Why are tapes user unfriendly? For each read/write operation you need to visit the copy shop. This limits the use cases to ones where like backups and archival. But you can't really use them for archival as when you don't archive the drive together with your tapes, who guarantees that a copy shop will exist in 10 years that has drives that can read such old tapes? Yes, tape drives have limited read compatibility to older standards. https://en.wikipedia.org/wiki/Linear_Tape-Open#Compatibility https://en.wikipedia.org/wiki/Linear_Tape-Open#Compatibility But maybe there is a niche for it somewhere, who knows.
      - waste_monk 6y ago
        There are a few good reasons why not: 1) An LTO-8 tape (capacity approx 30TB) is around $200 to $300 AUD (depending on brand), so that figure could only work if you're excluding the cost of the tape. LTO-7 (~15TB) costs around $100 to $150 AUD and would probably be more suitable, but it's still a lot of money for what's essentially a consumable item. Also, I would imagine that if the choice was between buying a tape that requires you to go to a third party to use or invest thousands of dollars in a tape drive, or just buying an external HDD you can use at home and unplugging it when not in use (so cryptolockers etc. can't eat your backups), most people would go for the external HDD. Unfortunately LTO-8 drives can't read LTO-6 or earlier (previously the rule of thumb was drives could read/write N -1 and read N - 2 of the drive generation e.g. an LTO-7 could read but not write LTO-5 tapes), and it probably wouldn't make sense to invest in outdated drives capable of writing LTO-6 (~6.25TB) or earlier, even if you could obtain the older LTO-6 or smaller media relatively cheaply. Vendors would not want to support it. 2) Most backup software I'm aware of is designed around recurring jobs (e.g. BackupExec), not one-off jobs, and to store data from a number of backup targets onto a single tape (or set of tapes). You could work around this, but there's no off-the-shelf software for doing so I'm aware of (I know some versions of windows server backup can write direct to tape, but this would have to cope with a variety of source media and also make sure that jobs from different customers are never co-mingled on the same tape, etc.). 3) The cost of the hardware required to achieve a decent turnaround time on the tapes (depending on target time e.g. some customers might pay a premium for same-day conversion) would probably exceed what you're likely to earn selling this service. I doubt it would be financially viable, simply due to the low number of customers.
        StillBored 6y ago
        "You could work around this, but there's no off-the-shelf software for doing so I'm aware of (I know some versions of windows server backup can write direct to tape," I think what you are looking for is LTFS.
        waste_monk 6y ago
        Not so much the filesystem, more that assuming you have a server with a multitude of our customers HDD's attached as well as one or more tape libraries / drives, you need to: * Map 1 to N HDDs to one customer (by USB port ID or perhaps SATA port if you shuck them for performance reasons, maybe a mix of SATA and USB drives) - will need to have some means of clearly indicating ports, or have blinky LEDs next to them and have the server indicate where stuff ought to be plugged in. And handle SATA hot-swap if required. * Allocate a tape or tapes to them (and if they provide their own tape/s make sure to use specific slot/s in the tape library for that customer) and otherwise handle the tape library logistics. This becomes more complex if you have multiple libraries in play. * Perform scheduling and ETA estimation to see if you'll actually get their data onto the tape within the deadline (you might offer a turnaround of a day, but offer a premium service to have it done within N hours) and see if it will affect other customers. This may even involve deciding that an existing currently running job is early enough that you might stop it and scratch the tapes, then run the priority job instead. * Handle predictive maintenance, drive cleaning, etc. for the tape library (and how it will affect customer deadlines / capacity planning) * Actually copying the data onto the tape/s and verifying the result * Integrate into a monitoring and notification system so customers are alerted when their order is ready to pick up, talk to customer billing/sales systems, etc. * Probably a bunch of other stuff I haven't thought of yet The system would generally have to be resilient enough that it can run unattended overnight / for long periods of time, but flexible enough in scheduling that it can handle sudden rush orders, drives being offline for maintenance, and so on. And ideally could be operated by staff with minimal technical background. Maybe Iron Mountain or someone like that have this sort of system already, but I am not aware of any commercially or generally available software that could do the job.
      - KaiserPro 6y ago
        Thats not where tape it useful. Tape only makes sense when you have tape robot. A bit rack type thing that has more than 5 drives and thousands of slots. Unless you are on a very tight power/heat budget and you have a brilliant asset management/storage system then tape isn't the answer.
  - Dylan16807 6y ago
    I don't see how that matters in a discussion of speed. Pick any budget. For that budget, hard drives are faster than tape. Tape's advantages are elsewhere.
- StillBored 6y ago
  Tapes have become "enterprise" gear.. that is why they are expensive. HD's are looking to replicate those markups if the pricing on WD reds vs the usb drives are any indication. There was a time that a tape drives cost about the same as a floppy/zip/etc drive. AKA a hundred to two bucks. In the early 1990's QIC80 tapes were pretty popular in certain circles as an exchange medium since they fit a hundred or so floppies in a package that was smaller and a little thicker. Later things like the 3080 and the like bumped the capacity into the GB range for similar pricing. Back then pretty much ever novell/etc server in existence had a tape drive in the front being used as a backup mechanism. The sysadmin dropped a tape in on friday, and took it offsite on monday (or whatever). In unix land it was possible to completely single step boot from tape and recover using things like AIX's mksysb. The death of tape, wasn't really a death so much as a giant margin increase, where a drive that cost $200 suddenly cost $2000 and came with a fibre channel interface rather than IDE..
  - xbmcuser 6y ago
    Cost per gb tapes are not expensive tape drives on the other hand
- KaiserPro 6y ago
  LTO-8 is 12tb a 12tb drive is more expensive than LTO. Even at volume. 2*6tb drives is still more expensive. However thats not the use case right, HDDs do general storage really well. A tape robot is the place where your data goes to rest. Its the last line in your tiered storage. In AWS land it goes RAM->EBS->s3->glacier tape is the same stage as glacier. You have RAM cache (be it redis et al, or some fancy block storage) Screaming fast SSDs, but only a few TBs, Slower cheaper SSDs, but ten times as more. Then if you need lots of storage you'd have a few PB of Spinny disks. After that it goes to Tape. Its denser, cooler and far less expensive to run than the equivalent spinny disk array. Now this only makes sense if you have data in the PB range, and its more data than you can store online. Anything less and it doesn't make sense. It never did, its just that the capacity of storage has ballooned so much in the last 15 years that people's perceptions haven't caught up.
  - Dylan16807 6y ago
    You're neglecting that the tapes have to go in something. LTO-8 drives are vastly more expensive than 12TB hard drives. A pile of tapes, where the vast majority of the tapes are not in drives, is cheaper than hard drives. But then you're not getting those high speeds anymore. - Let's do some simplified math. 12TB hard drives for $26/TB, LTO-8 tapes for $9/TB, and tape drives for $3500. Since you're saving $17/TB, you need 205TB before tape gets cheaper. So 17 tapes. Each hard drive takes about 18 hours to fill. So 17 hard drives take 18 hours to fill, at a total speed of 3GB/s. Each tape takes about 9 hours to fill. So 17 tapes take six and a half days to fill, at a total speed of 360MB/s. If you have enough tapes per drive to make tape cheaper than hard drives, it's slooowwww. If you have fewer tapes per drive, it can be reasonably fast, but never as fast as hard drives. Tape has a lot of advantages. But not speed.
    - caf 6y ago
      Does the $26/TB for hard drives include the cost of the JBOD chassis you'll need to to have them all online at once, in order to get those speeds?
      - 6y ago
ramraj07 6y ago
My lab acquired a lot of imaging data (10 TB a year) and notoriously would store it long-term by putting the hardrive back in thr grey static plastic bag and putting it in a wooden office drawer. Surprise, 80% of the drives were unreadable after a few years. I suggested tape, but it was so hard as a grad student to figure out anything about how to even get it set up! Not to mention academic labs in bio departments are notoriously stingy! I found a tape drive in Newegg and took the better part of a year to set it up (learning half baked that I needed this card and that cable etc). And then finally set up the tapes to back up. Its been years since I left the lab and I heard that no one there knows how to read the data back from the tapes! I wish things were better than this! My only anecdotal insight was that hard drives don't fail easily if they are continuously running. So I managed to hoard enough drives to keep at least two copies of data I cared about, and used a program called Mirror folder to make backups that were still easily accessible without any software lock-in by any grad student in the future.
- KaiserPro 6y ago
  Oh yes, the software that runs tape is appalling.
- IanCutress 6y ago
  You can now get tape drives that run over Thunderbolt 3.

Dylan16807 6y ago

> Linear speed 360 megabytes per second for LTO-8. You can consistently match/beat that linear speed with two hard drives, each costing a tenth as much as a tape drive.

dTal 6y ago
I feel this comparison is a little glib, considering that when the hard drives fill up you need to buy two new ones, while the tape drive only needs a new tape.
- skoskie 6y ago
  Fair, but the upfront price of a tape drive is $3-5k. The cost benefit comes down to the equilibrium and how much storage you need.
  - sukilot 6y ago
    Can I take a drive to a copy shop to convert it to tape for $50? (If not, why not?)
    - est31 6y ago
      Pretty sure that there are colocation places out there that have this, but for copy shops I think it wouldn't work because in general, people have too few data. Most people are perfectly served by the HDDs/SSDs that ship with their computers, and the minority that's not served often buys one or two external HDDs of the newest generation. That's enough for the largest part of the population. Fewest private people have data on the order of dozens of TB where HDDs become so expensive that you are motivated to use the user unfriendly tapes. Why are tapes user unfriendly? For each read/write operation you need to visit the copy shop. This limits the use cases to ones where like backups and archival. But you can't really use them for archival as when you don't archive the drive together with your tapes, who guarantees that a copy shop will exist in 10 years that has drives that can read such old tapes? Yes, tape drives have limited read compatibility to older standards. https://en.wikipedia.org/wiki/Linear_Tape-Open#Compatibility https://en.wikipedia.org/wiki/Linear_Tape-Open#Compatibility But maybe there is a niche for it somewhere, who knows.
    - waste_monk 6y ago
      There are a few good reasons why not: 1) An LTO-8 tape (capacity approx 30TB) is around $200 to $300 AUD (depending on brand), so that figure could only work if you're excluding the cost of the tape. LTO-7 (~15TB) costs around $100 to $150 AUD and would probably be more suitable, but it's still a lot of money for what's essentially a consumable item. Also, I would imagine that if the choice was between buying a tape that requires you to go to a third party to use or invest thousands of dollars in a tape drive, or just buying an external HDD you can use at home and unplugging it when not in use (so cryptolockers etc. can't eat your backups), most people would go for the external HDD. Unfortunately LTO-8 drives can't read LTO-6 or earlier (previously the rule of thumb was drives could read/write N -1 and read N - 2 of the drive generation e.g. an LTO-7 could read but not write LTO-5 tapes), and it probably wouldn't make sense to invest in outdated drives capable of writing LTO-6 (~6.25TB) or earlier, even if you could obtain the older LTO-6 or smaller media relatively cheaply. Vendors would not want to support it. 2) Most backup software I'm aware of is designed around recurring jobs (e.g. BackupExec), not one-off jobs, and to store data from a number of backup targets onto a single tape (or set of tapes). You could work around this, but there's no off-the-shelf software for doing so I'm aware of (I know some versions of windows server backup can write direct to tape, but this would have to cope with a variety of source media and also make sure that jobs from different customers are never co-mingled on the same tape, etc.). 3) The cost of the hardware required to achieve a decent turnaround time on the tapes (depending on target time e.g. some customers might pay a premium for same-day conversion) would probably exceed what you're likely to earn selling this service. I doubt it would be financially viable, simply due to the low number of customers.
      - StillBored 6y ago
        "You could work around this, but there's no off-the-shelf software for doing so I'm aware of (I know some versions of windows server backup can write direct to tape," I think what you are looking for is LTFS.
        waste_monk 6y ago
        Not so much the filesystem, more that assuming you have a server with a multitude of our customers HDD's attached as well as one or more tape libraries / drives, you need to: * Map 1 to N HDDs to one customer (by USB port ID or perhaps SATA port if you shuck them for performance reasons, maybe a mix of SATA and USB drives) - will need to have some means of clearly indicating ports, or have blinky LEDs next to them and have the server indicate where stuff ought to be plugged in. And handle SATA hot-swap if required. * Allocate a tape or tapes to them (and if they provide their own tape/s make sure to use specific slot/s in the tape library for that customer) and otherwise handle the tape library logistics. This becomes more complex if you have multiple libraries in play. * Perform scheduling and ETA estimation to see if you'll actually get their data onto the tape within the deadline (you might offer a turnaround of a day, but offer a premium service to have it done within N hours) and see if it will affect other customers. This may even involve deciding that an existing currently running job is early enough that you might stop it and scratch the tapes, then run the priority job instead. * Handle predictive maintenance, drive cleaning, etc. for the tape library (and how it will affect customer deadlines / capacity planning) * Actually copying the data onto the tape/s and verifying the result * Integrate into a monitoring and notification system so customers are alerted when their order is ready to pick up, talk to customer billing/sales systems, etc. * Probably a bunch of other stuff I haven't thought of yet The system would generally have to be resilient enough that it can run unattended overnight / for long periods of time, but flexible enough in scheduling that it can handle sudden rush orders, drives being offline for maintenance, and so on. And ideally could be operated by staff with minimal technical background. Maybe Iron Mountain or someone like that have this sort of system already, but I am not aware of any commercially or generally available software that could do the job.
    - KaiserPro 6y ago
      Thats not where tape it useful. Tape only makes sense when you have tape robot. A bit rack type thing that has more than 5 drives and thousands of slots. Unless you are on a very tight power/heat budget and you have a brilliant asset management/storage system then tape isn't the answer.
- Dylan16807 6y ago
  I don't see how that matters in a discussion of speed. Pick any budget. For that budget, hard drives are faster than tape. Tape's advantages are elsewhere.
StillBored 6y ago
Tapes have become "enterprise" gear.. that is why they are expensive. HD's are looking to replicate those markups if the pricing on WD reds vs the usb drives are any indication. There was a time that a tape drives cost about the same as a floppy/zip/etc drive. AKA a hundred to two bucks. In the early 1990's QIC80 tapes were pretty popular in certain circles as an exchange medium since they fit a hundred or so floppies in a package that was smaller and a little thicker. Later things like the 3080 and the like bumped the capacity into the GB range for similar pricing. Back then pretty much ever novell/etc server in existence had a tape drive in the front being used as a backup mechanism. The sysadmin dropped a tape in on friday, and took it offsite on monday (or whatever). In unix land it was possible to completely single step boot from tape and recover using things like AIX's mksysb. The death of tape, wasn't really a death so much as a giant margin increase, where a drive that cost $200 suddenly cost $2000 and came with a fibre channel interface rather than IDE..
- xbmcuser 6y ago
  Cost per gb tapes are not expensive tape drives on the other hand
KaiserPro 6y ago
LTO-8 is 12tb a 12tb drive is more expensive than LTO. Even at volume. 2*6tb drives is still more expensive. However thats not the use case right, HDDs do general storage really well. A tape robot is the place where your data goes to rest. Its the last line in your tiered storage. In AWS land it goes RAM->EBS->s3->glacier tape is the same stage as glacier. You have RAM cache (be it redis et al, or some fancy block storage) Screaming fast SSDs, but only a few TBs, Slower cheaper SSDs, but ten times as more. Then if you need lots of storage you'd have a few PB of Spinny disks. After that it goes to Tape. Its denser, cooler and far less expensive to run than the equivalent spinny disk array. Now this only makes sense if you have data in the PB range, and its more data than you can store online. Anything less and it doesn't make sense. It never did, its just that the capacity of storage has ballooned so much in the last 15 years that people's perceptions haven't caught up.
- Dylan16807 6y ago
  You're neglecting that the tapes have to go in something. LTO-8 drives are vastly more expensive than 12TB hard drives. A pile of tapes, where the vast majority of the tapes are not in drives, is cheaper than hard drives. But then you're not getting those high speeds anymore. - Let's do some simplified math. 12TB hard drives for $26/TB, LTO-8 tapes for $9/TB, and tape drives for $3500. Since you're saving $17/TB, you need 205TB before tape gets cheaper. So 17 tapes. Each hard drive takes about 18 hours to fill. So 17 hard drives take 18 hours to fill, at a total speed of 3GB/s. Each tape takes about 9 hours to fill. So 17 tapes take six and a half days to fill, at a total speed of 360MB/s. If you have enough tapes per drive to make tape cheaper than hard drives, it's slooowwww. If you have fewer tapes per drive, it can be reasonably fast, but never as fast as hard drives. Tape has a lot of advantages. But not speed.
  - caf 6y ago
    Does the $26/TB for hard drives include the cost of the JBOD chassis you'll need to to have them all online at once, in order to get those speeds?
    - 6y ago

ramraj07 6y ago

My lab acquired a lot of imaging data (10 TB a year) and notoriously would store it long-term by putting the hardrive back in thr grey static plastic bag and putting it in a wooden office drawer. Surprise, 80% of the drives were unreadable after a few years. I suggested tape, but it was so hard as a grad student to figure out anything about how to even get it set up! Not to mention academic labs in bio departments are notoriously stingy! I found a tape drive in Newegg and took the better part of a year to set it up (learning half baked that I needed this card and that cable etc). And then finally set up the tapes to back up. Its been years since I left the lab and I heard that no one there knows how to read the data back from the tapes! I wish things were better than this! My only anecdotal insight was that hard drives don't fail easily if they are continuously running. So I managed to hoard enough drives to keep at least two copies of data I cared about, and used a program called Mirror folder to make backups that were still easily accessible without any software lock-in by any grad student in the future.

KaiserPro 6y ago
Oh yes, the software that runs tape is appalling.
IanCutress 6y ago
You can now get tape drives that run over Thunderbolt 3.