15 ms·

People who don't use and who don't understand the differences between tape and disk have been saying tape is dying for years. The most common incorrect asserti

People who don't use and who don't understand the differences between tape and disk have been saying tape is dying for years. The most common incorrect assertion is that drives don't fail THAT often, so why not just back up to drives and put them on a shelf? But to be generally safe, all data would have to be on at least two drives, which changes the cost noticeably. Older, used LTO drives are actually quite affordable, so I use LTO-4 and LTO-5 tapes to back up all sorts of things for both myself and for my clients. It's surprisingly easy and inexpensive, and pulling files back off is as simple as running pax.

hotpotamus 3y ago
I remember a gmail outage circa 2008 where they said they were restoring from tape, and people found the idea comically retrograde even back then. But I started at a company with large Commvault tape archives in 2011 and they mentioned that it was just more economical at scale for the workload (the backup team had some sweet gear that was apparently purchased with a kickback program on their LTO(4?) tape purchases; perhaps that assisted the engineering decisions) . However, by the end of that decade, they had migrated to mostly using large drive arrays with tape used only for offsite, so my impression was that tape was indeed a shrinking, niche method of data storage.
- throwaway29303 3y ago
  An interesting Google Tech Talk video from 2013 about Google backing up the internet (and gmail): https://www.youtube.com/watch?v=eNliOm9NtCM https://www.youtube.com/watch?v=eNliOm9NtCM
  - tpmx 3y ago
    Fantastic talk, and I'm just 12 minutes in on a 1h14m video. Take the Cartesian product, in fact, of those factors. I want location isolation, isolation from application layer problems, isolation from storage layer problems, isolation from media failure. Edit: They do RAID4 on the tape level. 5 tapes, one of them the parity tape. When that fails: Reconstruction at sub-tape level. "We don't really have data loss." "We make the backups as complicated and take as long as they need. The restores have to be quick and automatic. Recovery should be stupid, fast and simple."
bluGill 3y ago
15 years ago I worked for a tape backup company that got bought out by a company making disk backups. Before they bought us out the CEO was constantly saying "tape is dead" in interviews, but once there was money for him to make in tape he stopped saying that. I remember when someone asking him about this and he responded something "amazing how money can make you change your tune"
- sdifognioqnio 3y ago
  [dead]
- anthlax 3y ago
  W transparency
- hinkley 3y ago
  > It is difficult to get a man to understand something when his salary depends upon his not understanding it.
achow 3y ago
Counter: Retrieval (recovery) from disk is easier and faster - ability to randomly read and write. Tapes are sequential - to get to a data which is only at the end, one has to go through all the tapes that is there before that.
- Waterluvian 3y ago
  What is the value of that? Is that a common use case for those buying exabytes of tape?
- sadhorse 3y ago
  The disk also has to move its head to the location of the data. It is faster, but still sequential.
- Karunamon 3y ago
  That detail is abstracted away from the user though, tapes north of LTO 5 (which is pretty old) allow addressing tapes like a normal file system, and if you are doing backup things, the contents of the tape are held separately from the data itself. As an off-line archival backup medium, access times really are not a concern in the real world.
- fh973 3y ago
  Tapes are not strictly sequential, they are also a 2D medium, admittedly with one sequential dimension: https://en.wikipedia.org/wiki/Linear_Tape-Open#Physical_structure https://en.wikipedia.org/wiki/Linear_Tape-Open#Physical_stru... https://en.wikipedia.org/wiki/Linear_Tape-Open#Positioning_times https://en.wikipedia.org/wiki/Linear_Tape-Open#Positioning_t...
- adrian_b 3y ago
  The maximum tape seek time is around a couple of minutes. The average seek time is about half of that, i.e. around 1 minute (these example values are valid for LTO-7). These seek times are seldom a problem. If one would keep on HDDs as much data as it is normally kept on tapes, i.e. at least a few hundred terabytes, one would have the same slow seek times as someone would have to plug and unplug external HDDs, unless the storage would belong to a very large company that could afford the high costs of keeping online many HDDs. As an individual user, I keep in my computer, on an SSD, an index with the content of all tapes. When I need some file, I get it in at most 5 minutes, which includes starting the tape drive, going to a cabinet and taking the tape from a shelf, inserting it in the drive and waiting for the seek and copy operations. This is perfectly acceptable, because I do not need files from the tapes every day. Storing the data on external HDDs would not reduce the time wasted with manual operations, it would be much less reliable and it would be much more expensive for great amounts of data. The sequential transfer speed of tapes is greater than that of HDDs. Therefore, if after a seek you copy large files, e.g. a movie in BluRay format, the whole seek + copy operation takes less than from a HDD.
bityard 3y ago
It depends on the scale, doesn't it? My issue with tape is that unless your operation is big enough to justify a robotic tape library as a budget line item with support contracts and all, then you're down to paying someone (or eating the cost yourself) to physically swap tapes which is much more expensive and boring than deploying a ZFS server with gobs of disk that once configured, just sits in the rack and does its job quietly.
- Karunamon 3y ago
  More realistically you are deploying two of the servers; redundancy is not backup. And now you have two servers to administer. The whole point of something like tape is having an off-line copy of your data, ideally in a separate physical location. A second server with a bunch of disk in the same location connected to the same network isn't that, and will not save you from ransomware or natural disaster.
  - deleted 3y ago
    [deleted]
  - rbanffy 3y ago
    > And now you have two servers to administer. If you use even crude tools such as clusterssh, managing a bunch of machines isn’t linearly harder than managing one. While tape renders itself easily to be offline storage (it’s offline as soon as you eject it after all), you can have that with remote servers that pull data to back it up instead of receiving pushed data. If no server can push data to any other server, only pull from them, a ransomware attack becomes a lot harder. Also, while tape in a warehouse (or on the desk) is offline, tapes in the robot are no more difficult to destroy than hard disks. They are just slower.
- eschneider 3y ago
  Yeah, but that ZFS server is likely in the same rack that's going to get knocked out when the data center gets sucked into a tornado, or whatever. Weather it makes sense to have someone on staff swapping tapes, rotating backups, shipping them offsite, and keeping records of what exactly is where not to mention handling AND TESTING restores really depends on what your data's worth. For most places, it's cheap insurance.
ramraj07 3y ago
One important thing is (at least a few years back) we realised that HDDs fail at an alarming rate (> 80%) if you disconnect them and leave them for a few years. Either that or someone brought a gamma ray source through our lab. Keeping the disks hot was a different story though.
- sgerenser 3y ago
  Interesting, I've heard of issues with poor shelf-life for SSDs, but not HDDs. Any data/articles on this topic or is it strictly anecdotal?
  - hinkley 3y ago
    Bearings seizing up is the example I usually hear.
    - rbanffy 3y ago
      Most revive after being left in the fridge for some time. The bits and pieces contract at different rates, so cooling (or, I must assume, heating) is enough to get them unstuck. Many drives I did that with lived happy afterlives after being revived, some having been brought back from the dead more than once.
  - adrian_b 3y ago
    I am not aware of any large-scale study, but poor HDD reliability in cold storage corresponds to my experience too. I have archived data (almost a couple hundred TB in double copies) on more than 60 HDDs, about half from WD and half from Seagate, from several HDD generations, i.e. with capacities of 2 TB, 3 TB, 4 TB, 6 TB and 8 TB, most being from the 4 up to 8 TB generations. All HDDs were more expensive WD and Seagate models (i.e. with longer warranty times), not their cheapest consumer HDDs. All files were checksummed for error detection. When the HDD content was transferred to tapes after some 3 to 6 years since the initial archival, almost all HDDs had a few errors, which sometimes were not reported by the drives even when they were detected by the file checksums. Only the fact that all files were stored on at least 2 HDDs has prevented data loss, because even when both HDDs had errors they were in different locations.
Fgehono 3y ago
I'm pretty sure big cloud companies will keep using tape for a long time. For a home/small business it's still too much effort and expensive. I tried revisiting this topic so often and HDD was always so much cheaper for just 5-30tb which is quite a lot for quite a lot of companies.
- kevin_thibedeau 3y ago
  Used older generation LTO drives are cheap.
epilys 3y ago
> Older, used LTO drives are actually quite affordable They are, but If I'm not mistaken you need a SAS controller, and those go for ~1k euros in my area, even used.
- bombcar 3y ago
  You can often find compatible SAS controllers used on eBay for quite cheap - often flashing a “raid” controller into a simpler SAS controller. Fiber channel drives also exist.
reaperducer 3y ago
I use LTO-4 and LTO-5 tapes to back up all sorts of things for both myself and for my clients My archive is getting to the size that optical discs are becoming cumbersome. Do you know of a tape solution that works with macOS?
sterlind 3y ago
I have wanted to use a tape drive for years, for storing backups. Partly because tapes are cheap and numerous, partly for the aesthetic, honestly. LTO-5 is indeed quite cheap now, you've inspired me to give it a go.
jszymborski 3y ago
Every few months I go to check out prices for Ultrium Tape Drives and I've had a super hard time finding affordable ones (everything is almost $1K). Any tips? Particularly on finding something I can interface with SATA?

hotpotamus 3y ago

I remember a gmail outage circa 2008 where they said they were restoring from tape, and people found the idea comically retrograde even back then. But I started at a company with large Commvault tape archives in 2011 and they mentioned that it was just more economical at scale for the workload (the backup team had some sweet gear that was apparently purchased with a kickback program on their LTO(4?) tape purchases; perhaps that assisted the engineering decisions) . However, by the end of that decade, they had migrated to mostly using large drive arrays with tape used only for offsite, so my impression was that tape was indeed a shrinking, niche method of data storage.

throwaway29303 3y ago
An interesting Google Tech Talk video from 2013 about Google backing up the internet (and gmail): https://www.youtube.com/watch?v=eNliOm9NtCM https://www.youtube.com/watch?v=eNliOm9NtCM
- tpmx 3y ago
  Fantastic talk, and I'm just 12 minutes in on a 1h14m video. Take the Cartesian product, in fact, of those factors. I want location isolation, isolation from application layer problems, isolation from storage layer problems, isolation from media failure. Edit: They do RAID4 on the tape level. 5 tapes, one of them the parity tape. When that fails: Reconstruction at sub-tape level. "We don't really have data loss." "We make the backups as complicated and take as long as they need. The restores have to be quick and automatic. Recovery should be stupid, fast and simple."

bluGill 3y ago

15 years ago I worked for a tape backup company that got bought out by a company making disk backups. Before they bought us out the CEO was constantly saying "tape is dead" in interviews, but once there was money for him to make in tape he stopped saying that. I remember when someone asking him about this and he responded something "amazing how money can make you change your tune"

sdifognioqnio 3y ago
[dead]
anthlax 3y ago
W transparency
hinkley 3y ago
> It is difficult to get a man to understand something when his salary depends upon his not understanding it.

achow 3y ago

Counter: Retrieval (recovery) from disk is easier and faster - ability to randomly read and write. Tapes are sequential - to get to a data which is only at the end, one has to go through all the tapes that is there before that.

Waterluvian 3y ago
What is the value of that? Is that a common use case for those buying exabytes of tape?
sadhorse 3y ago
The disk also has to move its head to the location of the data. It is faster, but still sequential.
Karunamon 3y ago
That detail is abstracted away from the user though, tapes north of LTO 5 (which is pretty old) allow addressing tapes like a normal file system, and if you are doing backup things, the contents of the tape are held separately from the data itself. As an off-line archival backup medium, access times really are not a concern in the real world.
fh973 3y ago
Tapes are not strictly sequential, they are also a 2D medium, admittedly with one sequential dimension: https://en.wikipedia.org/wiki/Linear_Tape-Open#Physical_structure https://en.wikipedia.org/wiki/Linear_Tape-Open#Physical_stru... https://en.wikipedia.org/wiki/Linear_Tape-Open#Positioning_times https://en.wikipedia.org/wiki/Linear_Tape-Open#Positioning_t...
adrian_b 3y ago
The maximum tape seek time is around a couple of minutes. The average seek time is about half of that, i.e. around 1 minute (these example values are valid for LTO-7). These seek times are seldom a problem. If one would keep on HDDs as much data as it is normally kept on tapes, i.e. at least a few hundred terabytes, one would have the same slow seek times as someone would have to plug and unplug external HDDs, unless the storage would belong to a very large company that could afford the high costs of keeping online many HDDs. As an individual user, I keep in my computer, on an SSD, an index with the content of all tapes. When I need some file, I get it in at most 5 minutes, which includes starting the tape drive, going to a cabinet and taking the tape from a shelf, inserting it in the drive and waiting for the seek and copy operations. This is perfectly acceptable, because I do not need files from the tapes every day. Storing the data on external HDDs would not reduce the time wasted with manual operations, it would be much less reliable and it would be much more expensive for great amounts of data. The sequential transfer speed of tapes is greater than that of HDDs. Therefore, if after a seek you copy large files, e.g. a movie in BluRay format, the whole seek + copy operation takes less than from a HDD.

bityard 3y ago

It depends on the scale, doesn't it? My issue with tape is that unless your operation is big enough to justify a robotic tape library as a budget line item with support contracts and all, then you're down to paying someone (or eating the cost yourself) to physically swap tapes which is much more expensive and boring than deploying a ZFS server with gobs of disk that once configured, just sits in the rack and does its job quietly.

Karunamon 3y ago
More realistically you are deploying two of the servers; redundancy is not backup. And now you have two servers to administer. The whole point of something like tape is having an off-line copy of your data, ideally in a separate physical location. A second server with a bunch of disk in the same location connected to the same network isn't that, and will not save you from ransomware or natural disaster.
- deleted 3y ago
  [deleted]
- rbanffy 3y ago
  > And now you have two servers to administer. If you use even crude tools such as clusterssh, managing a bunch of machines isn’t linearly harder than managing one. While tape renders itself easily to be offline storage (it’s offline as soon as you eject it after all), you can have that with remote servers that pull data to back it up instead of receiving pushed data. If no server can push data to any other server, only pull from them, a ransomware attack becomes a lot harder. Also, while tape in a warehouse (or on the desk) is offline, tapes in the robot are no more difficult to destroy than hard disks. They are just slower.
eschneider 3y ago
Yeah, but that ZFS server is likely in the same rack that's going to get knocked out when the data center gets sucked into a tornado, or whatever. Weather it makes sense to have someone on staff swapping tapes, rotating backups, shipping them offsite, and keeping records of what exactly is where not to mention handling AND TESTING restores really depends on what your data's worth. For most places, it's cheap insurance.

ramraj07 3y ago

One important thing is (at least a few years back) we realised that HDDs fail at an alarming rate (> 80%) if you disconnect them and leave them for a few years. Either that or someone brought a gamma ray source through our lab. Keeping the disks hot was a different story though.

sgerenser 3y ago
Interesting, I've heard of issues with poor shelf-life for SSDs, but not HDDs. Any data/articles on this topic or is it strictly anecdotal?
- hinkley 3y ago
  Bearings seizing up is the example I usually hear.
  - rbanffy 3y ago
    Most revive after being left in the fridge for some time. The bits and pieces contract at different rates, so cooling (or, I must assume, heating) is enough to get them unstuck. Many drives I did that with lived happy afterlives after being revived, some having been brought back from the dead more than once.
- adrian_b 3y ago
  I am not aware of any large-scale study, but poor HDD reliability in cold storage corresponds to my experience too. I have archived data (almost a couple hundred TB in double copies) on more than 60 HDDs, about half from WD and half from Seagate, from several HDD generations, i.e. with capacities of 2 TB, 3 TB, 4 TB, 6 TB and 8 TB, most being from the 4 up to 8 TB generations. All HDDs were more expensive WD and Seagate models (i.e. with longer warranty times), not their cheapest consumer HDDs. All files were checksummed for error detection. When the HDD content was transferred to tapes after some 3 to 6 years since the initial archival, almost all HDDs had a few errors, which sometimes were not reported by the drives even when they were detected by the file checksums. Only the fact that all files were stored on at least 2 HDDs has prevented data loss, because even when both HDDs had errors they were in different locations.

Fgehono 3y ago

I'm pretty sure big cloud companies will keep using tape for a long time. For a home/small business it's still too much effort and expensive. I tried revisiting this topic so often and HDD was always so much cheaper for just 5-30tb which is quite a lot for quite a lot of companies.

kevin_thibedeau 3y ago
Used older generation LTO drives are cheap.

epilys 3y ago

> Older, used LTO drives are actually quite affordable They are, but If I'm not mistaken you need a SAS controller, and those go for ~1k euros in my area, even used.

bombcar 3y ago
You can often find compatible SAS controllers used on eBay for quite cheap - often flashing a “raid” controller into a simpler SAS controller. Fiber channel drives also exist.

reaperducer 3y ago

I use LTO-4 and LTO-5 tapes to back up all sorts of things for both myself and for my clients My archive is getting to the size that optical discs are becoming cumbersome. Do you know of a tape solution that works with macOS?

sterlind 3y ago

I have wanted to use a tape drive for years, for storing backups. Partly because tapes are cheap and numerous, partly for the aesthetic, honestly. LTO-5 is indeed quite cheap now, you've inspired me to give it a go.

jszymborski 3y ago

Every few months I go to check out prices for Ultrium Tape Drives and I've had a super hard time finding affordable ones (everything is almost $1K). Any tips? Particularly on finding something I can interface with SATA?