81 ms·

Why? Hard drives are as cheap as tape now: $10/TB. Tapes are sequential access media, wear out, and malfunction, whereas hard drives are random access media. I

Why? Hard drives are as cheap as tape now: $10/TB. Tapes are sequential access media, wear out, and malfunction, whereas hard drives are random access media. I know all about StorageTek silos and 4U tape robots but still don't bother with them.

adrian_b 3y ago
Tape is $5 per TB at retail and you can safely store data on it for at least 10 years, after which you may need to make a new copy not due to the risk of aging, but due to the risk of no longer finding new tape drives compatible with the tape format. Your price of $10/TB might apply when HDDs are purchased in bulk, because at retail I see prices between $15/TB and $20/TB. No HDD may be trusted to store data for more than 5 years and this duration is valid only for the more expensive models. Assuming your price of $10/TB, one must buy at least a double HDD capacity than tape capacity, due to the short lifetime, so tape is at least 4 times cheaper. At the prices that I see at retail the difference is even greater. The sequential transfer speed of tape is greater than that of HDDs, so archiving or retrieving many GB of data takes less time with tapes. HDDs wear out and malfunction more frequently than tapes. The only real disadvantage of tapes is the high cost of the tape drives, which makes tapes preferable only when more than 100 or 200 TB of data have to be stored.
- riobard 3y ago
  Where do you buy $5/TB tape at retail?
  - adrian_b 3y ago
    Amazon, either USA or Germany, 12 TB (real capacity) LTO-8 cartridges for $60 a piece.
    - magicalhippo 3y ago
      Right but from what I can see a LTO-8 drive ain't cheap, even used.
      - adrian_b 3y ago
        Many years ago, when LTO-7 was new and state-of-the-art, I have bought a new tabletop tape drive (Quantum) for $3000, which is less than an Apple HMD, but much more useful for me. After writing several hundred TB, I have achieved a decent money saving in comparison with using HDDs. On the other hand, when someone needs to store only 100 TB or less, there is no chance to recover the cost of the tape drive, so tapes are inappropriate for such a case.
        throwaway2037 3y ago
        After writing several hundred TB Sheesh, what are you writing!?
        adrian_b 3y ago
        With scanned books and Blu-ray movies, the TBs add very quickly when you do not want to degrade their quality with additional compression, which is my case, because after digitizing or ripping them I do not keep the originals, due to not having enough space for them, so I want the retained copies to have a quality as high as possible, which leads to multi-GB file sizes.
        leptons 3y ago
        LTO-5 worked great for my use case and I find new/old-stock LTO5 tapes for about $5/TB on ebay. I verify every tape after writing, and I include about 25% to 30% parity data on each tape to combat bit rot. I don't really need to write more than 1TB at a time of data, and the rest of the room on the tape is parity data (PAR files). The drive was super cheap, $150 used. I don't expect it to last forever, I plan to buy another tape drive as a backup and eventually upgrade to an LTO-6 drive. After making 2 copies of all my important data, about 30TB on LTO-5 tape, I don't have that much to back up, maybe 2TB a month, but it's easy to justify buying a few more tapes every now and then. Buying 2 hard drives for redundancy is just not anywhere near as cheap as buying 2 LTO tapes for the same amount of data, even for people with less than 100TB of data.
- justsomehnguy 3y ago
  > The only real disadvantage of tapes is the high cost of the tape drives The real disadvantage of tapes is what the drive may die at any time and if you don't have another drive then you can't recover. And a replacement drive wouldn't be cheap (new) or reliable (used).
  - adrian_b 3y ago
    While this is true, HDDs are much worse, because the probability that a HDD will die at any time and without warnings is many times higher than for any tape drive. When a HDDs dies, that is far worse than when a tape drive dies, because you not only lose some money, but you also lose data, which may be priceless, unless you have been careful to have backup copies. While there are some companies that offer data recovery services from defective HDDs, for recent HDD models such services can be very expensive, comparable with the cost of a tape drive and much more expensive than the service of copying a good tape on a HDD, which can be done when it is not possible to buy a replacement drive immediately and the data is needed urgently.
    - justsomehnguy 3y ago
      > HDD will die at any time and without warnings is many times higher than for any tape drive It's same most of the time and LTO drive dies if you have at least any amount of dust while HDD doesn't give a fuck > but you also lose data, which may be priceless No difference if you sat on the tape with yours only one copy of your favourite porn. And if you only have one copy of data then it doesn't really matter on which media it resides. And oh, you CAN write the same data to two HDDs simultaneously with mirroring or just having a two copy jobs to a two separate hard drives, which would not only give you a physical separation but logical as well. For you to do the same with the tape - shell another $3000.
  - leptons 3y ago
    The drives are relatively cheap. I picked up an LTO-5 drive for about $150. I don't care that it's used. At that price I can buy 3 of them so that I have spares. And the next gen tape drive will always read the previous gen tapes. So it's an easy upgrade path. I'll go to LTO-6 when those drives come down a little in price on the used market. Then LTO-7 if I'm even alive lone enough to need that much storage. Realistically, I have about 30-40 years left in me and after that I don't really care what happens to my backups. They're encrypted, with parity files, so it's not unreasonable to think that these backups will last me the rest of my life.
- in_vestor 3y ago
  Where do you get the idea that HDDs only retain data for 5 years? The physics I learned in college suggests they will retain magnetic domains for hundreds or even thousands of years. Yes they can fail mechanically (is this what you mean?) but you don’t necessarily lose your data.
  - adrian_b 3y ago
    They will fail mostly mechanically, but also the very small magnetized bits of modern HDDs will flip spontaneously at normal temperature after a much shorter time than "hundreds or even thousands of years" (because the energy needed to flip a small bit is not large enough in comparison to the thermal fluctuations to make the flipping probability negligible). Many of these bit flips, but not all, will be corrected when the sectors are read, due to the error-correcting codes that are used in HDDs. This is not theory, I have stored data for several years on more than 60 HDDs of various capacities from both WD and Seagate, most of them being the more expensive models with extended warranty durations, but even so, only few of the HDDs did not have any non-correctable error after several years. (Fortunately I was careful to use redundancy, so there was no data loss.) Moreover, some of the biggest HDDs that are available now are no longer suitable for long term data storage, because in order to improve the performance they store metadata in a flash memory, which has a more limited data retention time. After more than 5 years the complete loss of a HDD should be expected at any time, but even after 2 or 3 years a few non-correctable errors are probable. When a HDD fails mechanically, one might pay a data recovery service, but that might have a price similar to a new HDD, so if you plan to not replace your HDDs often enough with the hope of using data recovery, it is pretty much certain that the cost will be much higher than replacing any HDD preemptively when its warranty expires.
    - in_vestor 3y ago
      Have you truly observed this type of degradation? I do archive work and have 20+ discs from the 2010 era. Mostly the first generation of PMR drives. I have never had any data degradation problems. You can also find lots of YouTube videos of people spinning up drives from the 80s and 90s which still hold their data without problem. More scientifically, the phenomenon you talk about is modeled by the Arrhenius equation (1), where the activation energy to flip a grain is given by KuV/KbT, where Ku is the anisotropy of the magnetic media, V is the volume of a grain, Kb is the Boltzmann constant, and T is temp in Kelvin. HDD manufacturers engineer this ratio to be >60 (usually targeting 70-90 to be safe). Media manufacturing is imperfect, so there is a log normal distribution of grains on real-world media, but if we assume that 60 is the energy barrier for all grains, a KuV/KbT of 60 would mean it takes 362 million years for half the grains to flip, assuming an attempt frequency of 10^10. Where is my math wrong? (1) https://en.m.wikipedia.org/wiki/Arrhenius_equation https://en.m.wikipedia.org/wiki/Arrhenius_equation

adrian_b 3y ago

Tape is $5 per TB at retail and you can safely store data on it for at least 10 years, after which you may need to make a new copy not due to the risk of aging, but due to the risk of no longer finding new tape drives compatible with the tape format. Your price of $10/TB might apply when HDDs are purchased in bulk, because at retail I see prices between $15/TB and $20/TB. No HDD may be trusted to store data for more than 5 years and this duration is valid only for the more expensive models. Assuming your price of $10/TB, one must buy at least a double HDD capacity than tape capacity, due to the short lifetime, so tape is at least 4 times cheaper. At the prices that I see at retail the difference is even greater. The sequential transfer speed of tape is greater than that of HDDs, so archiving or retrieving many GB of data takes less time with tapes. HDDs wear out and malfunction more frequently than tapes. The only real disadvantage of tapes is the high cost of the tape drives, which makes tapes preferable only when more than 100 or 200 TB of data have to be stored.

riobard 3y ago
Where do you buy $5/TB tape at retail?
- adrian_b 3y ago
  Amazon, either USA or Germany, 12 TB (real capacity) LTO-8 cartridges for $60 a piece.
  - magicalhippo 3y ago
    Right but from what I can see a LTO-8 drive ain't cheap, even used.
    - adrian_b 3y ago
      Many years ago, when LTO-7 was new and state-of-the-art, I have bought a new tabletop tape drive (Quantum) for $3000, which is less than an Apple HMD, but much more useful for me. After writing several hundred TB, I have achieved a decent money saving in comparison with using HDDs. On the other hand, when someone needs to store only 100 TB or less, there is no chance to recover the cost of the tape drive, so tapes are inappropriate for such a case.
      - throwaway2037 3y ago
        After writing several hundred TB Sheesh, what are you writing!?
        adrian_b 3y ago
        With scanned books and Blu-ray movies, the TBs add very quickly when you do not want to degrade their quality with additional compression, which is my case, because after digitizing or ripping them I do not keep the originals, due to not having enough space for them, so I want the retained copies to have a quality as high as possible, which leads to multi-GB file sizes.
      - leptons 3y ago
        LTO-5 worked great for my use case and I find new/old-stock LTO5 tapes for about $5/TB on ebay. I verify every tape after writing, and I include about 25% to 30% parity data on each tape to combat bit rot. I don't really need to write more than 1TB at a time of data, and the rest of the room on the tape is parity data (PAR files). The drive was super cheap, $150 used. I don't expect it to last forever, I plan to buy another tape drive as a backup and eventually upgrade to an LTO-6 drive. After making 2 copies of all my important data, about 30TB on LTO-5 tape, I don't have that much to back up, maybe 2TB a month, but it's easy to justify buying a few more tapes every now and then. Buying 2 hard drives for redundancy is just not anywhere near as cheap as buying 2 LTO tapes for the same amount of data, even for people with less than 100TB of data.
justsomehnguy 3y ago
> The only real disadvantage of tapes is the high cost of the tape drives The real disadvantage of tapes is what the drive may die at any time and if you don't have another drive then you can't recover. And a replacement drive wouldn't be cheap (new) or reliable (used).
- adrian_b 3y ago
  While this is true, HDDs are much worse, because the probability that a HDD will die at any time and without warnings is many times higher than for any tape drive. When a HDDs dies, that is far worse than when a tape drive dies, because you not only lose some money, but you also lose data, which may be priceless, unless you have been careful to have backup copies. While there are some companies that offer data recovery services from defective HDDs, for recent HDD models such services can be very expensive, comparable with the cost of a tape drive and much more expensive than the service of copying a good tape on a HDD, which can be done when it is not possible to buy a replacement drive immediately and the data is needed urgently.
  - justsomehnguy 3y ago
    > HDD will die at any time and without warnings is many times higher than for any tape drive It's same most of the time and LTO drive dies if you have at least any amount of dust while HDD doesn't give a fuck > but you also lose data, which may be priceless No difference if you sat on the tape with yours only one copy of your favourite porn. And if you only have one copy of data then it doesn't really matter on which media it resides. And oh, you CAN write the same data to two HDDs simultaneously with mirroring or just having a two copy jobs to a two separate hard drives, which would not only give you a physical separation but logical as well. For you to do the same with the tape - shell another $3000.
- leptons 3y ago
  The drives are relatively cheap. I picked up an LTO-5 drive for about $150. I don't care that it's used. At that price I can buy 3 of them so that I have spares. And the next gen tape drive will always read the previous gen tapes. So it's an easy upgrade path. I'll go to LTO-6 when those drives come down a little in price on the used market. Then LTO-7 if I'm even alive lone enough to need that much storage. Realistically, I have about 30-40 years left in me and after that I don't really care what happens to my backups. They're encrypted, with parity files, so it's not unreasonable to think that these backups will last me the rest of my life.
in_vestor 3y ago
Where do you get the idea that HDDs only retain data for 5 years? The physics I learned in college suggests they will retain magnetic domains for hundreds or even thousands of years. Yes they can fail mechanically (is this what you mean?) but you don’t necessarily lose your data.
- adrian_b 3y ago
  They will fail mostly mechanically, but also the very small magnetized bits of modern HDDs will flip spontaneously at normal temperature after a much shorter time than "hundreds or even thousands of years" (because the energy needed to flip a small bit is not large enough in comparison to the thermal fluctuations to make the flipping probability negligible). Many of these bit flips, but not all, will be corrected when the sectors are read, due to the error-correcting codes that are used in HDDs. This is not theory, I have stored data for several years on more than 60 HDDs of various capacities from both WD and Seagate, most of them being the more expensive models with extended warranty durations, but even so, only few of the HDDs did not have any non-correctable error after several years. (Fortunately I was careful to use redundancy, so there was no data loss.) Moreover, some of the biggest HDDs that are available now are no longer suitable for long term data storage, because in order to improve the performance they store metadata in a flash memory, which has a more limited data retention time. After more than 5 years the complete loss of a HDD should be expected at any time, but even after 2 or 3 years a few non-correctable errors are probable. When a HDD fails mechanically, one might pay a data recovery service, but that might have a price similar to a new HDD, so if you plan to not replace your HDDs often enough with the hope of using data recovery, it is pretty much certain that the cost will be much higher than replacing any HDD preemptively when its warranty expires.
  - in_vestor 3y ago
    Have you truly observed this type of degradation? I do archive work and have 20+ discs from the 2010 era. Mostly the first generation of PMR drives. I have never had any data degradation problems. You can also find lots of YouTube videos of people spinning up drives from the 80s and 90s which still hold their data without problem. More scientifically, the phenomenon you talk about is modeled by the Arrhenius equation (1), where the activation energy to flip a grain is given by KuV/KbT, where Ku is the anisotropy of the magnetic media, V is the volume of a grain, Kb is the Boltzmann constant, and T is temp in Kelvin. HDD manufacturers engineer this ratio to be >60 (usually targeting 70-90 to be safe). Media manufacturing is imperfect, so there is a log normal distribution of grains on real-world media, but if we assume that 60 is the energy barrier for all grains, a KuV/KbT of 60 would mean it takes 362 million years for half the grains to flip, assuming an attempt frequency of 10^10. Where is my math wrong? (1) https://en.m.wikipedia.org/wiki/Arrhenius_equation https://en.m.wikipedia.org/wiki/Arrhenius_equation