21 ms·

Tape still beats SSDs and hard drives when it comes to price per byte

230 points by jaytaylor 7y ago

abalone 7y ago
Did anyone ever figure out what Amazon Glacier uses? They only publicly talk about using custom software to "optimize the sequence of inputs and outputs"...[1] [1] https://aws.amazon.com/glacier/faqs/ https://aws.amazon.com/glacier/faqs/
- cobookman 7y ago
  Note cloud vendors get the benefit of mixing high IOPs workloads with low IOPs workloads. Say a cloud vendor needs a bunch of HDD for a database workload with high IOPs. Using modern disks (4+ TiB) this leaves a lot of stranded disk storage that can only be used for low IOPs workloads. Archive storage uniquely meets the criteria of lots of storage, yet low IOPs. Disclaimer work for Google but have no knowledge of how AWS glacier works.
  - ggm 7y ago
    I could believe that we have a lot of unused space which is amenable to a different overlay use, but the potential for mis-design is huge here. If I pay high IOPs cost and you have to replace my high IOPS data backend on my paypoint, migrating the low paypoint secondary customer is non-trivial if not designed in day one (I am sure you did: cheap players might not understand the risk exposure here) The telco equivalent was ISDN, where a single linecard with two sub-rate ports might be sold to a bank, and a domestic user. The bank could (and would) demand card reset 24/7 to restore service to their ATM, the domestic user simply lost carrier, and game state.
- colanderman 7y ago
  Rumor has it Blu-Ray (specifically BDXL): https://storagemojo.com/2014/04/25/amazons-glacier-secret-bdxl/ https://storagemojo.com/2014/04/25/amazons-glacier-secret-bd... Though a comment on that page mentions this HN comment claiming custom low-speed HDDs: https://news.ycombinator.com/item?id=4416065 https://news.ycombinator.com/item?id=4416065
  - sshagent 7y ago
    I certainly recall a Facebook video showcasing their bluray archival system. I was seriously impressed by the whole setup. And here we are archiving to LTO like primitives :)
    - jolmg 7y ago
      Even with BDXL, which seems to be able to store up to 128GB per disc, you'd have to stack 100 of them to store the 12TB that an LTO-8 tape cartridge can store raw/natively. Tape cartridges also seem to be more physically durable than optical discs. This doesn't mean that they seem simply superior, though. I imagine optical discs have way better seek times, for example, even if read/write speeds are better with tape.
- milesward 7y ago
  :) I know :)
- etaioinshrdlu 7y ago
  Rumor (from this site) consensus seems to be custom low RPM hard drives optimized for long life.
basicplus2 7y ago
"As for transfer speeds, they can reach 300MBps" I am getting over 500MBps on my setup
- jaytaylor 7y ago
  Impressive. Will you share more information about the specific details (i.e. hardware and configuration) for the set up you're running with?
  - merb 7y ago
    well probably with compression
    - jolmg 7y ago
      I wonder if the 300MBps figure might also be with compression.
      - adrian_b 7y ago
        No, the LTO-7 transfer speed is 3 Gb/s a.k.a. 300 MB/s without compression. The compression in drive may be useful for automated backups, but if you use the tape for an archive of files selected by yourself, it is better to disable the drive compression. If you have compressible files, you can compress them with better algorithms on the computer, before storing them on the tape.
        merb 7y ago
        > you can compress them with better algorithms on the computer is that always true? what if I have dozen's of pdf's which are similar and compressed (most pdfs are)? wouldn't de-duplication also reduce the size? I mean with millions/billions/trillions of files of the same format.
        adrian_b 7y ago
jolmg 7y ago
> Even if LTO-8 tapes are now in stock, you can buy cheap LTO-7, I wonder if this was a typo. Did they mean that LTO-8 tapes are out of stock? Otherwise, the sentence structure "even if ... are now in stock, you can buy ..." is kind of weird. > reformat them to M8 and get 9TB of native storage (22.5TB compressed). What does this mean? Doesn't how much it compresses depend on the data not the storage device?
- bengerbil 7y ago
  There was a dispute involving LTO-8 that just recently was resolved, so LTO-8 tape should now be available in the US. I wasn't following it closely, so I can't give any real insight, but it explains the wording.
  - mycall 7y ago
    3TB for $153. Not impressed. https://www.google.com/shopping/product/10502190238469223139 https://www.google.com/shopping/product/10502190238469223139
    - SethTro 7y ago
      I think it's 12TB Google says "HP LTO-8 Ultrium, 30 TB, RW Data Cartridge Tape. Exhaustively tested, HPE LTO Ultrium cartridges meet all your demands for maximum reliability when restoring data, offering high storage density, ease of management and scalable storage and backup performance." TapeandMedia.com says "1 - HPE LTO Ultrium 8 Tape with a capacity of 12 TB and up to 30 TB compressed capacity."
    - jolmg 7y ago
      That says 30TB. Here's a 15TB compressed for $63: https://www.amazon.com/HP-C7977A-Cartridge-Compressed-Capacity/dp/B01NCB4ZZP/ref=mp_s_a_1_10?keywords=lto7+tape&qid=1569376299&s=gateway&sprefix=lto&sr=8-10 https://www.amazon.com/HP-C7977A-Cartridge-Compressed-Capaci...
      - tw04 7y ago
jaytaylor 7y ago
I'd be curious to hear from any folks who actually run tape drive setups for personal use. Where did you get the gear? How cost-effective is the overall setup compared to spinning rust? How is driver support in popular operating systems? Is there more you'd recommend taking into consideration or learning before jumping in? Especially beyond the obvious realities of extremely high latency, of course ;)
- netrap 7y ago
  eBay. 1-1 HDD backups is probably close in price even in the used tape market. I use LTO-4 with FC. Works fine in Windows with Retrospect. I'd expect it to work in Linux as well. The hard part is backup software. I guess with LTO-5+ you can just directly access the tapes with LTFS, but I haven't tried that yet.
  - mc32 7y ago
    Anything which supports ndmp should be good. Both bacula and Amanda among other SW support NDMP. I mean of course enterprise solutions support it too.
- klodolph 7y ago
  > Where did you get the gear? I never ended up using it personally, but when I was considering it, used gear on eBay was top of the list. > How cost-effective is the overall setup compared to spinning rust? For personal use, it is not even remotely cost-effective unless you are a data hoarder. The cost of the drive is simply too high. Also keep in mind that the drive can become damaged or need servicing. It only makes sense if you are dealing with e.g. large quantities of video. The article cites a minimum 140TB where tape becomes cheaper but this involves some dubious decision-making. > How is driver support in popular operating systems? I have only ever used tape with Linux + custom software so I can’t speak of software support in general. > Is there more you'd recommend taking into consideration or learning before jumping in? Especially beyond the obvious realities of extremely high latency, of course ;) The obvious reality to me here is that it just doesn’t make sense unless you’re in spitting distance of a petabyte. The only thing I can imagine someone doing personally is video, and it would have to be a lot of video like some kind of long-running documentary project that you are producing in your spare time. The other factor is to be more paranoid about mixing tape generations than the vendors suggest. Ignore the specs and stick to same-generation media and drives. Efficiently writing to a tape means filling e.g. a 20 MB/s write channel for LTO-6, or 750 MB/s for LTO-8. Can you sustain 750 MB/s? It’s not so easy in practice. You throw away tape capacity if you get buffer underruns.
robjan 7y ago
Another crucial importance is bytes per square centimetre. When archiving something, there's no point in archiving hundreds of storage controllers with it especially if you need to store them in a safe (which you probably should). MicroSD cards may beat tape but they are a lot slower and harder to label.
- koolba 7y ago
  Which one wins on long term storage durability, shock proof, and water damage?
  - blotter_paper 7y ago
    https://en.wikipedia.org/wiki/M-DISC https://en.wikipedia.org/wiki/M-DISC
  - bt848 7y ago
    LTO doesn't suffer from those things. Shock can destroy the case and dislodge the leader pin of an LTO cartridge but AFAIK it can't damage the data on the tape. Water is bad for the transport but as long as you dry your tapes out they should still work. The operating specs for LTO are crazy, you can read and write them under all practical atmospheric conditions.
    - mkj 7y ago
      Smoke haze will cause all kinds of trouble for tape drives though.
      - zaarn 7y ago
        So don't run the backups when the DC is on fire, got it.
      - bacon_waffle 7y ago
        As will very dry conditions. The IceCube neutrino telescope at South Pole used LTO-3 tapes to store data until they switched to only hard drives around 5 years ago. At that point, the newer tape drives had been tried (IIRC LTO-5 was the industry standard by then) but they never seemed to work, presumably because the humidity on station is incredibly low. It was considered more reasonable to keep the old LTO-3 equipment in use, versus building a humidor for the tapes and drives.
techntoke 7y ago
Price per byte maybe, but in value you are sacrificing a lot. I prefer continuous backups and versioned storage.
- viraptor 7y ago
  You're not sacrificing anything. Or specifically having a long term, offline tape storage does not prevent you from having short term, online, versioned file/block storage. Different use cases.
  - techntoke 7y ago
    Well, my online versioned storage via Google is unlimited for $12/mo which syncs my files locally and then I have another backup provider that does the same thing via CrashPlan Small Business which is also unlimited for $10/mo. So that is $22/mo for unlimited continuous backup and versioning. I don''t see how tapes would offer much value, and storage location of those tapes especially for a home use case is likely going to be a single point of failure.
    - viraptor 7y ago
      Ok, but how fast can you restore if something fails? And how fast can you add 200gb of new data to that service? It's not for home use, but they're are valid use cases.
      - techntoke 7y ago
        As fast as a local CrashPlan backup can restore based on the storage medium.
mrfusion 7y ago
What about tape gives it superior compression? Why can’t that same compression be applied to any storage?
- jarfil 7y ago
  Nothing. They estimate an average compression based on an average dataset with uncompressed text data. That wouldn't work for things like already compressed video, audio, encrypted data, and so on.
  - cl0ckt0wer 7y ago
    works great on giant uncompressed database backups though
    - sliken 7y ago
      Sure, or just datadump | gzip > /dev/tape or similar. Tape loves to brag about bandwidth/capacity and then hide the assumed 2.5x compression in a footnote.
      - namibj 7y ago
        Please consider zstd instead. It is superior in every way except memory usage and compatibility with old systems.
        sliken 7y ago
        Sure. Anything, not like text compression is hard.
        erik_seaberg 7y ago
        zstd looks good but it's only three years old, so use cases need to fit in the lifetime of today's platform. gzip is about to turn 27 and everyone has heard of it, which makes me pretty confident in finding decoders after another decade or two. (Actually this problem is why I haven't owned a tape drive for a while.)
mrtweetyhack 7y ago
I'm shocked at this new development!
jonah-archive 7y ago
If folks are curious about the benefits/drawbacks/etc of the medium, I would highly recommend checking out the media presentations given at the Library of Congress Designing Storage Architectures meeting (most slides are online -- scroll down to the "Designing Storage Architectures Meetings" section and click through to each year's list of presentations): http://www.digitalpreservation.gov/meetings/index.html http://www.digitalpreservation.gov/meetings/index.html The 2019 meeting just happened and had a lot of good info about current use of tape, slides should be online in another couple weeks. (I presented there this year and work at the Internet Archive, though we do not currently use tape in our storage infrastructure.)
- brador 7y ago
  > slides should be online in another couple weeks. Why does it take so long?
  - hamandcheese 7y ago
    They have to be loaded from tape.
    - deleted 7y ago
      [deleted]
  - pletnes 7y ago
    The tape has to rewind.
  - jhoechtl 7y ago
    People have something better to do instead providing information to you for free?
    - monocasa 7y ago
      To be fair, the job of the Library of Congress is providing information for free.
vadansky 7y ago
Is Tape worth it for a movie collection? I don’t think I will see any benefits of the compression and the prices seem comparable to just buying a 8TB HD
- klodolph 7y ago
  > Is Tape worth it for a movie collection? I doubt your movie collection is large enough that tape would be cheaper. A single tape has about ~50x the capacity of a Blu-ray Disc, but you need to buy a lot of tapes before the whole setup becomes cheaper than HDD. > I don’t think I will see any benefits of the compression… Ignore the compression, it’s garbage. > …the prices seem comparable to just buying a 8TB HD… By “comparable”… tape is clearly cheaper per byte when you compare prices, it’s just not so radically cheaper that it changes your life.
  - adrian_b 7y ago
    The most common tape is LTO-7, which is 6 TB (non-compressed). The most common Blu-ray disks store 50 GB, so a tape equals 120 discs, not 50. I have mentioned in another message that the breaking point is around 75 TB or around 50 TB, depending on how many copies you store, i.e. around 1000 to 1500 Blu-ray movies. I agree that many people, probably most people, do not need to store so much data to make tape storage economical, but for those who have a large enough data archive, like myself, the tape is really radically cheaper to have changed my life. Obviously, there is a vicious circle, few people buy tape drives because they are too expensive, so that tape drives are expensive because few of them are sold. If they would have been mass produced, tape drives should have been 5 to 10 times cheaper, and then tape archives would have made sense even for the smallest movie collections.
    - klodolph 7y ago
      > The most common tape is LTO-7, which is 6 TB (non-compressed). The most common Blu-ray disks store 50 GB, so a tape equals 120 discs, not 50. I was being conservative and using BD XL. Seems like you came to the same conclusion about the break-even point—around 150TB before you consider replication.
sliken 7y ago
Beware of two things with tape. They often have a VERY low storage temperature. I lost $5k in tapes with a minor AC problem and the room only got up to 115F for a bit. Also keep in mind that capacity and bandwidth often assume 2.5x compression of pure text, which doesn't seem like a common scenario. The article compares SSDs to tape is silly, access times are in the minutes when a robot is involved, and many minutes of you are swapping trays. Why compare to SSDs with that can to 50k IOPs and up?
marrickvillain 7y ago
Disappointing article. Sure $/GB is a factor. But what about RPO and RTO? Or $/IOPS? When those factors are taken into consideration, disk often wins. Which is why the backup to disk market has gone bonkers in recent years and tape hasn't.
ijiiijji1 7y ago
I worked at IT for a large university. We had an HP/Compaq SSL2020 with dual AIT drives made by Sony. The issue was never determined (perhaps micro debris in the air), but the drives has to be replaced nine (9) times due to failure. Furthermore, backup and verification is a tedious process, and combined with local and offsite vaulting and tape management, having proper backups is a royal PITA but there's no substitute. Replication isn't a backup because it propagates errors and you can't do disaster recovery without proper software, offsite vaulting and practice. Backups aren't valid until they're tested.
userbinator 7y ago
IMO one of the worst parts about LTO is that it's another one of those formats with "open" in its name, but you can't just download a PDF of the standard or even buy it from something like ISO/IEC. There's some convoluted licensing process involved. Only the first generation was actually open, but I guess they decided to close it after that: https://www.ecma-international.org/publications/standards/Ecma-319.htm https://www.ecma-international.org/publications/standards/Ec...
- bt848 7y ago
  What would you do with a totally open specification?
  - doikor 7y ago
    Read it?
  - nitrogen 7y ago
    It's easier to recover corrupted media if you know everything about the underlying format and protocols. As an example, I was able to recover a partition that was partially overwritten by just a few sectors by finding a backup copy of the superblock and writing it into the correct place armed only with hexdump and dd. That would have been impossible without detailed info about ext2/3/4 available. Also look at the floppy imagers people have made, that operate at the magnetic flux level.
    - Annatar 7y ago
      "That would have been impossible without detailed info about ext2/3/4 available." Try telling that to IKARI+TALENT, Legend, Skid Row, PaRaDoX and all the other cracking groups from Commodore64, Amiga, ATARI ST, PlayStations, Nintendos, groups which competed on who is going to one-file a program or convert a secret disk coding format to a regular filesystem, with all the protections disabled and the programs bug and NTSC+PAL fixed while they were at it...
      - 7y ago
Mave83 7y ago
If you only save your data as a single copy on a single tape, then the price is quite low (around 1/3 of Glacier). However you can reach prices of below Glacier using online storage with GB/s performance and access times of milliseconds using Ceph with the right hardware selection including every cost. You can learn more about how from croit.io.
raws 7y ago
In terms of longevity MDisc is king (1k years estimate). How does tape compare to drives and ssds without power? and what kind of environment is required? Someone mentioned losing 5k$ worth of tapes to temperature hike.
- meshr 7y ago
  I can confirm that they are extremely unreliable compared with other hardware. There is also no support from vendors.
- Youden 7y ago
  MDisc is not king. Their DVDs were outperformed by competitors in a test done by the French National Archives [0]. Further, DVDs are effectively obsolete these days and MDisc's Blu-Rays have no scientific studies backing their claims to longevity compared to competitors like the DVDs did. One of their largest advantages relative to DVDs is completely gone in Blu-Rays as well: The inorganic dyes and recording layer (these are now standard for essentially all BD-Rs). [0]: https://www.lne.fr/sites/default/files/inline-files/syylex-glass-dvd-accelerated-aging-report.pdf https://www.lne.fr/sites/default/files/inline-files/syylex-g... and full test and results in French: https://francearchives.fr/file/de7f8ea96ceb4ce38eb6d9278b3df49260752449/static_2140.pdf https://francearchives.fr/file/de7f8ea96ceb4ce38eb6d9278b3df...
Dylan16807 7y ago
> you can buy cheap LTO-7, reformat them to M8 and get 9TB of native storage Note: You can't reformat them this way. This is a permanent choice you make with a blank tape.
- mmaunder 7y ago
  Also IIRC you cant do this yourself with a regular LTO7 tape and drive. I think you need some kind of drive shuttle or you need to buy the tapes preformatted by someone who has the right gear. I have an LTO7/8 drive and I cant do this.
mister_hn 7y ago
sure, but they are really slow. It depends on your use cases.
- adrian_b 7y ago
  Tapes are slow only at the latency of random access. Their sequential transfer speed is lower than that of good SSDs, but higher than that of HDDs.
hoseja 7y ago
Cookie consent, adblock warning, autoplaying video, some kind of newsettler form that I just glanced at before closing the tab. What a miserable website.
- aspyct 7y ago
  Got JavaScript disabled (thanks firefox and noscript), and I have none of that bullshit. Give it a try :)
  - moviuro 7y ago
    Even just the "Reader" view of Firefox fixes that: https://support.mozilla.org/en-US/kb/firefox-reader-view-clutter-free-web-pages https://support.mozilla.org/en-US/kb/firefox-reader-view-clu...
- Delfino 7y ago
  Yes, I am an IB Computer Science teacher and we still teach tape drives as a form of storage so I was excited to share this story with my students based on the title. Then I went to the website. No way I'm sending a link to a site like that to them.
  - randomchars 7y ago
    Try it this way: https://outline.com/hDqg78 https://outline.com/hDqg78 Generally, you can add outline.com before any news or blog domain, and you get a reader view to share.
NohatCoder 7y ago
Note that the article is straight up lying about the price of HDD capacity, 12TB drives generally cost more per TB than lower capacity drives, the best capacity deals are generally in the range 3 to 8 TB. Comparing cheapest to cheapest, tapes only go for around half the price of hard disks.
- adrian_b 7y ago
  Normal HDDs are at least 3 times the price of tapes. To reach only double price of tapes you must choose some super-ancient junk, e.g. 1 TB 3.5" HDDs, which are much slower than tapes and which require a huge size and weight for the same capacity as tapes. 12 TB HDDs have about the same volume as LTO-7 tapes (i.e. about the same volume as two 6 TB tapes), but they are much heavier. LTO-8 tapes have double capacity compared to LTO-7.
- mxfh 7y ago
  Dunno, if shucking is an option, external consumer WD drives are regularily available under EUR 200 for 10 TB, while bulk drives of same capacity or higher for dedicated storage use are at least 50% more expensive. In my experience this is a releatively new development, that those two markets diverged that much. 2015 and before bulk HDD where usually cheaper than external ones. If you do care about warranty you can just keep them in the external usb casing and enjoy having an abundance of power supplies and USB 3 cables. If it's 24/7 hot storage it also uses 50kWH per Year at 6W. That's also about 15EUR/Year minimum energy costs per drive. Me feeling is, that all HDD manufacterers are extracting as as much value from the market as they get away with (meaning having more of a smell of informal price fixing, then of free market) while they still can, before they will be made osolete to solid state storage in 5 years latest.
  - NohatCoder 7y ago
    If we compare them to tapes, it doesn't seem fair that the hard disks should be kept spinning, booting them up on demand still gives you vastly lower latency than tapes. Me feeling is that the competition in the tape market is also a bit rusty, $3000 for a fancier VCR, what a steal! ;-)
jammygit 7y ago
I was told once that certain types of storage medium will demagnetize over time and that you need to spin them up to prevent data loss
jart 7y ago
If you want to talk about MBps, imagine a truck full of tape cassettes driving from the Google data center to off-site storage.
- perl4ever 7y ago
  From the sound of your comment, you haven't heard of this: https://aws.amazon.com/snowmobile/ https://aws.amazon.com/snowmobile/ ...putting stuff on a big truck is a very viable alternative to using a series of tubes.
segfaultbuserr 7y ago
I'm well-aware that tape is still the king for datacenter-level storage, and I'd love to have a tape drive for my ever-increasing size of my personal backup. Their advantages to me is obvious: its inexpensive price and great longterm reliability beats any alternative choices: hard drives, optical drives, or solid-state drives, and sometimes cloud. Also, it's cool to put my "tape archive" (.tar) files on a real tape! But I found for personal uses, tapes aren't cost-effective, the entry-cost and initial investment of the tape drive is extremely high, it's all enterprise-grade SCSI or SAN. Unless you already have tens of terabytes of data (video studio?), buying a tape drive is unjustified. HDDs are still sometimes the only cost-effective choice for personal use. Now I mainly use HDDs for large backups (> 300 GiB), multiple Blu-ray disks for medium-size backups (< 100 GiB), and use cloud for the smallest archives. I don't know if Amazon's "mail your drive to the cloud" service is available at my location, but it's also an attractive solution. I'd like to hear about your experience.
- Moru 7y ago
  Did you try backblaze.com yet?
  - segfaultbuserr 7y ago
    I think any cloud solution is limited by the broadband uplink. It takes 24 hours to upload "only" 200 GiB of data over a 20 Mbps uplink. It's fine for longterm backup that literally nobody touches unless the computer breaks, but one still has to be patient enough to backup a RAID array...
    - throw0101a 7y ago
      Backblaze has a service where they ship you a disk pack, you back up to the unit, and then ship it back: * https://www.backblaze.com/b2/solutions/datatransfer/fireball.html https://www.backblaze.com/b2/solutions/datatransfer/fireball...
      - segfaultbuserr 7y ago
stiray 7y ago
The tape story is really killing me. I even bought one old tandberg for 200 euros, used and run it on an ancient hardware just for backups. I just dont get it that in all those years not a one single vendor decided to make less expensive tape devices for the consumer market.
- EvanAnderson 7y ago
  There used to be less expensive consumer-oriented tape drives (floppy-based QIC, various parallel port drives, VXA, an DDS/DAT). That segment of the market has been subsumed by backup services, USB-based solid state or hard disk-based storage, and writable optical media.
- acuozzo 7y ago
  > not one single vendor decided to make less expensive tape devices https://www.youtube.com/watch?v=TUS0Zv2APjU https://www.youtube.com/watch?v=TUS0Zv2APjU
lmilcin 7y ago
I think this is a bit misleading. Tape is not just the tape cartridge itself. Quality drives are very expensive and so is tape management. Managing access to tapes at scale is also not trivial and complicates a lot of your processes. Another important factor everybody seems to forget is that you don't want the tape to be your only backup. They are so slow to restore that you typically only use tape as last resort. For my personal backups I calculated a bunch of regular 4TB drives, external USB 3.1 bay which takes 2 of them at once and an off-site storage (place where I can keep drives) to be best solution.
- mcv 7y ago
  Exactly what I was thinking. Tapes may be efficient while you're not accessing the data on them, but data you're not accessing is ultimately useless. You need the infrastructure to restore from backup, and if you've got so much data backed up that the cost savings of tape really matter, you might have to backup (and restore) so much data that the lower speed also really matters.
- op00to 7y ago
  Tape drives are consumable. LTO tape is abrasive, this cleans the heads but also wears them down. I used to manage a scientific computing facility with a PB scale rape archive. We ingested maybe 50 TB a day to tape, and would chew through one of the 8 drives at least twice a week.
  - ummwhat 7y ago
    I know this is a typo, and I don't usually point out such things, but maybe fix "rape archive"
    - arvinsim 7y ago
      It certainly made me stop and reread the statement.
      - grawprog 7y ago
cf141q5325 7y ago
Does anyone have any experience with encrypted tapes and could give a few words about their setup? Mainly other options then the manufacturers encryption.
- adrian_b 7y ago
  I do not trust any kind of encryption inside devices, because I cannot verify how the keys are managed. The essence of encryption is that the key must not be stored together with the encrypted data, but if a key is ever used inside a device you cannot be sure that it can no longer be retrieved from it in the future. The data files that I am writing to the tapes have their metadata written into a database to be able to retrieve them in the future, then I group them in chunks of an approximately fixed size (50 GB in my case), then I compress them, then I encrypt them and then I add some redundancy (with par2) for error recovery. A simple solution for compression with encryption is the lrzip program, but for more flexibility in encryption you could use e.g. the openssl program invoked with the desired parameters. There are many encryption programs and any of them is better than using the encryption option of a tape drive. So I queue the files to be written on the tape in a directory and when the size of the directory exceeds a threshold (50 GB in my case) a script is invoked that does the processing mentioned above, including the encryption, then it writes the result on the tape. A similar script does the inverse operations when I retrieve a file or group of files from the tape, giving its position as a tape number + a tape file number as obtained from the file database.
  - cf141q5325 7y ago
    Thank you for the detailed answer.
meshr 7y ago
This is not true at least in my experience. Price doesn’t tell you about reliability. I have 10% of broken cartridges and 66% of broken tape drives. These devices have mechanical parts and are sensitive to magnetic fields so they could be damaged easily. If you don't believe me then I can send you all this garbage to you
ai_ja_nai 7y ago
Assuming that you already own the tape recorder, WTF
unnouinceput 7y ago
For me there is only one factor that matters in backup. Not restore/backup speed, not price per byte, not fashion - only reliability. Does the backup medium will perform without corruption when times come to read it? That's the only thing that matters. I had this problem - store and forget, long time, as in decades. Tried a lot of stuff and: Personal conclusion - offline backup sucks, regardless of medium. Tape, harddisks, zip drives - demagnetization of medium. CD, bluray - scratch and you're done. Thumb stick, SSD, non-volatile memory in general - needs a plug-in once in a while. So what I use is encrypted volumes and upload to cloud...and a dedicated SSD that I have on me at all times if I need offline access (which gets plugged in daily).
- Erlich_Bachman 7y ago
  How are you gonna scratch your optical discs? Do you move them around all the time? Also with something like Rar recovery records or or other redundancy measures, a scratch does not mean that you are "done", because the data can be recovered anyway (with storage size penalty, but setting aside 10% of storage for being able to recover from random scratches is well worth it).
  - masklinn 7y ago
    > How are you gonna scratch your optical discs? IME the problem of optical discs is that unless they're molded (ROM) they degrade over time.
    - Erlich_Bachman 7y ago
      Not M-Disc which is rated to last for 1000 years and can be burned with consumer grade burner.
      - masklinn 7y ago
        That's just the company's claims. When the Laboratoire National de Métrologie et d’Essais performed an extreme conditions accelerated aging test (90°C and 85% RH) they found the M-Disc to be no better than most quality DVD+R (<250h), with MPO's Gold and Northern Star's DataTresorDisc reaching 250h (but not 500) and Syylex's GlassMasterDisc surviving almost unscathed to 1000h. Sadly it looks like Syylex is as dead as Millenniata, Inc (though it looks like Verbatim still sells m-discs).
hyperpallium 7y ago
If you overshoot demand on one performance metric, buyers shift to another as yet unsatisfied criterion. e.g. 8" HDDs are also cheaper per byte than 5.25", than 3.5", than SSD's.
learnstats2 7y ago
"But there's something else that tape offers that no other storage medium currently offers and that's on-the-fly, transparent compression which can go up to 2.5:1" I don't understand this point. Why would tape allow compression that other forms of media don't?
- apolog 7y ago
  Not an expert here, but could it be because you can actually slow down the tape 2,5x and still be readable?
- makapuf 7y ago
  I think what is unique here could be the "on-the-fly, transparent". But this kind of compression can be better achieved I think with one core of your CPU, plaintext compression of 3x shouldn't need lzma (and if you really need to compress 22TB of plain text with no binary or multimedia files, well maybe you could compress it slightly with lz4 or a simple zlib compression before sending it).
- tenebrisalietum 7y ago
  Lots of tape drives offer built-in compression. The drive firmware does it. On AIT3 drives you can enable or disable it. Compression may have been a big deal in the ages of sub-1Ghz single core systems but not anymore. Especially since you ought to be encrypting before writing anyway. Why does it continue? Probably so they can double the capacity that they put on the tape label, which is always the compressed capacity and not the native or real capacity. Plus backward compatible with such encoded tapes.
- Spooky23 7y ago
  People are very conservative about backup, and some backup solutions don’t compress in software.
FrozenVoid 7y ago
Every week without fail i see articles shilling tape storage. Do people realize only really Big Data benefits from tape? Unless you write multiple petabytes of data that you rarely need each day, tape is inferior in every way to hard disks. A huge corporation that counts every cent and needs lots of cold storage might benefit from tape, and companies still prefer better storage ( https://engineering.fb.com/core-data/under-the-hood-facebook-s-cold-storage-system/ https://engineering.fb.com/core-data/under-the-hood-facebook... ). "Good enough for Google, good enough for you"(a lie that neglects economies of scale that force Google to use tape). https://www.overlandstorage.com/blog/?p=323 https://www.overlandstorage.com/blog/?p=323
- adrian_b 7y ago
  No, tapes become preferable to HDDs at less than 100 TB written, not at multiple petabytes. For many applications and for many people, including for myself in my home, it makes absolutely no sense to use HDDs for anything, but only SSDs & tapes. For long term storage tape is superior in every way to HDDs, while for fast access SSDs are superior in every way to HDDs. Obviously, because the drives are extremely expensive, whoever needs no more than 20 or 30 TB of storage cannot afford tapes and must use a few HDDs. The best storage technology changes in time. 20 years ago I was using optical disks and 10 years ago I was using HDDs, but since I went back to tapes, like I was previously using 30 years ago, I have saved a lot of money and I have much less worries about the HDDs becoming defective.
  - FrozenVoid 7y ago
    How does it compare with 8TB USB3 drives(140$)? https://pcpartpicker.com/product/YpJtt6/western-digital-elements-8tb-external-hard-drive-wdbwlg0080hbk-nesn https://pcpartpicker.com/product/YpJtt6/western-digital-elem...
    - adrian_b 7y ago
      LTO-7 tapes are $10 per TB at Amazon (of course uncompressed), so they are still cheaper, but this $17.50 USD per TB is indeed the cheapest HDD that I have ever seen. Nevertheless, these HDDs are much more expensive than they look at the first glance. The real cost of archival storage is given by the price per TB and per warranty year, because after the warranty time you must replace the storage medium. I have seen countless HDDs that have died just a few months after their warranty time expired. So these drives with 2-year warranty cost $8.75 per TB and per year. The warranty time of the tapes is 30 years, but in fact you must replace them much faster because you will no longer find tape drives able to read them. For LTO tapes, you should expect to replace them after 6 to 10 years. Even with only a 6 year lifetime, the price for tape would be $1.67 per TB per year, i.e. more than 5 times cheaper than those WD HDDs. If you want to just have some backups against accidents, which you expect to erase after a few weeks, then those HDDs could be a good choice. For archival purposes they are still much more expensive than tapes.
not_a_cop75 7y ago
And why wouldn't it? Tapes as far as I know cost pennies to manufacture. It's easily over 90% plastic in most cases. The low quality ingredient on a low end tape is probably still ferric oxide - not exactly an expensive chemical if I know anything about chemistry. Perhaps cobalt is still heavily used as part of the coating as well.
ryanmercer 7y ago
It never ceases to amaze me how much tape I still see imported, mind you I see way more SSDs, then HDDs but still a fantastical amount of tape.
stelonix 7y ago
Are there any cheap tape drives? If not, what's the issue? Is it the fact the prices are kept high because it's used mostly by business/enterprises customers? If so, are there any hopes of LTO becoming mainstream for home desktop usage? Couldn't find much info on Google about this.
- NeedMoreTea 7y ago
  DAT (Well DDS) used to fill that niche. Used to be pretty reasonably priced - for tape. Went for a few hundred a drive, an order of magnitude less than LTO. Smaller capacity each generation though, but is an older standard. Disappeared for no apparent reason about a decade or so ago.
  - deleted 7y ago
    [deleted]
craig1f 7y ago
This is the principle behind AWS Glacier
bufferoverflow 7y ago
That's if you don't count the extra time required to deal with tape backups.
idlerboris 7y ago
It's crucially important to take managing and lifetime of such storages into account. I doubt it can fight with SSD or HDD.
scraft 7y ago
We had a Ditto drive and a series of tapes, bought circa 1996. We used the software it came with on Windows to perform full back ups of the machine, cycling between a set of tapes (maybe 6). Backup would complete overnight. Never had to restore from it. It was a consumer friendly, easy to use product and cheap. My father was an accountant that ran his own practise, so needed to know his files were safe. That went from carbon copies into a second filing cabinet, to 5 1/4" floppies for groups of clients, to 3.5" floppies, to Ditto tapes to USB sticks to Microsoft Cloud storage (which sinks between laptop and PC). The oldest days of filing cabinets and floppies were easy, the middle days of Ditto and USB storage was more worrying, and the final days of cloud storage went back to being simple again. Now he is retired so none of it matters. This was all donkeys years ago, so it is interesting to hear how things have changed.
vonseel 7y ago
As someone totally unfamiliar with tape aside from knowing some places still use it for archival and recording audio to tape - something totally different - is incredibly expensive and most (if not all?) recording tape is out of production, I’m surprised storage tape is still a thing. How does storage tape even work? Is this literal tape in a cartridge much like tape in a cassette tape? I assume we’re not talking about data stored in 1s and 0s? Is this technically an analog medium!? Going to do some googling now to quell my curiosity. The article does seem targeted at consumers. I’m going to guess I’d need some expensive hardware to even use one of these $60 tape drives.

230 points by jaytaylor 7y ago

Did anyone ever figure out what Amazon Glacier uses? They only publicly talk about using custom software to "optimize the sequence of inputs and outputs"...[1] [1] https://aws.amazon.com/glacier/faqs/ https://aws.amazon.com/glacier/faqs/

cobookman 7y ago
Note cloud vendors get the benefit of mixing high IOPs workloads with low IOPs workloads. Say a cloud vendor needs a bunch of HDD for a database workload with high IOPs. Using modern disks (4+ TiB) this leaves a lot of stranded disk storage that can only be used for low IOPs workloads. Archive storage uniquely meets the criteria of lots of storage, yet low IOPs. Disclaimer work for Google but have no knowledge of how AWS glacier works.
- ggm 7y ago
  I could believe that we have a lot of unused space which is amenable to a different overlay use, but the potential for mis-design is huge here. If I pay high IOPs cost and you have to replace my high IOPS data backend on my paypoint, migrating the low paypoint secondary customer is non-trivial if not designed in day one (I am sure you did: cheap players might not understand the risk exposure here) The telco equivalent was ISDN, where a single linecard with two sub-rate ports might be sold to a bank, and a domestic user. The bank could (and would) demand card reset 24/7 to restore service to their ATM, the domestic user simply lost carrier, and game state.
colanderman 7y ago
Rumor has it Blu-Ray (specifically BDXL): https://storagemojo.com/2014/04/25/amazons-glacier-secret-bdxl/ https://storagemojo.com/2014/04/25/amazons-glacier-secret-bd... Though a comment on that page mentions this HN comment claiming custom low-speed HDDs: https://news.ycombinator.com/item?id=4416065 https://news.ycombinator.com/item?id=4416065
- sshagent 7y ago
  I certainly recall a Facebook video showcasing their bluray archival system. I was seriously impressed by the whole setup. And here we are archiving to LTO like primitives :)
  - jolmg 7y ago
    Even with BDXL, which seems to be able to store up to 128GB per disc, you'd have to stack 100 of them to store the 12TB that an LTO-8 tape cartridge can store raw/natively. Tape cartridges also seem to be more physically durable than optical discs. This doesn't mean that they seem simply superior, though. I imagine optical discs have way better seek times, for example, even if read/write speeds are better with tape.
milesward 7y ago
:) I know :)
etaioinshrdlu 7y ago
Rumor (from this site) consensus seems to be custom low RPM hard drives optimized for long life.

basicplus2 7y ago

"As for transfer speeds, they can reach 300MBps" I am getting over 500MBps on my setup

jaytaylor 7y ago
Impressive. Will you share more information about the specific details (i.e. hardware and configuration) for the set up you're running with?
- merb 7y ago
  well probably with compression
  - jolmg 7y ago
    I wonder if the 300MBps figure might also be with compression.
    - adrian_b 7y ago
      No, the LTO-7 transfer speed is 3 Gb/s a.k.a. 300 MB/s without compression. The compression in drive may be useful for automated backups, but if you use the tape for an archive of files selected by yourself, it is better to disable the drive compression. If you have compressible files, you can compress them with better algorithms on the computer, before storing them on the tape.
      - merb 7y ago
        > you can compress them with better algorithms on the computer is that always true? what if I have dozen's of pdf's which are similar and compressed (most pdfs are)? wouldn't de-duplication also reduce the size? I mean with millions/billions/trillions of files of the same format.
        adrian_b 7y ago

jolmg 7y ago

> Even if LTO-8 tapes are now in stock, you can buy cheap LTO-7, I wonder if this was a typo. Did they mean that LTO-8 tapes are out of stock? Otherwise, the sentence structure "even if ... are now in stock, you can buy ..." is kind of weird. > reformat them to M8 and get 9TB of native storage (22.5TB compressed). What does this mean? Doesn't how much it compresses depend on the data not the storage device?

bengerbil 7y ago
There was a dispute involving LTO-8 that just recently was resolved, so LTO-8 tape should now be available in the US. I wasn't following it closely, so I can't give any real insight, but it explains the wording.
- mycall 7y ago
  3TB for $153. Not impressed. https://www.google.com/shopping/product/10502190238469223139 https://www.google.com/shopping/product/10502190238469223139
  - SethTro 7y ago
    I think it's 12TB Google says "HP LTO-8 Ultrium, 30 TB, RW Data Cartridge Tape. Exhaustively tested, HPE LTO Ultrium cartridges meet all your demands for maximum reliability when restoring data, offering high storage density, ease of management and scalable storage and backup performance." TapeandMedia.com says "1 - HPE LTO Ultrium 8 Tape with a capacity of 12 TB and up to 30 TB compressed capacity."
  - jolmg 7y ago
    That says 30TB. Here's a 15TB compressed for $63: https://www.amazon.com/HP-C7977A-Cartridge-Compressed-Capacity/dp/B01NCB4ZZP/ref=mp_s_a_1_10?keywords=lto7+tape&qid=1569376299&s=gateway&sprefix=lto&sr=8-10 https://www.amazon.com/HP-C7977A-Cartridge-Compressed-Capaci...
    - tw04 7y ago

jaytaylor 7y ago

I'd be curious to hear from any folks who actually run tape drive setups for personal use. Where did you get the gear? How cost-effective is the overall setup compared to spinning rust? How is driver support in popular operating systems? Is there more you'd recommend taking into consideration or learning before jumping in? Especially beyond the obvious realities of extremely high latency, of course ;)

netrap 7y ago
eBay. 1-1 HDD backups is probably close in price even in the used tape market. I use LTO-4 with FC. Works fine in Windows with Retrospect. I'd expect it to work in Linux as well. The hard part is backup software. I guess with LTO-5+ you can just directly access the tapes with LTFS, but I haven't tried that yet.
- mc32 7y ago
  Anything which supports ndmp should be good. Both bacula and Amanda among other SW support NDMP. I mean of course enterprise solutions support it too.
klodolph 7y ago
> Where did you get the gear? I never ended up using it personally, but when I was considering it, used gear on eBay was top of the list. > How cost-effective is the overall setup compared to spinning rust? For personal use, it is not even remotely cost-effective unless you are a data hoarder. The cost of the drive is simply too high. Also keep in mind that the drive can become damaged or need servicing. It only makes sense if you are dealing with e.g. large quantities of video. The article cites a minimum 140TB where tape becomes cheaper but this involves some dubious decision-making. > How is driver support in popular operating systems? I have only ever used tape with Linux + custom software so I can’t speak of software support in general. > Is there more you'd recommend taking into consideration or learning before jumping in? Especially beyond the obvious realities of extremely high latency, of course ;) The obvious reality to me here is that it just doesn’t make sense unless you’re in spitting distance of a petabyte. The only thing I can imagine someone doing personally is video, and it would have to be a lot of video like some kind of long-running documentary project that you are producing in your spare time. The other factor is to be more paranoid about mixing tape generations than the vendors suggest. Ignore the specs and stick to same-generation media and drives. Efficiently writing to a tape means filling e.g. a 20 MB/s write channel for LTO-6, or 750 MB/s for LTO-8. Can you sustain 750 MB/s? It’s not so easy in practice. You throw away tape capacity if you get buffer underruns.

robjan 7y ago

Another crucial importance is bytes per square centimetre. When archiving something, there's no point in archiving hundreds of storage controllers with it especially if you need to store them in a safe (which you probably should). MicroSD cards may beat tape but they are a lot slower and harder to label.

koolba 7y ago
Which one wins on long term storage durability, shock proof, and water damage?
- blotter_paper 7y ago
  https://en.wikipedia.org/wiki/M-DISC https://en.wikipedia.org/wiki/M-DISC
- bt848 7y ago
  LTO doesn't suffer from those things. Shock can destroy the case and dislodge the leader pin of an LTO cartridge but AFAIK it can't damage the data on the tape. Water is bad for the transport but as long as you dry your tapes out they should still work. The operating specs for LTO are crazy, you can read and write them under all practical atmospheric conditions.
  - mkj 7y ago
    Smoke haze will cause all kinds of trouble for tape drives though.
    - zaarn 7y ago
      So don't run the backups when the DC is on fire, got it.
    - bacon_waffle 7y ago
      As will very dry conditions. The IceCube neutrino telescope at South Pole used LTO-3 tapes to store data until they switched to only hard drives around 5 years ago. At that point, the newer tape drives had been tried (IIRC LTO-5 was the industry standard by then) but they never seemed to work, presumably because the humidity on station is incredibly low. It was considered more reasonable to keep the old LTO-3 equipment in use, versus building a humidor for the tapes and drives.

techntoke 7y ago

Price per byte maybe, but in value you are sacrificing a lot. I prefer continuous backups and versioned storage.

viraptor 7y ago
You're not sacrificing anything. Or specifically having a long term, offline tape storage does not prevent you from having short term, online, versioned file/block storage. Different use cases.
- techntoke 7y ago
  Well, my online versioned storage via Google is unlimited for $12/mo which syncs my files locally and then I have another backup provider that does the same thing via CrashPlan Small Business which is also unlimited for $10/mo. So that is $22/mo for unlimited continuous backup and versioning. I don''t see how tapes would offer much value, and storage location of those tapes especially for a home use case is likely going to be a single point of failure.
  - viraptor 7y ago
    Ok, but how fast can you restore if something fails? And how fast can you add 200gb of new data to that service? It's not for home use, but they're are valid use cases.
    - techntoke 7y ago
      As fast as a local CrashPlan backup can restore based on the storage medium.

mrfusion 7y ago

What about tape gives it superior compression? Why can’t that same compression be applied to any storage?

jarfil 7y ago
Nothing. They estimate an average compression based on an average dataset with uncompressed text data. That wouldn't work for things like already compressed video, audio, encrypted data, and so on.
- cl0ckt0wer 7y ago
  works great on giant uncompressed database backups though
  - sliken 7y ago
    Sure, or just datadump | gzip > /dev/tape or similar. Tape loves to brag about bandwidth/capacity and then hide the assumed 2.5x compression in a footnote.
    - namibj 7y ago
      Please consider zstd instead. It is superior in every way except memory usage and compatibility with old systems.
      - sliken 7y ago
        Sure. Anything, not like text compression is hard.
      - erik_seaberg 7y ago
        zstd looks good but it's only three years old, so use cases need to fit in the lifetime of today's platform. gzip is about to turn 27 and everyone has heard of it, which makes me pretty confident in finding decoders after another decade or two. (Actually this problem is why I haven't owned a tape drive for a while.)

mrtweetyhack 7y ago

I'm shocked at this new development!

jonah-archive 7y ago

If folks are curious about the benefits/drawbacks/etc of the medium, I would highly recommend checking out the media presentations given at the Library of Congress Designing Storage Architectures meeting (most slides are online -- scroll down to the "Designing Storage Architectures Meetings" section and click through to each year's list of presentations): http://www.digitalpreservation.gov/meetings/index.html http://www.digitalpreservation.gov/meetings/index.html The 2019 meeting just happened and had a lot of good info about current use of tape, slides should be online in another couple weeks. (I presented there this year and work at the Internet Archive, though we do not currently use tape in our storage infrastructure.)

brador 7y ago
> slides should be online in another couple weeks. Why does it take so long?
- hamandcheese 7y ago
  They have to be loaded from tape.
  - deleted 7y ago
    [deleted]
- pletnes 7y ago
  The tape has to rewind.
- jhoechtl 7y ago
  People have something better to do instead providing information to you for free?
  - monocasa 7y ago
    To be fair, the job of the Library of Congress is providing information for free.

vadansky 7y ago

Is Tape worth it for a movie collection? I don’t think I will see any benefits of the compression and the prices seem comparable to just buying a 8TB HD

klodolph 7y ago
> Is Tape worth it for a movie collection? I doubt your movie collection is large enough that tape would be cheaper. A single tape has about ~50x the capacity of a Blu-ray Disc, but you need to buy a lot of tapes before the whole setup becomes cheaper than HDD. > I don’t think I will see any benefits of the compression… Ignore the compression, it’s garbage. > …the prices seem comparable to just buying a 8TB HD… By “comparable”… tape is clearly cheaper per byte when you compare prices, it’s just not so radically cheaper that it changes your life.
- adrian_b 7y ago
  The most common tape is LTO-7, which is 6 TB (non-compressed). The most common Blu-ray disks store 50 GB, so a tape equals 120 discs, not 50. I have mentioned in another message that the breaking point is around 75 TB or around 50 TB, depending on how many copies you store, i.e. around 1000 to 1500 Blu-ray movies. I agree that many people, probably most people, do not need to store so much data to make tape storage economical, but for those who have a large enough data archive, like myself, the tape is really radically cheaper to have changed my life. Obviously, there is a vicious circle, few people buy tape drives because they are too expensive, so that tape drives are expensive because few of them are sold. If they would have been mass produced, tape drives should have been 5 to 10 times cheaper, and then tape archives would have made sense even for the smallest movie collections.
  - klodolph 7y ago
    > The most common tape is LTO-7, which is 6 TB (non-compressed). The most common Blu-ray disks store 50 GB, so a tape equals 120 discs, not 50. I was being conservative and using BD XL. Seems like you came to the same conclusion about the break-even point—around 150TB before you consider replication.

sliken 7y ago

Beware of two things with tape. They often have a VERY low storage temperature. I lost $5k in tapes with a minor AC problem and the room only got up to 115F for a bit. Also keep in mind that capacity and bandwidth often assume 2.5x compression of pure text, which doesn't seem like a common scenario. The article compares SSDs to tape is silly, access times are in the minutes when a robot is involved, and many minutes of you are swapping trays. Why compare to SSDs with that can to 50k IOPs and up?

marrickvillain 7y ago

Disappointing article. Sure $/GB is a factor. But what about RPO and RTO? Or $/IOPS? When those factors are taken into consideration, disk often wins. Which is why the backup to disk market has gone bonkers in recent years and tape hasn't.

ijiiijji1 7y ago

I worked at IT for a large university. We had an HP/Compaq SSL2020 with dual AIT drives made by Sony. The issue was never determined (perhaps micro debris in the air), but the drives has to be replaced nine (9) times due to failure. Furthermore, backup and verification is a tedious process, and combined with local and offsite vaulting and tape management, having proper backups is a royal PITA but there's no substitute. Replication isn't a backup because it propagates errors and you can't do disaster recovery without proper software, offsite vaulting and practice. Backups aren't valid until they're tested.

userbinator 7y ago

IMO one of the worst parts about LTO is that it's another one of those formats with "open" in its name, but you can't just download a PDF of the standard or even buy it from something like ISO/IEC. There's some convoluted licensing process involved. Only the first generation was actually open, but I guess they decided to close it after that: https://www.ecma-international.org/publications/standards/Ecma-319.htm https://www.ecma-international.org/publications/standards/Ec...

bt848 7y ago
What would you do with a totally open specification?
- doikor 7y ago
  Read it?
- nitrogen 7y ago
  It's easier to recover corrupted media if you know everything about the underlying format and protocols. As an example, I was able to recover a partition that was partially overwritten by just a few sectors by finding a backup copy of the superblock and writing it into the correct place armed only with hexdump and dd. That would have been impossible without detailed info about ext2/3/4 available. Also look at the floppy imagers people have made, that operate at the magnetic flux level.
  - Annatar 7y ago
    "That would have been impossible without detailed info about ext2/3/4 available." Try telling that to IKARI+TALENT, Legend, Skid Row, PaRaDoX and all the other cracking groups from Commodore64, Amiga, ATARI ST, PlayStations, Nintendos, groups which competed on who is going to one-file a program or convert a secret disk coding format to a regular filesystem, with all the protections disabled and the programs bug and NTSC+PAL fixed while they were at it...
    - 7y ago

Mave83 7y ago

If you only save your data as a single copy on a single tape, then the price is quite low (around 1/3 of Glacier). However you can reach prices of below Glacier using online storage with GB/s performance and access times of milliseconds using Ceph with the right hardware selection including every cost. You can learn more about how from croit.io.

raws 7y ago

In terms of longevity MDisc is king (1k years estimate). How does tape compare to drives and ssds without power? and what kind of environment is required? Someone mentioned losing 5k$ worth of tapes to temperature hike.

meshr 7y ago
I can confirm that they are extremely unreliable compared with other hardware. There is also no support from vendors.
Youden 7y ago
MDisc is not king. Their DVDs were outperformed by competitors in a test done by the French National Archives [0]. Further, DVDs are effectively obsolete these days and MDisc's Blu-Rays have no scientific studies backing their claims to longevity compared to competitors like the DVDs did. One of their largest advantages relative to DVDs is completely gone in Blu-Rays as well: The inorganic dyes and recording layer (these are now standard for essentially all BD-Rs). [0]: https://www.lne.fr/sites/default/files/inline-files/syylex-glass-dvd-accelerated-aging-report.pdf https://www.lne.fr/sites/default/files/inline-files/syylex-g... and full test and results in French: https://francearchives.fr/file/de7f8ea96ceb4ce38eb6d9278b3df49260752449/static_2140.pdf https://francearchives.fr/file/de7f8ea96ceb4ce38eb6d9278b3df...

Dylan16807 7y ago

> you can buy cheap LTO-7, reformat them to M8 and get 9TB of native storage Note: You can't reformat them this way. This is a permanent choice you make with a blank tape.

mmaunder 7y ago
Also IIRC you cant do this yourself with a regular LTO7 tape and drive. I think you need some kind of drive shuttle or you need to buy the tapes preformatted by someone who has the right gear. I have an LTO7/8 drive and I cant do this.

mister_hn 7y ago

sure, but they are really slow. It depends on your use cases.

adrian_b 7y ago
Tapes are slow only at the latency of random access. Their sequential transfer speed is lower than that of good SSDs, but higher than that of HDDs.

hoseja 7y ago

Cookie consent, adblock warning, autoplaying video, some kind of newsettler form that I just glanced at before closing the tab. What a miserable website.

aspyct 7y ago
Got JavaScript disabled (thanks firefox and noscript), and I have none of that bullshit. Give it a try :)
- moviuro 7y ago
  Even just the "Reader" view of Firefox fixes that: https://support.mozilla.org/en-US/kb/firefox-reader-view-clutter-free-web-pages https://support.mozilla.org/en-US/kb/firefox-reader-view-clu...
Delfino 7y ago
Yes, I am an IB Computer Science teacher and we still teach tape drives as a form of storage so I was excited to share this story with my students based on the title. Then I went to the website. No way I'm sending a link to a site like that to them.
- randomchars 7y ago
  Try it this way: https://outline.com/hDqg78 https://outline.com/hDqg78 Generally, you can add outline.com before any news or blog domain, and you get a reader view to share.

NohatCoder 7y ago

Note that the article is straight up lying about the price of HDD capacity, 12TB drives generally cost more per TB than lower capacity drives, the best capacity deals are generally in the range 3 to 8 TB. Comparing cheapest to cheapest, tapes only go for around half the price of hard disks.

adrian_b 7y ago
Normal HDDs are at least 3 times the price of tapes. To reach only double price of tapes you must choose some super-ancient junk, e.g. 1 TB 3.5" HDDs, which are much slower than tapes and which require a huge size and weight for the same capacity as tapes. 12 TB HDDs have about the same volume as LTO-7 tapes (i.e. about the same volume as two 6 TB tapes), but they are much heavier. LTO-8 tapes have double capacity compared to LTO-7.
mxfh 7y ago
Dunno, if shucking is an option, external consumer WD drives are regularily available under EUR 200 for 10 TB, while bulk drives of same capacity or higher for dedicated storage use are at least 50% more expensive. In my experience this is a releatively new development, that those two markets diverged that much. 2015 and before bulk HDD where usually cheaper than external ones. If you do care about warranty you can just keep them in the external usb casing and enjoy having an abundance of power supplies and USB 3 cables. If it's 24/7 hot storage it also uses 50kWH per Year at 6W. That's also about 15EUR/Year minimum energy costs per drive. Me feeling is, that all HDD manufacterers are extracting as as much value from the market as they get away with (meaning having more of a smell of informal price fixing, then of free market) while they still can, before they will be made osolete to solid state storage in 5 years latest.
- NohatCoder 7y ago
  If we compare them to tapes, it doesn't seem fair that the hard disks should be kept spinning, booting them up on demand still gives you vastly lower latency than tapes. Me feeling is that the competition in the tape market is also a bit rusty, $3000 for a fancier VCR, what a steal! ;-)

jammygit 7y ago

I was told once that certain types of storage medium will demagnetize over time and that you need to spin them up to prevent data loss

jart 7y ago

If you want to talk about MBps, imagine a truck full of tape cassettes driving from the Google data center to off-site storage.

perl4ever 7y ago
From the sound of your comment, you haven't heard of this: https://aws.amazon.com/snowmobile/ https://aws.amazon.com/snowmobile/ ...putting stuff on a big truck is a very viable alternative to using a series of tubes.

segfaultbuserr 7y ago

I'm well-aware that tape is still the king for datacenter-level storage, and I'd love to have a tape drive for my ever-increasing size of my personal backup. Their advantages to me is obvious: its inexpensive price and great longterm reliability beats any alternative choices: hard drives, optical drives, or solid-state drives, and sometimes cloud. Also, it's cool to put my "tape archive" (.tar) files on a real tape! But I found for personal uses, tapes aren't cost-effective, the entry-cost and initial investment of the tape drive is extremely high, it's all enterprise-grade SCSI or SAN. Unless you already have tens of terabytes of data (video studio?), buying a tape drive is unjustified. HDDs are still sometimes the only cost-effective choice for personal use. Now I mainly use HDDs for large backups (> 300 GiB), multiple Blu-ray disks for medium-size backups (< 100 GiB), and use cloud for the smallest archives. I don't know if Amazon's "mail your drive to the cloud" service is available at my location, but it's also an attractive solution. I'd like to hear about your experience.

Moru 7y ago
Did you try backblaze.com yet?
- segfaultbuserr 7y ago
  I think any cloud solution is limited by the broadband uplink. It takes 24 hours to upload "only" 200 GiB of data over a 20 Mbps uplink. It's fine for longterm backup that literally nobody touches unless the computer breaks, but one still has to be patient enough to backup a RAID array...
  - throw0101a 7y ago
    Backblaze has a service where they ship you a disk pack, you back up to the unit, and then ship it back: * https://www.backblaze.com/b2/solutions/datatransfer/fireball.html https://www.backblaze.com/b2/solutions/datatransfer/fireball...
    - segfaultbuserr 7y ago

stiray 7y ago

The tape story is really killing me. I even bought one old tandberg for 200 euros, used and run it on an ancient hardware just for backups. I just dont get it that in all those years not a one single vendor decided to make less expensive tape devices for the consumer market.

EvanAnderson 7y ago
There used to be less expensive consumer-oriented tape drives (floppy-based QIC, various parallel port drives, VXA, an DDS/DAT). That segment of the market has been subsumed by backup services, USB-based solid state or hard disk-based storage, and writable optical media.
acuozzo 7y ago
> not one single vendor decided to make less expensive tape devices https://www.youtube.com/watch?v=TUS0Zv2APjU https://www.youtube.com/watch?v=TUS0Zv2APjU

lmilcin 7y ago

I think this is a bit misleading. Tape is not just the tape cartridge itself. Quality drives are very expensive and so is tape management. Managing access to tapes at scale is also not trivial and complicates a lot of your processes. Another important factor everybody seems to forget is that you don't want the tape to be your only backup. They are so slow to restore that you typically only use tape as last resort. For my personal backups I calculated a bunch of regular 4TB drives, external USB 3.1 bay which takes 2 of them at once and an off-site storage (place where I can keep drives) to be best solution.

mcv 7y ago
Exactly what I was thinking. Tapes may be efficient while you're not accessing the data on them, but data you're not accessing is ultimately useless. You need the infrastructure to restore from backup, and if you've got so much data backed up that the cost savings of tape really matter, you might have to backup (and restore) so much data that the lower speed also really matters.
op00to 7y ago
Tape drives are consumable. LTO tape is abrasive, this cleans the heads but also wears them down. I used to manage a scientific computing facility with a PB scale rape archive. We ingested maybe 50 TB a day to tape, and would chew through one of the 8 drives at least twice a week.
- ummwhat 7y ago
  I know this is a typo, and I don't usually point out such things, but maybe fix "rape archive"
  - arvinsim 7y ago
    It certainly made me stop and reread the statement.
    - grawprog 7y ago

cf141q5325 7y ago

Does anyone have any experience with encrypted tapes and could give a few words about their setup? Mainly other options then the manufacturers encryption.

adrian_b 7y ago
I do not trust any kind of encryption inside devices, because I cannot verify how the keys are managed. The essence of encryption is that the key must not be stored together with the encrypted data, but if a key is ever used inside a device you cannot be sure that it can no longer be retrieved from it in the future. The data files that I am writing to the tapes have their metadata written into a database to be able to retrieve them in the future, then I group them in chunks of an approximately fixed size (50 GB in my case), then I compress them, then I encrypt them and then I add some redundancy (with par2) for error recovery. A simple solution for compression with encryption is the lrzip program, but for more flexibility in encryption you could use e.g. the openssl program invoked with the desired parameters. There are many encryption programs and any of them is better than using the encryption option of a tape drive. So I queue the files to be written on the tape in a directory and when the size of the directory exceeds a threshold (50 GB in my case) a script is invoked that does the processing mentioned above, including the encryption, then it writes the result on the tape. A similar script does the inverse operations when I retrieve a file or group of files from the tape, giving its position as a tape number + a tape file number as obtained from the file database.
- cf141q5325 7y ago
  Thank you for the detailed answer.

meshr 7y ago

This is not true at least in my experience. Price doesn’t tell you about reliability. I have 10% of broken cartridges and 66% of broken tape drives. These devices have mechanical parts and are sensitive to magnetic fields so they could be damaged easily. If you don't believe me then I can send you all this garbage to you

ai_ja_nai 7y ago

Assuming that you already own the tape recorder, WTF

unnouinceput 7y ago

For me there is only one factor that matters in backup. Not restore/backup speed, not price per byte, not fashion - only reliability. Does the backup medium will perform without corruption when times come to read it? That's the only thing that matters. I had this problem - store and forget, long time, as in decades. Tried a lot of stuff and: Personal conclusion - offline backup sucks, regardless of medium. Tape, harddisks, zip drives - demagnetization of medium. CD, bluray - scratch and you're done. Thumb stick, SSD, non-volatile memory in general - needs a plug-in once in a while. So what I use is encrypted volumes and upload to cloud...and a dedicated SSD that I have on me at all times if I need offline access (which gets plugged in daily).

Erlich_Bachman 7y ago
How are you gonna scratch your optical discs? Do you move them around all the time? Also with something like Rar recovery records or or other redundancy measures, a scratch does not mean that you are "done", because the data can be recovered anyway (with storage size penalty, but setting aside 10% of storage for being able to recover from random scratches is well worth it).
- masklinn 7y ago
  > How are you gonna scratch your optical discs? IME the problem of optical discs is that unless they're molded (ROM) they degrade over time.
  - Erlich_Bachman 7y ago
    Not M-Disc which is rated to last for 1000 years and can be burned with consumer grade burner.
    - masklinn 7y ago
      That's just the company's claims. When the Laboratoire National de Métrologie et d’Essais performed an extreme conditions accelerated aging test (90°C and 85% RH) they found the M-Disc to be no better than most quality DVD+R (<250h), with MPO's Gold and Northern Star's DataTresorDisc reaching 250h (but not 500) and Syylex's GlassMasterDisc surviving almost unscathed to 1000h. Sadly it looks like Syylex is as dead as Millenniata, Inc (though it looks like Verbatim still sells m-discs).

hyperpallium 7y ago

If you overshoot demand on one performance metric, buyers shift to another as yet unsatisfied criterion. e.g. 8" HDDs are also cheaper per byte than 5.25", than 3.5", than SSD's.

learnstats2 7y ago

"But there's something else that tape offers that no other storage medium currently offers and that's on-the-fly, transparent compression which can go up to 2.5:1" I don't understand this point. Why would tape allow compression that other forms of media don't?

apolog 7y ago
Not an expert here, but could it be because you can actually slow down the tape 2,5x and still be readable?
makapuf 7y ago
I think what is unique here could be the "on-the-fly, transparent". But this kind of compression can be better achieved I think with one core of your CPU, plaintext compression of 3x shouldn't need lzma (and if you really need to compress 22TB of plain text with no binary or multimedia files, well maybe you could compress it slightly with lz4 or a simple zlib compression before sending it).
tenebrisalietum 7y ago
Lots of tape drives offer built-in compression. The drive firmware does it. On AIT3 drives you can enable or disable it. Compression may have been a big deal in the ages of sub-1Ghz single core systems but not anymore. Especially since you ought to be encrypting before writing anyway. Why does it continue? Probably so they can double the capacity that they put on the tape label, which is always the compressed capacity and not the native or real capacity. Plus backward compatible with such encoded tapes.
Spooky23 7y ago
People are very conservative about backup, and some backup solutions don’t compress in software.

FrozenVoid 7y ago

Every week without fail i see articles shilling tape storage. Do people realize only really Big Data benefits from tape? Unless you write multiple petabytes of data that you rarely need each day, tape is inferior in every way to hard disks. A huge corporation that counts every cent and needs lots of cold storage might benefit from tape, and companies still prefer better storage ( https://engineering.fb.com/core-data/under-the-hood-facebook-s-cold-storage-system/ https://engineering.fb.com/core-data/under-the-hood-facebook... ). "Good enough for Google, good enough for you"(a lie that neglects economies of scale that force Google to use tape). https://www.overlandstorage.com/blog/?p=323 https://www.overlandstorage.com/blog/?p=323

adrian_b 7y ago
No, tapes become preferable to HDDs at less than 100 TB written, not at multiple petabytes. For many applications and for many people, including for myself in my home, it makes absolutely no sense to use HDDs for anything, but only SSDs & tapes. For long term storage tape is superior in every way to HDDs, while for fast access SSDs are superior in every way to HDDs. Obviously, because the drives are extremely expensive, whoever needs no more than 20 or 30 TB of storage cannot afford tapes and must use a few HDDs. The best storage technology changes in time. 20 years ago I was using optical disks and 10 years ago I was using HDDs, but since I went back to tapes, like I was previously using 30 years ago, I have saved a lot of money and I have much less worries about the HDDs becoming defective.
- FrozenVoid 7y ago
  How does it compare with 8TB USB3 drives(140$)? https://pcpartpicker.com/product/YpJtt6/western-digital-elements-8tb-external-hard-drive-wdbwlg0080hbk-nesn https://pcpartpicker.com/product/YpJtt6/western-digital-elem...
  - adrian_b 7y ago
    LTO-7 tapes are $10 per TB at Amazon (of course uncompressed), so they are still cheaper, but this $17.50 USD per TB is indeed the cheapest HDD that I have ever seen. Nevertheless, these HDDs are much more expensive than they look at the first glance. The real cost of archival storage is given by the price per TB and per warranty year, because after the warranty time you must replace the storage medium. I have seen countless HDDs that have died just a few months after their warranty time expired. So these drives with 2-year warranty cost $8.75 per TB and per year. The warranty time of the tapes is 30 years, but in fact you must replace them much faster because you will no longer find tape drives able to read them. For LTO tapes, you should expect to replace them after 6 to 10 years. Even with only a 6 year lifetime, the price for tape would be $1.67 per TB per year, i.e. more than 5 times cheaper than those WD HDDs. If you want to just have some backups against accidents, which you expect to erase after a few weeks, then those HDDs could be a good choice. For archival purposes they are still much more expensive than tapes.

not_a_cop75 7y ago

And why wouldn't it? Tapes as far as I know cost pennies to manufacture. It's easily over 90% plastic in most cases. The low quality ingredient on a low end tape is probably still ferric oxide - not exactly an expensive chemical if I know anything about chemistry. Perhaps cobalt is still heavily used as part of the coating as well.

ryanmercer 7y ago

It never ceases to amaze me how much tape I still see imported, mind you I see way more SSDs, then HDDs but still a fantastical amount of tape.

stelonix 7y ago

Are there any cheap tape drives? If not, what's the issue? Is it the fact the prices are kept high because it's used mostly by business/enterprises customers? If so, are there any hopes of LTO becoming mainstream for home desktop usage? Couldn't find much info on Google about this.

NeedMoreTea 7y ago
DAT (Well DDS) used to fill that niche. Used to be pretty reasonably priced - for tape. Went for a few hundred a drive, an order of magnitude less than LTO. Smaller capacity each generation though, but is an older standard. Disappeared for no apparent reason about a decade or so ago.
- deleted 7y ago
  [deleted]

craig1f 7y ago

This is the principle behind AWS Glacier

bufferoverflow 7y ago

That's if you don't count the extra time required to deal with tape backups.

idlerboris 7y ago

It's crucially important to take managing and lifetime of such storages into account. I doubt it can fight with SSD or HDD.

scraft 7y ago

We had a Ditto drive and a series of tapes, bought circa 1996. We used the software it came with on Windows to perform full back ups of the machine, cycling between a set of tapes (maybe 6). Backup would complete overnight. Never had to restore from it. It was a consumer friendly, easy to use product and cheap. My father was an accountant that ran his own practise, so needed to know his files were safe. That went from carbon copies into a second filing cabinet, to 5 1/4" floppies for groups of clients, to 3.5" floppies, to Ditto tapes to USB sticks to Microsoft Cloud storage (which sinks between laptop and PC). The oldest days of filing cabinets and floppies were easy, the middle days of Ditto and USB storage was more worrying, and the final days of cloud storage went back to being simple again. Now he is retired so none of it matters. This was all donkeys years ago, so it is interesting to hear how things have changed.

vonseel 7y ago

As someone totally unfamiliar with tape aside from knowing some places still use it for archival and recording audio to tape - something totally different - is incredibly expensive and most (if not all?) recording tape is out of production, I’m surprised storage tape is still a thing. How does storage tape even work? Is this literal tape in a cartridge much like tape in a cassette tape? I assume we’re not talking about data stored in 1s and 0s? Is this technically an analog medium!? Going to do some googling now to quell my curiosity. The article does seem targeted at consumers. I’m going to guess I’d need some expensive hardware to even use one of these $60 tape drives.