25 ms·
Fujifilm Created a Magnetic Tape That Can Store 580 Terabytes
- dijit 6y agoGreat, so when can I buy one and how much will the tape drive cost? The largest issue with tape is that the drives themselves cost absurd amounts, and you better not cheap out because failure is both time consuming and scary. Swapping tapes continues to be human intensive and restore times long. But the tapes themselves are so cheap that at this scale it becomes worthwhile again.
- vondur 6y agoThis is geared for enterprise settings, not home use. I believe someone mentioned these drives costing $25,000. I do agree reasonably priced tape drives with TB of space for home users would be great.
- weare138 6y agoThe prices are much cheaper now. You can get a single tape drive for under $300 and enterprise grade single tape drives for under $3k. https://www.newegg.com/p/2BM-000A-000M5 https://www.newegg.com/p/2BM-000A-000M5 https://www.provantage.com/quantum-tc-l72an-br~7QUAT0JW.htm https://www.provantage.com/quantum-tc-l72an-br~7QUAT0JW.htm
- monocasa 6y agoThat $300 drive is an LTO-2 drive. It's so old that it's parallel SCSI. I'd be surprised if you could even get tapes for it that aren't sketch grey market stuff that you wouldn't want to use for backups anyway.
- segfaultbuserr 6y agoOne day I found a really cheap LTO-4 drive that costed $150. Interesting, but I found it was not practical despite the price and decided against it. First, an 800 GB LTO-4 tape is no longer high density by today's standard, it couldn't even hold a 1 TB HDD image, also, I still had to pay for the necessary SAS peripherals to get it working, finally, the mechanical assembly inside a 10-year-old tape drive was not something that inspires confidence for data backup... Last time I've looked up, the cheapest decommissioned LTO-5 drive still costs $1000.
- prirun 6y agoThe first tape drive is a 200GB (native / uncompessed) internal SCSI drive. Not so easy for a consumer with a laptop to use. The tapes are pretty cheap at $8 on Amazon if you buy 20.
- weare138 6y agoWell they don't really make these things for laptops unfortunately but they also don't cost $25k+ like they used to. It's still more than the average consumer is willing to spend but they are affordable enough now to be a viable option for professionals and small businesses.
- ardy42 6y ago> The prices are much cheaper now. You can get a single tape drive for under $300 and enterprise grade single tape drives for under $3k. Those tapes only hold 200 GB, and you can get a 10 TB hard drive today for less than $300: https://www.newegg.com/red-wd101efax-10tb/p/N82E16822234407 https://www.newegg.com/red-wd101efax-10tb/p/N82E16822234407 While tapes are theoretically cool, the the drives are just too rare for them to be of any practical value to a home user. Even if the media has a better archival life than a hard disk (say 50 years), it won't do you a damn bit of good if there are no drives available in 50 years to read it. IMHO, hard drives are better for backup (even offline backup, just get a hot-swap bay and imagine the drive is tape [1]). Archival is a harder problem, but I've settled on using high-grade optical media, burned slow for fewer errors, making a bunch of redundant copies. Even though the media might not last as long, I'm pretty much guaranteed to be able to find a drive for the next several decades [2]. [1] "pretend this hard drive is a tape" is even an enterprise product: https://en.wikipedia.org/wiki/RDX_Technology https://en.wikipedia.org/wiki/RDX_Technology [2] For instance, you can still attach a 5.25 floppy drive to a modern computer: https://www.ebay.com/sch/i.html?_nkw=5.25+floppy+drive https://www.ebay.com/sch/i.html?_nkw=5.25+floppy+drive + http://www.deviceside.com/fc5025.html http://www.deviceside.com/fc5025.html
- klodolph 6y ago> Those tapes only hold 200 GB, and you can get a 10 TB hard drive today for less than $300: That was a link to an obsolete LTO-2 drive. If you’re using LTO-8, which is the current generation, you’d get a 10-pack of 12TB tapes for around $500. Noticeably cheaper per byte than hard drives. I don’t recommend LTO-2 just because I don’t think that the drives are well-supported. > While tapes are theoretically cool, the the drives are just too rare for them to be of any practical value to a home user. Even if the media has a better archival life than a hard disk (say 50 years), it won't do you a damn bit of good if there are no drives available in 50 years to read it. If you’re evaluating just on the basis of the price of drives / media, then there’s a cutoff where tape becomes cheaper than hard drives. The easy way to calculate this cutoff is to divide the overhead cost (tape drive price) by the difference in the cost per TB of tapes and hard drives, which in the example here, is around $3k divided by $25/TB, or 120 TB. In other words, if you have more than about 120 TB of data, then it is cheaper to buy a tape drive. I think any comments about whether tape drives are suitable for home use are really comments about whether you are interested in the use case of people who need to store >120 TB at home. If you are running a YouTube channel as a hobby, or you run a side business doing videography for weddings, the tape drive starts to sound a lot better. The 120 TB cutoff might be around 300 hours of video, which might be only 50 events. There are a lot of other reasons why you might NOT want to use tape, but it’s easy to have enough data that tape is the cheapest storage option. At “enterprise” scale the cost calculus is completely different and involves things like support contracts with Oracle (not necessary for hard drives), power/cooling in your DC (tape is very low-power), etc. And let’s not forget that if you have 120TB of hard drives, you’re in the regime where you start having to buy multiple machines. As for longetivity—I don’t have the data handy. If you are storing data on tape, you need to migrate to newer generations of tape, as old generations become obsolete and unavailable. If you are storing on hard disks, you also need to migrate because hard disks eventually fail (even if they are not powered on).
- dijit 6y agoWhat makes you think I meant home use?
- throwaway2245 6y agoThe first person pronoun you used ("I") strongly implied personal use.
- jtxx 6y agoNo, it just means the person has the ability to purchase. They could be purchasing it for themselves or on the behalf of another organization.
- throwaway2245 6y agoI accept that, in a literal sense. I was attempting only to answer the question and explain the source of confusion. In native English speech, it would usually be expected that a business-case purchasor would speak of a more general case, perhaps by using "we".
- jtxx 6y agoAh - I took the question as saying “I did nothing to indicate the sort and could in fact be purchasing for a business”, so I was trying to explain that side!
- Blikkentrekker 6y agoIn English, when purchasing on behalf of a corporation, “we” would almost always be used. “we” is even frequently employed by auctors writing papers to depersonalize.
- jtxx 6y agoI guess it’s just me then. I wouldn’t write “we” in a comment online without explaining who “we” is, it seems like an unnecessary detail to explain
- chrisseaton 6y agoDoesn't really sound like it's aimed at you. Tape is for serious long-term, high-volume storage. If you've got a limited budget for the tape device it's probably not designed for you in the first place.
- throwaway2245 6y agoI understood the exact opposite that you have understood, and I find your comment quite uncomfortable, as a result? The parent is clearly seeking to confirm serious long-term use, as you say, in considering the costs of failure cases.
- chrisseaton 6y ago> I understood the exact opposite that you have understood, and I find your comment quite uncomfortably presumptuous, as a result? Is this a question? I can't tell you if that's how you find my comment or not, sorry. They wants a cheap tape device because they just want to use it in the home. Tape devices aren't cheap... because they aren't aimed at use just in the home. They aren't aiming at home users - I think that's a simple fact not presuming anything.
- throwaway2245 6y agoThe parent clarifies above that they are not a home user. Even if you believed they were a home user, they clearly demonstrated in their original comment that they did know that it was for long-term, high volume storage, and they did know the cost, and you appeared still to belittle them for not knowing. It's a question in the sense of - why did you choose to say it in this way, which seemed impolite? What did you intend your comment to add?
- GordonS 6y agoIs it possible to buy tape drives in the 2nd hand market for a reasonable price?
- bombcar 6y agoIt is - the kicker is (and always has been) the media.
- klodolph 6y ago> Swapping tapes continues to be human intensive and restore times long. Restore times are definitely long, but you can mostly avoid human labor by putting the tapes in a tape library, which will load tapes in the drives using robots. You still need technicians around because tapes / drives / robots will break, but individual restore operations can be completely automated.
- jariel 6y agoWe reached an inflection 'amazing!' point when we are able to put '100 songs in your pocket'. That was really a shocking thing given the limitations of CD/Tape etc.. But the real inflection point may come when 'all relevant information in your local disk'. The tapes here in question can maybe store every book every written! We may be able to put the entirety of Wikipedia, every film, TV Show ever made, every book every lecture on a little disk. The only thing we'd need to access in realtime would be contemporary data like traffic flows, weather situation etc.. The ability to store 'that much data quickly and easily' locally, may quite fundamentally change the equilibrium we have right now with the cloud and lead to a more natural decentralization. "All of YouTube from 2008 until the present, 99 cents at the local gas station, on usb-like contraption"
- avdlinde 6y agoLove that last line! Wonder what will happen drm wise in the future if we get to this point.
- fiddlerwoaroof 6y agoBuying something would just transfer the encryption key
- jaynetics 6y agoI love this thought, but have some doubts. So far, storage is still following something resembling Moores law, but it will probably hit physical limits way before a year of youtube (probably 30 of these new tapes or so) fits into your hand.
- barbacoa 6y agoHDDs are hitting physical limits, SSDs still have growing room. You can already buy 100 TB SSDs. https://www.newegg.com/nimbus-data-dc-100tb/p/2U3-002M-00004 https://www.newegg.com/nimbus-data-dc-100tb/p/2U3-002M-00004 Following Moore's law of 2x density per 18 months, in 10 years we will see single drives of: 100 tb * 2^(10 years/1.5) = 10,159 TB
- webmobdev 6y agoThat's good. But when will it ever move out of the enterprise market? What I really want is an affordable Tape storage for us common folks / home users to archive data long term. It's really a pain to keep transferring your old backups every 2-5 years to a new CD / DVD / portable HDD. There is a market for home users that really don't want to put their data on the "cloud". And this generation is really creating so much data, some of which, I am sure they would like to store long-term.
- chrisseaton 6y agoI store on an external HD, and every year I buy a new HD and copy across to it and verify checksums against my records. Costs basically zero and low-effort. What am I missing?
- sgt 6y agoEvery year you buy a new HD and yet the cost is basically zero?
- chrisseaton 6y agoYeah what does a new HD cost? Like £100 max for a big high quality one? Compared to buying a £5k tape drive and expensive tapes... yeah that's basically zero.
- jjulius 6y ago$100 is definitely not "basically zero". Not only that, but a $5k tape drive and tapes are designed to last a significantly longer period of time, essentially bringing the cost of long-term storage closer to $0/year than your option of spending $100/year.
- aidenn0 6y agoIt's 50 years before $100/year catches up with a one-time cost of $5k. On top of that, the total capacity of the mediumgoes up steadily over time, while you will need to drop another $5k every 10 years or so or span your backups across multiple tapes.
- londons_explore 6y agoLast I looked into this, if you had a lot of data to back up (Say 1 Gbyte per second, continuous, with a retention time of 1 year), it was still far cheaper to simply use hard drives. One employee can keep up with all drive replacements, hardware setup, etc with time to spare. Drives aren't super power hungry, so any old office building is suitable. Encrypt the drive contents on another site so you don't need 24/7 security. Total system cost was sub $1M with a running cost of $500k/year and storage of 50PB. Bargain. Now that GDPR applies, most companies need to rewrite backups every 30 days anyway to remove data where a GDPR deletion request applies. That tips the scale further in the direction of always spinning hard drives. Just hook up 64 drives to each machine, make sure you only do streaming writes of 1Gb+ files, do some ZFS raid-like scheme, and away you go.
- jeffbee 6y agoThat didn’t add up. 1GB per second for a month is 2.6PB. Do you have a source of 40TB disk drives?
- londons_explore 6y ago1GB/second for a year... Plus spare drives for redundancy, hotspotting, a test/training setup, and unplanned for increases in storage requirements.
- kaslai 6y ago"to each machine" implies that there's more than one machine with 64 drives attached.
- pessimizer 6y agoAnd I'm pretty sure "any old office building" means buying a large office or small office building and filling it up with them.
- Znafon 6y agoYou don't have to do it this way, you can just use encryption at rest with a different key for each user and throw the key when a user ask for deletion of their data. No need to get all the backup back to scrub them one by one.
- anonu 6y agoIf the tape was twice as long, it could hold 1.16 petabytes. That would have been a more interesting headline.
- w0mbat 6y agoMy reaction was that any kind of tape can hold 580 Terabytes if it's long enough.
- rahimnathwani 6y agoHow long would the tape from an audio (compact) cassette need to be, to hold 580TB? How thick would that tape need to be to allow the motorized spool to pull the tape from the other (passive) spool? Imagine you were to make an assembly that could hold two spools of this tape, how large would it be? It might be hard to fit it through the door of your server room. And it might not even fit in the van that carries away your off-site backups :)
- fernly 6y agoYeah! Somehow my brain started to channel Shakespeare and say "Why, so can I, and so can any man, be the tape but long enough!" (Then I had to look it up. Henry IV Part I, Act III, Glendower: I can call spirits from the vasty deep; Hotspur: Why so can I, and so can any man, but will they come when you do call them?)
- galkk 6y agoMost likely there are standard sizes of containers.
- linuxlizard 6y agoFinally a way I can store all those family photos I swear I'm going to sort through someday!
- ashton314 6y agoI remember watching the pilot episode of Star Trek (the original series) and chuckling when Spock reports that “the tapes are badly damaged” from the capsule they recovered near the barrier at the edge of the galaxy. Turns out it might not be that outdated after all.
- LASR 6y agoAbout a month ago found a stack of MiniDV tapes from about 15 years ago. It was my own home videos. I wanted to preserve them, so wanted to upload them to Google Photos. It took me some looking around in eBay to find a camcorder to play these back. When it came home, I realized that I needed a FireWire 400 port to capture full resolution. So more digging around for a FireWire PCI-E card. I was finally able to transcode and upload some 15GB worth of video. It took 3 minutes on my gigabit internet for the upload. About a month for the whole process acquiring the hardware etc. When I think about these tapes some 50 years from now, it might as well be completely unrecoverable, not because the tapes went bad, but because we have nothing to read them. Makes you wonder about galactic time scales like in Star Trek.
- peteradio 6y agoAnd in 2 years those tapes might be the only thing you got.
- jeffhuys 6y agoCould you elaborate?
- imoverclocked 6y agoMaybe that he uploaded them to an unpaid service which might make a policy shift to remove the data at some unknown time in the future?
- sciurus 6y agohttps://blog.google/products/photos/storage-policy-update/ https://blog.google/products/photos/storage-policy-update/
- fernly 6y agoAw, c'mon chaps! Can we just admire the tech for a second? > Data on the tape is stored at a record-breaking density of 317 gigabytes per square inch... > When tape is being read it is streamed over the head at a speed of about 15 km/h and with our new servo technologies we are still able to position the tape head with an accuracy that is about 1.5 times the width of a DNA molecule.
- daRealDodo 6y ago> Data on the tape is stored at a record-breaking density of 317 gigabytes per square inch Me wondering about those micro SD cards that can store 1Tb.
- VoxPelli 6y agoIt probably references the density within the data tape world, which is significant as there could be other ways to achieve a higher total storage, but this is one of the major components here it seems
- luma 6y agoMaybe around an order of magnitude more. The big difference is that square inch of tape might be 3-5 orders of magnitude cheaper to manufacture.
- deleted 6y ago[deleted]
- Widdershin 6y agoFair enough but chances are the tapes will have a much longer lifetime for storing data, which is the primary use case for this sort of thing.
- reitzensteinm 6y agoThose SD cards aren't close to a single layer, so it's not really equivalent. Ad absurdum, stacking two Micro SD cards on top of one another hasn't just doubled your density even though surface area is unchanged.
- zw123456 6y agoMercifully, there was no requisite simpleton analogy like "that is the equivalent of stacking VHS tapes to the moon" or some idiotic thing like that.
- toddmatthews 6y ago>To put 580 terabytes in perspective, it’s roughly the equivalent of 120,000 DVDs or 786,977 CDs — IBM notes that stacking that many CDs would result in a tower 3,097 feet (944m) tall, or taller than Burj Kalifa, the world’s tallest building.
- tra3 6y agoIs that just the media or the media + protective jewel cases?
- phpnode 6y agoJust the media
- PopeDotNinja 6y agoHow many CDs can you stack on top each other before the one at the bottom fails?
- throwaway189262 6y ago> when not in use, tape requires no energy unlike hard disks and flash, IBM writes IBM marketing smort
- dmitrybrant 6y agoYou mean you haven't been keeping your hard disks plugged into a power source at all times? Say goodbye to your data! /s
- peter303 6y agoLess than a day's new TikToks :-)
- sedatk 6y agoI'm more interested in the lifetime of the medium. Durable backup media for consumers is still a holy grail as I understand. M-DISC didn't hold up to their 1000 year lifetime promise. Archival-grade DVD's are also not good enough as I understand. Syylex went bankrupt. I want a consumer-grade backup medium that can provide at least a 100 years of lifetime. That said, I was able to recover 90% of the voice recordings my father made between 1959-1963 on reel-to-reel tapes 60 years later. Tape can be very durable but what I recovered was analog voice very tolerant of errors. I'm not so sure about gigabytes snuck into an inch-square.
- dudus 6y agopiql is the company that did the github arctic vault. And they offer it as a service. Maybe not for general public but still . https://www.piql.com/arctic-world-archive/ https://www.piql.com/arctic-world-archive/
- sedatk 6y agoArctic vault is VERY expensive, about $100 per gigabyte.
- elorant 6y agoThe lifetime of the medium is half the equation. The biggest problem imho is the lifetime of the device. Say I give you an old floppy drive for 8" floppies from the seventies. Where would you connect it?
- sedatk 6y agoYou're right but not all media are the same. Arctic Vault used an optical film with QR codes. You can theoretically even take its photo and decode it by hand if you want. They even added a Rosetta Stone to the entrance so, even if all the knowledge is lost, one can hypothetically decode the data stored there. For magnetic media, you need some more specialized equipment for sure.
- kmeisthax 6y ago
- watersb 6y agoI wonder about using these for science data archival, although these days I suppose that most projects can just provision archival from a commercial vendor.
- VoxPelli 6y agoA vendor like what AWS does with Glacier
- analog31 6y agoTime for the ever golden quote: Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway. https://en.wikiquote.org/wiki/Andrew_S._Tanenbaum https://en.wikiquote.org/wiki/Andrew_S._Tanenbaum
- unwoundmouse 6y agoDoing some guestimation, it seems like you could fit an exabyte into an AWS snowmobile. That's insane
- meonkeys 6y agoWhy guess? https://aws.amazon.com/snowmobile/ https://aws.amazon.com/snowmobile/ says 100PB.
- diveanon 6y agoWhen I was working in vfx we employed a guy who's sole job was to fly back and forth from LA to Melbourne with pelican cases full of hard drives. It was faster and more secure since we were working on air gapped networks.
- raihansaputra 6y agoThat feels like a dream job for a temporary amount of time. And very effective too. I assume the VFX studio is in Melbourne?
- bronco21016 6y agoMaybe if they’re flying first class? Otherwise that sounds like a nightmare that would require an insanely high paycheck.
- Raidion 6y agoIf you're flying back and forth on that route that often, you're going to get upgraded quickly, I would imagine. I've done 6 flights a month domestic and have been upgraded to first class for free fairly often.
- Blikkentrekker 6y ago> In addition, when stored properly, data recorded on tape today will still be readable in 30 years. I do not find this to be all that impressive, frankness be.
- codesnik 6y agomagnetic film!
- noisy_boy 6y agoQuestion to those who are familiar with latest tape backup performance: does it make sense for me to take tape backups of my data (e.g. using a second hand tape drive)? Or its a bad idea for home use?
- NiceWayToDoIT 6y ago580 Terabytes is really impressive. Although, I do not know why I am still waiting 5D crystal tech (maybe durability), and hopping it will be market available some time soon For now Microsoft has taken over the project and status is quite uncertain ...
- supernova87a 6y agoHow does tape reading/writing wear out the physical medium? Any practical limits?
- AleMonte 6y agoThat's a great effort in technology and it is remarkable. I have some doubt about the real benefits that are beyond the technology though. First of all about the security of this system. Yes, it is very avantgarde but being a physical support I will always have the worry that it can be damaged. If I immagine to use it to store sensitive information this worry rise up. It can easily be accessed or stolen or damaged. And what about the possible cost of a solution like this? a part from this aspect it is incredible what Fujifilm has been able to reach in terms of tech
- jl6 6y agoCurrent LTO-9 tapes can store 18TB and the LTO roadmap doubles capacity about every 3 years. So this tape tech would be on the same scale as we might expect LTO-14 to offer in 2035. So unless this is an incredibly radical breakthrough, that’s the timeframe I’d expect for the headline to become a real product. Note that the article shows a table from IBM claiming they achieved 35TB-per-cartridge capacity in 2010, and that still isn’t something you can buy.
- iso947 6y agoWhy do LTO manufacturers always quote compressed sizes, like we’re in the days of stacker?
- jl6 6y agoIMHO, partly for marketing bluster, but partly because hardware compression in the tape drive is a useful feature if you don’t want to handle 1000MB/sec of compression workload on the host during backups.
- ACAVJW4H 6y agoMemories of windows disk defragmenter come to mind. Might take a while