10 ms·
New 5D Storage to Offer 10,000x the Density of Blu-ray
- kingcharles 5y agoTo quote a comment from Reddit: "I can't wait to never hear about this again."
- dividuum 5y agoYep. Still waiting for my tesa drive.
- throwawayswede 5y agoWho comes up with garbage naming techniques such is 5D? What the fuck do they expect people to think when they hear that?
- GhettoComputers 5y agoIt’s magical storage that goes into the multiverse. Technology is magic!
- mattowen_uk 5y agoThis is all well-and-good, but how do we read all that data in 500 years time, or even 20 years time? People were busily archiving data to CDs and DVDs two decades ago, thinking 'yeah this digital storage media will last for centuries!'... Loads of bit-rot combined with complete lack of DVD drives still in common [computer] usage, has totally shown that the more complex the storage media the shorter it's useful lifetime is. Compare this to books, which can still be read a millennium later.
- 0des 5y agoLet's not forget actual rot too. I peered into a stack of CD's not long ago to find that a lot of them have quite literally rotted while sitting in a dark and dry closet. No dust, no humidity, no smoking, no abuse, just a spindle of CD's recognizable only by their sharpie'd identifiers covered in pits of rotted foil.
- throwaway946513 5y agoTemperature? I've yet to experience rot on decades old discs at home. Most sit in a basement that doesn't exceed 68*F
- 0des 5y agoA constant 72F, this is an indoor closet, top shelf, no moisture incidents.
- MivLives 5y agoIt's a well known problem: https://en.wikipedia.org/wiki/Disc_rot https://en.wikipedia.org/wiki/Disc_rot
- TacticalCoder 5y agoYup it's a well-known problem, but several of us still have CDs decades old that still do read totally fine. That doesn't mean we rely on them as backups: mine have long since been copied to DVDs, then more recent DVDs but also backup'ed on harddrive / servers etc.
- rbanffy 5y agoCDs and DVDs are pressed. Recordable media is not and, with the wrong conditions, will rot. DVD-RWs will get corrupted if you look funny at them.
- mixmastamyk 5y agoI still have music CDs from 1985 and CD-R from 1995 in perfect condition. None of them have gone bad.
- throwaway946513 5y agoThe linked article also explains: > DVDs are more resistant to rot > DVDs and CDs use aluminum layers for the reflective 'data encoded' layer > Discs using a gold layer are extremely resistant to corrosion and thus disc rot > Blu-ray discs use a silver-alloy for their reflective layer. Though not linked, or stated, it is quite likely that the BR discs are significantly less likely to be damaged due to disc rot for similar reasons as gold-layered discs are, though not necessarily as resistant as gold. Disc rot though appears to stem due to the damage of the lacquer or plastic layer on discs, which if the disc then remains undamaged, it should not be as susceptible to rot.
- MisterTea 5y ago> This is all well-and-good, but how do we read all that data in 500 years time, or even 20 years time? If the data is important enough, we'll find a way. Perhaps by then camera phones will have sensors with quantum resolution that can capture the optical pits in one snap and use AI or whatever to reconstruct the bits from the image on whatever magical storage exists.
- bufferoverflow 5y agoThere's no lack of DVD drives, they are widely available on Amazon, BestBuy, eBay.
- kipchak 5y agoI think right now the most permanent way to store digital data is have enough redundant copies on enough impermanent media one is likely to survive. For example my CD copy of 1998's HARDWAR might be lost due to disk rot, but there's enough copies of copies of copies floating around on various places it's still easily accessible.
- TacticalCoder 5y ago> This is all well-and-good, but how do we read all that data in 500 years time, or even 20 years time? I know it won't last due to bit-rot but all my CDs burned in the nineties are still readable. > Loads of bit-rot combined with complete lack of DVD drives still in common [computer] usage... They are still very common: both in desktop and as USB-connected readers/burners. You can find countless models on Amazon. I don't think they're going away anytime soon. > People were busily archiving data to CDs and DVDs two decades ago, ... That's precisely why they're not going away anytime soon. There are even companies like FB would, at some point, were burning insane amount of data to Blu-Ray discs. > Compare this to books, which can still be read a millennium later. Having a book from 1575 here I do agree (not quite a millenium but still). However the solution to disc bit-rot is simple: once every ten years or so, read your discs and burn them on the new fancy tech of the day (say CDs to DVDs, DVDs to Blu-Ray, now Blu-Ray to these 5D thinggies, rinse and repeat). It's not as if new mediums had less capacity than older ones.
- nomel 5y ago> I know it won't last due to bit-rot but all my CDs burned in the nineties are still readable. Were these the more expensive archival disks? I lost all ~50 of my old CDs and DVDs to rot, within 10 years. They were middle range Memorex.
- GhettoComputers 5y agoWhat kind of data is even worth storing? If it’s text based it’s not hard and it’s not very large. If it’s media it’s probably some uncompressed videos and pictures that are overly large, phones that take 16mb photos that aren’t going to have more useful information over compressing it to 400kb. I am sick of my phone taking large photos that don’t have any clear benefit to me aside from clogging my phone with unreasonably large files that don’t look better, Apple wants to sell me iCloud by not allowing me to take lower resolution images.
- vel0city 5y agoOptical drives aren't common in most consumer devices but that isn't really a hurdle if you're trying to recover archived data on optical media. If you're only worried about DVDs a USB DVD drive can easily be had for ~$20. Blu-Ray drives aren't very expensive, I included one in my last desktop build for the purpose of data archival. Many consumers figured optical disks would last a long time but many knew about disc rot from the get go. I picked optical media that marketed itself as archive quality. My oldest discs are reaching 20 years old now and I've lost less than 10% of the discs I've burned that were burned on good discs. These days I burn data to M-DISCs which use a ceramic layer instead of organic. They're rated to 1000 years and the Blu-Rays are quite durable. I've had one bouncing around my desk and outside for several years. The one on my desk has zero errors while the one outside has a few bit errors.
- wazoox 5y agoSome guys I know propose a storage on 35mm-film solution that looks pretty foolproof for a couple of centuries at least. https://digifilm-corp.com/preserve-your-works-for-eternity https://digifilm-corp.com/preserve-your-works-for-eternity To read back the data, you need something to digitize the film (a flatbed scanner would work, I've tried, or probably even a smartphone with a a DIY magnifier), then something to decode the data (the necessary source code is provided in clear on the first images of each roll).
- nomel 5y agoI'm having trouble finding the data density. How many lbs of film/GB?
- wazoox 5y agoIt's not very dense at the moment, around 600KB per frame IIRC, with lots of redundancy and while staying really easy to read even with DIY hardware. They think they should be able to cram about 2MB per frame without compromising durability and without any compression (but lots of in-picture redundancy). That should give about 200 GB per roll of film (the large ones).
- rbanffy 5y agoIt all depends on how technology changes. We can still read floppies written in the 70s with essentially a motor, something magnetic and a wire going around it (and a computer reading the currents). I’m pretty sure we’ll know how to cobble together an optical microscope and camera device to read at least one bit out of every volume cell. Let’s just hope the data format is simple enough.
- louwrentius 5y agoArchiving data is a process, not a one-time effort. You can't just store data on a medium and hope if that medium will last. You have to periodically check for defects and from time to time, transfer data to fresh media. At some point, you may even need to convert the data to keep it readable...
- MisterTea 5y agoThe other day I was talking with a friend about the high cost and perceived fragility of tape drives for long term personal archival while bemoaning the loss of higher density optical storage technology. It's not easy to backup or shuffle around terabytes of personal video, audio and pictures. Hard drives aren't exactly the right media for safe deposit boxes. In the 90's I had an Iomega Ditto 250MB drive that held 120MB per tape for $15-20 per tape and my hard drive was 540MB. This was when hard disc storage cost $1000/GB. The tape drive had a quarter the capacity of the hard disc and was cheap by comparison. It was practical. Later when CD burners came out they offered 650 then 700MB when hard drives were less than 10GB. Then they became too small and DVD came out but shortly was also passed up as 4.7GB wasn't much storage. Bluray was DoA with too little storage to be of real interest with a paltry 25-50GB. I'd love to see CHEAP >=1TB optical discs with a write speed of 100MB/s allowing us to burn a disc in ~10 minutes. I'd happily pay $1 a disc in bulk and upward of $1000 for a drive. I'd also happily pay $10+ for an archival disc that has a guaranteed shelf life of 50+ years. Bring back the 100 disc juke boxes while we're at it ;-) Cheap storage for everyone!
- zitterbewegung 5y agoWhy are tape drives fragile? I use them for my long term backups of personal files I thought they were robust ?
- jl6 5y agoWhat scale are you working at and how have you made it cost-effective versus a stack of external HDDs?
- adrian_b 5y agoI am not that poster, but I also use tapes. One may save about $14 per TB by using tapes instead of external HDDs (which strangely have become cheaper than internal HDDs). In order to recover the cost of the tape drive, you need to write about 200 TB. I have about 50 LTO-7 tapes, i.e. about 300 TB of compressed archives, so I have recovered much more than I have paid on a LTO-7 tape drive, 5 years ago. Nevertheless, even if would have written only 100 TB, I would still have bought the tape drive, just for the peace of mind, because after the optical discs became too small I have used HDDs for archival storage, but after a few years in cold storage they become unreliable. If I had not stored duplicates for each HDD, I would have lost a lot of data. Of course, someone who needs to archive just 20 TB or even 40 TB of raw data, can buy just 2 or 4 HDDs and store each file on 2 HDDs and that should be OK. For 100 TB or more of raw data (i.e. 200 TB, when counting the redundancy, because also for tapes you should store at least 2 copies, preferably not in the same location), tapes become cheaper.
- gjm11 5y agoIn case anyone is wondering what they mean by "5D": they mean it's a 3D array of marks, each of which has both size and orientation. The 3D array is three 2D layers. According to this definition, a DVD or Blu-Ray disc is "4D" because it can have multiple layers each of which stores 1-dimensional data (pit versus no pit). And a book is at least "7D" because it contains a 3D arrangement of marks, and marks can vary in (at least) what letter they are, how large, how bold, and how slanted. (It would be easy to add to that list.) I would describe this thing as 2.5D storage: it's basically 2-dimensional but with multiple layers. If it's genuinely 10000x denser than Blu-ray, great! If it's likely to last longer, also great! But there's no need for this "five-dimensional" bullshit. (It looks to me as if what they have at the moment is not anything like 10000x denser than Blu-ray. I'm not sure it's even 1x denser than Blu-ray at present.)
- chmod775 5y agoA better definition then would be: Assuming each dimension is infinite, how many dimensions do you need to uniquely address a bit. The infinite requirement is there so the answer can't be reduced to "1 dimension" every time (if you had a 10x10 matrix you could just turn it into an array of 100 elements).
- chriswarbo 5y agoSlight nit-pick, but any number of dimensions can be reduced to one, using a pairing function https://en.wikipedia.org/wiki/Pairing_function https://en.wikipedia.org/wiki/Pairing_function or a space-filling curve https://en.wikipedia.org/wiki/Space-filling_curve https://en.wikipedia.org/wiki/Space-filling_curve
- chmod775 5y agoRight. I don't immediately see how either of those would work for uncountable infinite sets, but it may be possible. Clearly some other limitations are required to make "dimensions" a useful metric in this context.
- louissan 5y agoLeto II approves of this technology :-)
- rdevsrex 5y agoYeah, my first thought even before reading the article, was "What about seek times?". Maybe it can be improved, but thinking of web apps specifically, I'm not sure what kind of use case you would have for something that is this slow to access.
- EastOfTruth 5y agoI don't care about seek times if we can have media with orders of magnitude more storage capacity and life expectancy. (of course if you can also have good seek times, it would be icing on the cake)
- GhettoComputers 5y agoWhat kind of data are you trying to store? How much of it? I can’t imagine my life being more than 100 GB and probably much smaller, probably under 20GB.
- EastOfTruth 5y agoScihub, libgen, datasets, etc... as much data as possible. My personal data is probably less than 1TB at this point, but a way to store it long term would be nice.
- azinman2 5y agoThis is for archival purposes. Seek times don’t matter for that use case.
- Hamuko 5y agoSeek times always matter even if they're not the top priority. And especially if we're talking about a storage medium that takes in a whopping 500 TB.
- azinman2 5y agoThe GP is talking about something interactive like a web app, where a seek time likely would matter. This isn’t intended to be used as such.
- Grustaf 5y ago> promising incommensurate storage in a palm-sized device Incommensurate with what, one might ask.
- tmikaeld 5y agoThey do mention parallell writes (multiple lasers at the same time) which would give writing speeds of 96MB/s (500 TB in 60 days). Which isn't half bad, certainly on pair with standard mechanical hard-drives.
- tiffanyh 5y agoOff topic: if I wanted to digital store files to pass down to my children that might not be access for decades, what's the best medium to store such data? Floppy/ZipDrive/CD-ROM/Blueray/USB-A are either obsolete or are on their way out. What's a future proof storage option? (and if you said Cloud Storage {X} - just look at how many services have folded in just the last few years alone). Secondly, what's the best file formats also that will be future proof?
- GhettoComputers 5y agoWhat kind of files and how much?
- IndexCardBox 5y agoI'd hedge my bets and store it multiple ways in multiple formats. Including physical (print off pictures, diaries, etc. on acid free paper with acid free inks)
- mixmastamyk 5y agoNo such thing. Other poster had a good idea, duplicate everything every ten years. I suspect flash drives will have the best longevity in the medium term however.
- zamadatix 5y agoAs far as formats stick to either very popular or very simple. E.g. for video in 100 years time there is still going to be a way to be able to decode h.264 videos just because of the shear amount of content from this era that was encoded in it. Alternatively use a format that is so simple it doesn't matter if there is a decoder in 1,000,000 years as it's trivial to make. On the simplicity side take PPM or bitmap images for example, one can get a decoder up and running from scratch in a matter of minutes. The harder half is the storage of the digital information over time. Interfaces, filesystems, formats, and devices are not made to last decades as there is no need for the vast majority of data to sit for decades without being transferred to newer, cheaper, and smaller media. Any solution you come up with to avoid copying will therefore be esoteric and one off, not to mention likely complicated for any significant amount of data, making it hard to guarantee easy enough readability even if it is physically fine in 100 years.
- jbarrs 5y agoI remember when I first heard about 5D storage back in 2013 [1][2]. The concept isn't at all new; in fact, there's a 5D storage device floating around in a Tesla in space, and Musk has another in his personal library. Nonetheless, it's great to see that it hasn't just dropped off the radar and that developments are still being made. I hope that this technology can one day be as widely available as CDs are; its potential usefulness to archivers/data hoarders really excites me, and (so long as the price is right) who can really sniff at extremely high-capacity, long-lasting storage? [1] https://www.orc.soton.ac.uk/news/4282 https://www.orc.soton.ac.uk/news/4282 [2] https://doi.org/10.1364%2FCLEO_SI.2013.CTh5D.9 https://doi.org/10.1364%2FCLEO_SI.2013.CTh5D.9
- ok123456 5y agoI remember a very similar research project from the 90s. A 3D Holographic cube that was supposed to have a, at the time, insane density. https://en.wikipedia.org/wiki/Holographic_Data_Storage_System https://en.wikipedia.org/wiki/Holographic_Data_Storage_Syste...
- ajuc 5y ago> The new approach can achieve write speeds of 1,000,000 voxels per second, the equivalent of 230 kilobytes of data per second > 500 TB of data on a small disc So, it would take 70 years to fill it. By the time you can fill it we will have RAM with higher density :)
- terafo 5y ago> `The researchers say that with improvements to the writing techniques, particularly by taking advantage of parallelism, they can design a system that can fill that same 500 TB in a mere 60 days.` It's possible to get it up to 100 megs per second, which is write speed of a good hard drive. And I suspect it could be scaled up even more. Progress in this field is quite good, 8 years ago they put 300kb on it, now it's 5gb.
- Snitch-Thursday 5y agoWhether it be this or project Silica, I want my digital permanent storage. Write it in glass, treat it nicely, and yes, you may end up with digital bitrot or whatever, but I want media that stores digital data that cannot physically degrade for hundreds of years, kind of like paper does. Paperbak being forked and made mainstream, I can accept that. Provided I can also do an automated book printing. I will also accept holographic stick gum storage a la isolinear chips. But there has to be more than just tape and blu rays.
- ChuckMcM 5y agoPretty neat, this is worrisome (from the paper): A book was recorded in 4-bit voxels with nearly 100% readout accuracy. Kind of like "this new car hardly kills anyone." :-) Given that data recovery is like the most significant aspect of any data archival solution. Not a mention of readout speed either, there are many read-only datasets that could benefit from something like this of course, but if it is way slower than LTO tape I'm not sure exactly where it fits in.
- jijji 5y agoThe thpical lifetime of a CD or DVD or BlueRay is about 3-5 years depending on the handling by the user. I'd like to see them outline the claim that this media can last "billions" of years. It sounds highly suspect, as does all the other claims in the article.
- terafo 5y agoIt is very durable. It withstood 1000 degree heat for 2 hours in referenced paper, there are no moving parts on the storage itself, it doesn't require additional energy to sustain records, records themselves don't deteriorate over time, and glass by nature is quite resistant to mechanical damage(outer few mm obviously shouldn't be used). If stored correctly, I see no reason for it not lasting a few million years.