7 ms·
10,000TB storage cartridges could become mainstream by 2030
- deleted 3y ago[deleted]
- swazzy 3y agoi wonder if we’ll ever get to a point where we don’t think about storage at all. or will we always fill up whatever storage that’s available.
- kyriakos 3y agoOnly if we reach a peak fidelity level. Since storage will never be infinite therefore unless we reach a level of media quality that is good enough for everything it will always be an arms race.
- rendall 3y agoI think even if we got storage that could store enough information to be indistinguishable from reality itself, we would still want to save variations, clips, duplicates, intermediates... I don't know that there is a peak fidelity.
- crote 3y agoWe are already doing that, though. The thing is, there are only 24 hours in a day. There is a hard upper limit to the amount of content you can consume. You're not going to be downloading 1000 hours worth of content every single day, just because it is physically impossible to use it.
- kyriakos 3y agoTell that to /r/datahoarders. Always amazed by what people are archive in that sub reddit
- wongarsu 3y agoI spend so much storage on just storing things that are readily available on the internet, either to speed up local access or because one day they might not be as readily available. I don't think we will "solve" storage for good unless we also "solve" bandwidth.
- crote 3y agoFor a lot of applications we have pretty much reached that point already. 2TB NVMe SSD are around the $100-$150 price point these days. Unless they are actively trying, the average desktop user is never going to fill that up. There are only so many holiday pictures you can take, after all. I think the size of audio files is a great example of why storage needs won't infinitely grow. Although we have orders of magnitude more storage space these days, audio files haven't really gotten any bigger since the CD era. If anything, they have gotten smaller as better compression algorithms were invented. The thing is, human hearing only has so much resolution. Sure, we could be sampling audio at 64-bits with a 1MHz sample rate these days if we wanted to, but there's just no reason to. Similarly with pictures: if you can't see any pixels standing a foot away from a poster-sized print, why bother increasing the resolution any more? The big consumer-range data hogs are 1) video torrents, and 2) games. Both of them have a natural upper bound due to human perception. They might still grow by an order of magnitude or two, but it won't be much more before it just becomes pointless. Enterprise is a bit of a different story, of course - especially now that AI is rapidly increasing the value of data.
- rowanG077 3y agoI don't think holding human perception as a standard for video games holds up. For audio and video it makes sense, but we are a long way of for video. For video games there is a huge "storage waste" factor. A factor that I think is more important than the human perception limit. If you look at modern games you can probably throw away double digit percentage signs of most games if a capable team would have the time to optimize for disk space. It's simply not done because it has little advantage. I think this wastage factor will scale with the complexity of video games regardless of graphical fidelity.
- znpy 3y ago> There are only so many holiday pictures you can take, after all. You miss the point, sadly. Yeah storage is getting larger and cheaper but modern cameras/phones/etc take photos that are way larger than they used to do. My first digital camera in like 2001 or so could only take either 20 "large" (640x480) bitmap photos or 80 "small" pictures, taking a few tens of kilobytes each at most. My 2022 iphone se takes high quality pictures that are easily in the 7-8MB range (i just checked). So yeah, disks keep getting larger, but so does the media.
- anonymouskimmer 3y agoMy 500 GB HDD has been way more than adequate for the past 10+ years, and that includes having a Windows 8 partition that I haven't booted into in years.
- geor9e 3y agoI think we will always fill up the storage available as we crank up the fidelity of our recordings (simulations). Video is an approximation of reality, and we can always ramp up the resolution, ramp up the dimensions, always striving to simulate the universe at the same fidelity as the universe itself, to the point that the map becomes the territory itself. In other words, until it's no longer approximating, no longer compressing. There will never be enough storage to store our simulations of our reality as long as we're forced to use that same reality to construct the storage.
- mikewarot 3y agoI'm already at that point, effectively. I spent a few decades worrying about fitting data into hard drives... most recently with my digital photos. I've stopped worrying. My first computer stored data on audio cassette tapes. When I started college, the new hot machine in the server room was a DEC VAX 11/780 running VMS. For nostalgia purposes, I have a virtual VAX 11/780 running VMS in my phone, it only takes about 3 GB. I don't do much else with the phone, so I have plenty of room left over. I can imagine ways to fill up a petabyte personally... but not much past that.
- alfl 3y ago> This process imprints holes – or no holes – onto the surface layer, which represents binary information. It’s high-density ceramic punch cards!
- beAbU 3y agoWell, that's how CDs and other optical media also work.
- fbdab103 3y agoSimilar idea to Microsoft's Project Silica[0], though this is claiming much higher data density. Were I Google or AWS, I would buy them out immediately. If only to deny the other from getting potentially cheaper archival storage. Or even to keep the rest of the market reliant upon tapes. Want long term data storage? Only the cloud has the real technology for that. [0] https://www.microsoft.com/en-us/research/project/project-silica/ https://www.microsoft.com/en-us/research/project/project-sil...
- dexwiz 3y agoThere is a big difference between managed archives and the media itself. Services like Iron Mountain predate the modern Cloud.
- MichaelRo 3y agoBollocks. Sensationalist fluff. Will never happen, by never meaning "not during our lives". I'm willing to bet you on that, it's a lock. Easiest money possible.
- Bluecobra 3y agoYep, we have been promised crazy storage like this for years now and it has always been vaporware. Remember this? https://www.engadget.com/2007-03-28-mempile-shows-off-teradisc-dvd-sized-optical-storage.html https://www.engadget.com/2007-03-28-mempile-shows-off-teradi... One can dream though....
- ianburrell 3y agoI am excited by the durability. We need an archival storage mechanism that can last for long time. Hopefully, this will be cheap and come in smaller size for home use. For example, backing up photos is annoying. Hard drives are unreliable, cloud storage can go away, and tape is expensive.
- Hamuko 3y agoIf you need cloud storage for archival, consider Amazon S3 Glacier Deep Archive. $1/TB/month and S3 is probably the last storage service on the Internet to "go away". Retrieval costs are pretty awful but helluva lot better than losing everything.
- _lvbh 3y agoWhy do people pay for things like Google Drive or Dropbox when S3 storage is so cheap?
- oefrha 3y agoWhy use Dropbox when you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem, back in 2007 even?
- Valgrim 3y agoI understand some of these words.
- chronogram 3y agoIt's a reference to the comments of the "Show HN" of Dropbox back in 2007: https://news.ycombinator.com/item?id=8863 https://news.ycombinator.com/item?id=8863
- mlyle 3y agoBecause I can't store my gmail and Google Docs documents on S3, and because Google Photos is easier to use than cramming my stuff to S3.
- hn_throwaway_99 3y agoThis article is really bad: "making slow hard drives and tapes obsolete" and "In contrast to data usually stored on the best hard drives and the best SSDs of today..." No. My understanding from the video is that it's a write-once solution, so if anything it would just be a replacement for tape backup, so not sure why it's comparing it to hard drives and SSDs.
- lcuff 3y agoNot only that, but the article describes a system that is in a remote data center, not something I can have in my laptop. Although if it fit in my laptop for the same price as a hard drive. I could probably 'struggle along' with a write-once solution and not worry about filling 10,000 Terabytes any time soon. Complete history on every file!
- mattashii 3y agoThe title could use some changes in wording, but "In contrast to data usually stored on the best hard drives and the best SSDs of today..." was clearly referring to the data retention time of those media, and that is indeed often quoted as being on the order of years, not thousands of years, for HDD/SSD data carriers. I'd say that's quite contrasting indeed, if that ceramic storage can keep its promise of 5000 years data retention.
- rowanG077 3y agoI always wonder about what we will do for reading. It's not easy to read back a few decade old tape even if the tape is pristine. You would really have to scavenge for a working tape drive. Multiply that timeframe and I really don't see how having it being stored for thousands of years as that appealing.
- hn_throwaway_99 3y ago> was clearly referring to the data retention time of those media, and that is indeed often quoted as being on the order of years, not thousands of years Correct, but the primary reason those technologies have shorter retention times is because they are rewritable. It's an apples-to-oranges comparison that doesn't make any sense. It would make sense if the article was clear that this was about improving archival storage options, and then comparing to things like CDs/DVDs, tape, and film.
- tromp 3y ago> This first demonstration unit doesn't compete with the very best data storage units out there, but the company plans to scale up its ceramics-based data storage system This begs the question: on which performance measures does it fail to compete? Supposedly not on capacity. Read speed? Write speed? Power use? Cost? Sadly they don't say.
- loeber 3y agoYou are misusing the term "begging the question", which is a formal logical fallacy and not synonymous with "raising the question". https://en.wikipedia.org/wiki/Begging_the_question https://en.wikipedia.org/wiki/Begging_the_question
- riffic 3y agoeveryone misuses the term which raises questions about prescriptivism vs descriptivism.
- anonymouskimmer 3y agoYou're begging the question with this dichotomy. It likely originally arose as a descriptive term among those who had the requisite knowledge base prescribed to them. :P
- anonymouskimmer 3y agoI think one important reason people misuse this so much is because "begging the question" does not straightforwardly convey the meaning assigned to it by whichever logicians coined the modern English term. It's a linguistically crappy term. I'd personally prefer that "beg the question" be repurposed in the way GP used it, and that another term be used to describe the logician's idea.
- wharvle 3y agoThis one’s lost. Carry on using it correctly among those who get it, avoiding the expression among those who probably don’t, and quietly accepting this usage from those who employ it (while, perhaps, marking them off your mental list of potential proof-readers). I don’t even feel too bad about this one, because it’s so easily misunderstood. May as well let it go.
- pelagicAustral 3y agoFar more hyped about DNA storage. > https://www.microsoft.com/en-us/research/project/dna-storage/ https://www.microsoft.com/en-us/research/project/dna-storage... > https://www.scientificamerican.com/article/dna-the-ultimate-data-storage-solution/ https://www.scientificamerican.com/article/dna-the-ultimate-...
- anonymouskimmer 3y agoDNA "storage" is great for parallel interrogation I've read, but for actual "storage" storage proteins are far longer lasting. With fossilized proteins having been recovered after over half a billion years. There's some loss in fidelity as the protein side chains undergo some chemical processes, but these processes are known and can be compensated for.
- mfld 3y agoDNA has been extracted from Neanderthal bones more than 50.000 years old. DNA is very durable, for all practical purposes. I've heard some DNA synthesis vendors like Twist Biosciences will offer it commercially next year.
- anonymouskimmer 3y agoStorage conditions really matter for DNA longevity, and a lot of the older DNA that can be extracted and sequenced is both fragmented and damaged in areas. I've had closed-circular plasmids fragment completely after being stored in water at 4C for about a year or so.
- gyrate 3y ago[dead]
- jvanderbot 3y agoIt's about a year to the outer planets on falcon heavy. Less on SLS, but I digress. Suppose a flyby mission records a couple of these cartidges and flies back to Earth. Assume 3 years round trip (1 there, 1 record, 1 back). That's Nx80,000,000,000,000 bits in 3 years. Or Nx10E+16 bits / 9E+7 seconds. Or about a gigabit per second effective rate per cartridge. Not bad. Psyche (our best optical link from deep space), has 2 megabits/s peak rate.
- hakuseki 3y agoWhat's so difficult about optical links from deep space compared to low earth orbit, where 200 gigabit throughput has been achieved? Is it just the attenuation? I would have imagined that we could upgrade the communication equipment on a space probe much more easily than we could add fuel for a return trip.
- mlyle 3y ago> What's so difficult about optical links from deep space compared to low earth orbit, where 200 gigabit throughput has been achieved? Is it just the attenuation? Yup; 300km vs 600000000km. Less than a trillionth of the power comes through for the same emitted power and apertures. So, you end up having to make apertures much larger and point much, much more precisely. You can also increase laser power some, but that's a small part of your solution. (Or, of course, you can reduce speeds to have more energy per bit). I don't think anyone is serious about shipping back physical data from deep space, but the station wagons full of tape thought experiment is always fun.
- jvanderbot 3y agoI am semi-serious. But it's more about a periodic backup of an on site science station. An on site (orbital) station could run all the models and algorithms and analysis you want on the gobs of imagery and spectral maps / cubes you can gather in real time. Analysis is an excellent compression algorithm. But, you'd want the raw data eventually, and that means either trickling back 0.1%, or waiting a few years for the full set. https://josh.vanderhook.info/publications.html#josr2022sspe https://josh.vanderhook.info/publications.html#josr2022sspe
- oefrha 3y agoStartup claims to have storage technology so groundbreaking that it can reach TBs/cm^2, GBps speed (read/write?), seconds (?) of read latency, low cost of <$1/TB, and durability of thousands of years. Even attaches a 2030 mass market timeline to it, which is not far into the future. I'm skeptical, and the 2030 number makes me even more skeptical. I didn't watch the hour-long presentations though. https://storagedeveloper.org/events/agenda/session/527 https://storagedeveloper.org/events/agenda/session/527 https://storagedeveloper.org/events/agenda/session/603 https://storagedeveloper.org/events/agenda/session/603
- GlibMonkeyDeath 3y agoThis is basically optical write-only tape drive technology. There is literally no way to get to TB/cm2 using their technique (that's ~100 bits/linear um) using visible range light and spatial light modulators in a single layer. They reveal that their future system will use 5 um thick tape (good luck! that's half the thickness of Saran Wrap) so I assume the claimed TB/cm2 density includes a bunch of layers. That's fine, but the claims about 5000 year lifetime on 5 um polymer tapes with nm thick ceramic are pretty extraordinary (requiring extraordinary evidence.)
- lofaszvanitt 3y agoNext: Cerabyte goes down the drain, by going bankrupt, simply dissipates into thin air or will be acquired, then we'll never hear about them again. I vividly remember multiple archiving products announced, all disappeared, somehow. Same with "groundbreaking" battery solutions.
- enslavedrobot 3y agoMaintaining the beam quality of 2000 laser beamlets ain't cheap.
- jimnotgym 3y agoI wonder what we will find to store in all that space? Stuff that shouldn't be stored, probably! https://news.ycombinator.com/item?id=38498090 https://news.ycombinator.com/item?id=38498090
- Aardwolf 3y agoWhat's a good write once backup solution for long term storage _today_ though? I saw writable bluray disks recommended as they would last for decades, however 25GB for a standard disk is by far not enough for a photo collection, and many disks gets clunky.
- unnouinceput 3y agoSee my other comment above about me using the cloud in an unorthodox way.
- mikewarot 3y agoI've seen data on floppy disks from the early 1990s a few weeks ago, on an Apple ][. Magnetic media lasted way longer than I expected. But, in this case... I have a hard time believing in a "Thousand Year Write" ;-) There's no way that something that thin and ceramic isn't going to somehow become embrittled, or fracture, in 100 or more years.