4 ms·
Archiving things for future civilizations to dig up is one thing. Then you may be talking about hundreds or thousands of years that the data just sits there. Th
by alkonaut 11y ago
Archiving things for future civilizations to dig up is one thing. Then you may be talking about hundreds or thousands of years that the data just sits there. Then you have to worry about the longevity of bits, and whether the technology and knowledge to read the data will exist in the future. Digital is very hard for that use case.
On the other hand, storing data for an active/continuously used archive doesn't seem so hard. You just keep the data digitally on some media and every few years you upgrade the tech to whatever technology and formats are "current" before the old format and tech is completely forgotten. Keeping digital data around for a decade is pretty easy. If you do that 100 times in a row you have kept it for a thousand years and you never had a file format or a storage media older than a decade. You could "forget" to upgrade the archive, but you could also "forget" to scribble on the calfs some time.
- DanBC 11y ago> You just keep the data digitally on some media and every few years you upgrade the tech to whatever technology and formats are "current" before the old format and tech is completely forgotten. But you need to avoid the mistakes that people have already made: using some weird format that dies, and then having all your data locked away in a semi-readable file.
- rayiner 11y agoI mean, what's a "weird format?" Did anyone anticipate EDBIC wouldn't survive into 2015?
- michrassena 11y agoOff the cuff, I'd point to any format for which you'd have to fire up an emulator to run the software to read it. I would also include any format which is not openly documented or which requires a propriety tool to view it. When you look at archiving digital formats, you have to look at the whole toolchain and realize that the substrate of the medium itself will degrade, the electrical or magnetic field will degrade, the storage format will quickly become obsolete, the digital file format will be and so forth. More relevant to the discussion at hand, supposing you do intend to store important documents in a digital format as a long-term archive. What format do you store the instructions for reading and decoding the archive? Wouldn't carbon pigment on vellum be stable enough? Edit: On further consideration, I think we have to regard digital media and formats like a form of encryption (a simile, I don't mean this literally) which gets stronger over time. Time will bury extant evidence of what is contained on the media and how it is stored. Why should not make the information as explicit (plain-text) as possible? People in 800 years will still have to spend their efforts decoding early 21st century English legalese.
- to3m 11y agoThere's a fun example here: https://en.wikipedia.org/wiki/BBC_Domesday_Project https://en.wikipedia.org/wiki/BBC_Domesday_Project Compare and contrast: https://en.wikipedia.org/wiki/Domesday_Book#Custodial_history https://en.wikipedia.org/wiki/Domesday_Book#Custodial_histor...
- michrassena 11y agoI wonder if we'll look back at today's technology in 25 years and it will look as unpolished as that Acorn and laserdisc.
- carboncopy 11y agoWhat are these "weird formats" that keep getting referenced here? LZH? ARC? PKZIP? WordPerfect? Filesystems formatted in FAT16? All of these formats are still very much readable. Some formats that are no longer usable in general, such as punch cards, were obsoleted consciously. And you might struggle to find a LaserDisc reader. Otherwise, we should be fine. Or am I missing something?
- nitrogen 11y agoYes, you're missing the hindsight bias. People using punch cards and laserdiscs weren't warned ahead of time that things stored on them would become unretrievable soon after. Nobody knows in advance which formats/media will be "obsoleted consciously".
- carboncopy 11y agoThe overlap between punch cards and other formats was decades. "During the 1960s, the punched card was gradually replaced as the primary means for data storage by magnetic tape, as better, more capable computers became available. ... [P]unched cards were still commonly used for data entry and programming until the mid-1980s when the combination of lower cost magnetic disk storage ... made punched cards obsolete ..."[1] Yes, I listed wikipedia as a primary source, deal with it. [1] https://en.wikipedia.org/wiki/Punched_card https://en.wikipedia.org/wiki/Punched_card
- clock_tower 11y agoAnd remember, punch cards and laserdiscs are famous obsolete storage formats. What if you put your important information on an Iomega Jaz cassette, or stored it in a Bernoulli Box cartridge?
- DanBC 11y agoTry MS Excel for Mac v1 - those spreadsheets are going to be unfun to open. Or MS Works spreadsheets without MS Works.
- brudgers 11y agoMUMPS.
- theandrewbailey 11y agoWe have a much better idea of what digital technologies and formats will last into the future than we did in the 1980s. Pick reasonably open formats and standards that have been around for a long time, and are used every day. HTML has been heavily used for more than 20 years, and it's only getting more widely used. UDF (the DVD file system) has been around for almost 20 years, and ISO 9660 (CD file system) even longer. Better yet: mix, match, and do all of them!
- chippy 11y agoThe problem is not making sure you upgrade, nor is it about technology. It's identifying the things to keep without knowing the future. 50 years ago and all those letters that were burnt or composted because no one bothered to keep the sentimental letters of soldiers in a world war. 20 years later and all those millions of photos that were on Flickr but no one bothered to upgrade. 100 years later and all those GPS logs from taxi cabs that were logged by firms are lost as no one thought they would be interesting or useful. 600 years later and these floppy disks with meeting minutes on them about an early internet organization are lost but might well be worth more than they are now.
- jsprogrammer 11y agoStorage space gets cheaper and more voluminous by the day. Everything can be kept and tomorrow you can keep even more.
- rtkwe 11y agoIt's not free though so every company with data makes decisions about what is worth saving. Sure the costs are always falling but you have to pay (at best) today's prices to store today's data.
- tptacek 11y agoNon-sequitur. The point of the thread is that it's not at all clear that reliable long-term storage is getting cheaper by the day.
- jsprogrammer 11y agoDefine 'reliable long-term'. Roll-over to new tech is safe. The content of discussion is inherently reliable due to its digital nature. Keeping multiple copies is necessary practice and the long-time trend is that the next copy you produce has no greater cost than the previous copy (if you wait years, it will be significantly cheaper).
- InclinedPlane 11y ago"doesn't seem so hard" ... "you just" Ah, the classic sign of someone about to say something rash about technology.
- alkonaut 11y agoI thought the idea that it would be difficult was a misunderstanding of the task to begin with. No one is trying to archive digitally for a thousand years. That's hard. Archiving for 10 years is easy. As many times as you want.
- quanticle 11y ago>You just keep the data digitally on some media and every few years you upgrade the tech to whatever technology and formats are "current" before the old format and tech is completely forgotten. "Just". Have fun justifying the cost of re-encoding and re-archiving all of the laws of the UK every few years. The advantage of vellum is that once it's written, that's it. You don't need to go back and re-encode and upgrade to the latest storage technology. It's there, and it'll last literally a thousand years if it's stored in a reasonably dry, cold, and dark location.
- alkonaut 11y agoHow much is it? One hard drive, max. Write it on a few hard drives and move it to something else in 10 or 20 years before the standard is outdated. I doubt we won't be able to read text data any time soon. It's not handling itself but the risk of data loss feels at least as low as someone stealing or sabotaging a single physical document.