4 ms·
Back in the day (1999) I wrote a data processing and archiving pipeline for a research satellite, and I was surprised to discover the specificity of their data
by gmisra 12y ago
Back in the day (1999) I wrote a data processing and archiving pipeline for a research satellite, and I was surprised to discover the specificity of their data archival requirements. Off the top of my head, these were decision points:
* Pretty much only used "gold" CDs, rated for longer storage.
* CD write speed was restricted - higher write speeds caused more write failures
* The total data payload was regulated (~600 MB I think), and one of the reasons why was because of the physical impact of human handling on the outermost areas of the disc.
* Storage temperature was also regulated
But ultimately, the two main data storage strategies employed were:
* Physical redundancy - six copies of everything, mailed out to six physically separate facilities every week.
* No archiving "forever" - Archived data was (theoretically) to be restored to contemporary storage technology whenever there was a significant step change in storage technology. At the time, another member of the lab was working on restoring old Voyager data feeds, although I have no awareness of how common this actually was.
- jacquesm 12y agoThere is something funny about how pirated movies and music are probably the most resilient data available and how irreplaceable data like the one that you reference is the most vulnerable. Maybe we could set up a scheme where a few megabytes of precious data are appended to torrents of movies and mp3s for extremely resilient back-ups?
- theandrewbailey 12y agoThe story mentioned that data was being transferred to servers. Maybe archivists could tell the internet to download everything?