5 ms·
I started at the Library of Congress 20 years ago this month working on their digitization effort. It was HUGE even then - an estimated 50TB/day needed to be di
by caseysoftware 5y ago
I started at the Library of Congress 20 years ago this month working on their digitization effort. It was HUGE even then - an estimated 50TB/day needed to be digitized - and that did not address the 200+ years of backlog or the advent of blogging (rare at the time), streaming media, and everything else online.
Our mission was to ensure that the assets we were working with were usable by someone ~100 years from now.
We would take something like an LP (Ray Charles was one of my favorites) and play it and record it in a lossless wav at a high bitrate. Then we'd take the physical record, the entire sleeve, and anything else included and scan that at 300dpi in tiff. Of course, we'd document every piece of equipment we used (down to the serial number) with every setting and who did the work. If we later realized a given piece of equipment was misconfigured, we could detect what had been created with it and adjust or re-create as necessary. I wrote some of the standards (XML schemas) for metadata collection which were still in use as late as 2015. (They may still be, I haven't checked.)
Whew.
Then we'd take a version of the audio and images to create mp3s and jpgs for listening+viewing in the reading rooms. The original physical media would go into nitrogen filled vaults in Culpepper, Virginia to prevent oxidization and further deterioration.*
We had similar processes for other media like books, wire spool recordings (popular in WW2), wax cylinder recordings, and pictures around the Civil War. Many of them were exceptionally fragile and required special equipment. My least favorite but most fascinating were Thomas Edison's first motion pictures.
Pop quiz: What does cotton and silver nitrate degrade into? ;)
But in all of those cases, we could go back to the original media which - hopefully - hadn't deteriorated beyond recovery. In a digital-first or digital-only environment, we're counting on fragile components storing rotting file formats on deprecated operating systems in physical locations that may suffer disasters.
* At the time, the RIAA was complaining endlessly and we may have made jokes about RIAA execs and those vaults but I can neither confirm nor deny any such proposals.
- detaro 5y ago> Pop quiz: What does cotton and silver nitrate degrade into? ;) Bright light if you look at it wrong?
- WalterBright 5y agoThis excessive detail seems nutty to me, and ultimately counterproductive. It's counterproductive in that it reduces the amount of things that can be preserved. What's more important - audio detail that cannot be heard, or more records? The best way to archive data long term is to make diverse copies. The easiest way to make diverse copies is to simply make them available to download off of the internet. Don't restrict it to "reading rooms".
- m4rtink 5y agoI think you want both - high fidelity and many copies and diverse storage locations.
- WalterBright 5y agoMe, no, I can't think of a use for the super hi fi version. Heck, just the snap crackle pop on the best vinyl pressings destroys any utility of super hi fi.
- caseysoftware 5y agoIt comes down to "what is the work?" Take one of Ray Charles' records.. Is it just the music itself? Does it include the physical media and its condition? If the media has deteriorated, how does that affect it? Does it include the cover art? Does it include the liner notes? Does it include the notes through the production process? For the recording of a live performance, does it include the recording equipment and the venue? Does it include the audience reaction? Does it include documenting everyone who performed backup vocals, accompanied via instruments, produced it, and the label itself? While normal listeners (99.9% of the public) would stop at the music, the audiophiles will want to understand the physical media and performance conditions, and the researchers/archivists want everything because it's a snapshot of a particular work at a particular time & place via particular means. But part of your comment is spot on.. diverse sources. It's not feasible for one Library to digitize absolutely everything in the world (or a country), so we participated in an exchange with a bunch (20? not sure anymore) of other libraries, museums, etc who could digitize their own collections and then we could all share them at scale with as little DRM as possible. Think of it as Napster for libraries. I advocated for no DRM (especially for old, out of production works) but that's why we butted heads with the RIAA.
- lifepillar 5y ago> Our mission was to ensure that the assets we were working with were usable by someone ~100 years from now. Is there any published technical document about the choices made by LoC to achieve that goal?