7 ms·

That quote makes it so clear how weak film is as an archival method. Yet I still hear people talking about how digitally shot films have an archival problem. I

That quote makes it so clear how weak film is as an archival method. Yet I still hear people talking about how digitally shot films have an archival problem. I really can't see how any well thought out digital archive wouldn't be significantly safer than film in a can.

jl6 10y ago
It's a different set of risks. Digital is easier to make fireproof (store copies offsite) but vulnerable to new threats such as automated bulk deletion (e.g. a backup script gone wrong).
- ghaff 10y ago
  It really goes beyond that. Th proverbial prints in the shoebox are subject to fire, flood, analog deteriorations, and just being lost in a move or when cleaning out the attic while digital can be backed up six ways from Sunday. Of course, in practice how many people are one head crash away from losing most of their photos stored on a consumer-grade USB drive? And of course, many people are also subject to cloud companies going out of business, computer equipment being stolen, losing access to a site online, etc. And the situation with gaining access to photos and other things if a family member dies or whatever is another whole issue.
paulmd 10y ago
Well, like anything it totally depends on the specifics. A modern B+W film, properly fixed and washed, is essentially forever. Metallic silver doesn't really degrade by itself (50+ year lifespan) - and you can use a toner like selenium to convert the metallic silver into silver selenide which is even more stable, easily 100-200 years+ (note that this is generally done with prints but it also works with film). Color films are generally considerably less permanent because you can't capture color using metallic silver unless you have one negative each for red, green, and blue. You can do that though, the TechniColor "3-strip" process did this and it's actually extremely stable for its time. It was also used for the color series of Magic Lantern glass plates from the Prokudin-Gorsky survey of 1905 [0]. Modern negative films are decent if stored properly, but early color negative films were pretty disastrous in terms of long-term stability. Color slide films in particular are actually quite decent as long as you store them properly (cold, and out of light). Kodachrome (K-14 process), in particular, was remarkably stable because there is actually no dye in the film itself - it's created by reaction during the (incredibly complex) processing, so (again, if properly washed) there is nothing left to continue reacting and "fade" the image out. Properly processed, Kodachrome does actually approach the stability of B+W film. The only thing that really degrades it is exposure to light (i.e. projecting it or leaving it in a window). Many early film bases were problematic though, because it was essentially a parallel effort with basic materials-science research in plastics. Nitrocellulose is super flammable and many early theaters burned down as a result, it was replaced with acetate "safety film" which tends to go vinegary, finally everyone settled on polyester. Nowadays you can have a pretty good expectation of the storage properties of our plastics. Anyway, to make a short story long - you are assuming the question by adding "a well-thought-out digital archive". You can't pick a random instance of film storage and pretend that's representative of a modern, well-thought-out effort at archival. People did all kinds of dumb things in the past - South Carolina's constitution is falling apart because they laminated them back in the 60s and now they're turning to vinegar too [1]. The equivalent in digital terms is a random paper tape from the 60s pulled from someone's attic, or a DVD that was scribbled on with an alcohol-based marker and left on a spindle somewhere. Or someone's VCR bootleg of the first rough-cuts and rotoscope layers of Star Wars (see "Deleted Magic" [2], it's amazing but wow is the quality terrible in some of them). It's always going to be possible to store some relatively stable physical media, that's the easy part. The problem with digital is you then add additional tasks of loading and interpretation on top. Even if it's stable, how do you get that Quadruplex video tape into your computer? Is there a driver that interprets the filesystem? A codec (ideally open-source)? Is there a sync signal in the medium that might degrade (like VHS)? etc etc. Yes, these are not insoluble problems, but they do add a huge threshold to get over. It might be worth doing to save Star Wars but it's not going to happen for Uncle John's photo collection. Consider something like the task of encoding a message to an alien race. You have to provide something like a Rosetta Stone to help make sense of your format. Or you could just send a phonograph which can be read by many different methods (laser measurement is the coolest). It's a pretty solid assumption that there will always be basic instruments for physical measurement like lasers and reflectometers, and that's all you need to interpret a basic medium like B+W film or a phonograph. Although I suppose how dare I assume the visual wavelength of an alien species! /s On the other hand - if you are going at it thoughtfully, i.e. periodically rotating your digital storage medium to whatever the standard is at the time, and you retain sufficient backups - the chances of there ever being a total discontinuity in migration path, or a total loss of knowledge on how to interpret the format is relatively low. The going at it thoughtfully part is the key though. Again, nobody apart from a handful of archival specialists or enthusiasts can really read a random minicomputer tape from the 80s, and that's only 30 years. I personally started noticing bit-rot on a significant fraction of DVDs that I burned within 5 years (at least 20% of discs). Now I use DVDisaster to encode extra parity/recovery data on a rolling basis - disc X contains a 20% parity file for disc X-1, with the most recent kept on my HDDs, so that I can "chain" backwards through them. Still though - imagine the kind of digital files that were made in 1995 with MOV files or whatever. Now compare them to the high-resolution 4K scans that are made from the same films. That's really another problem with digital, not just archival - the bits are what they are, you can't go back and try again when technology is better. [0.0] https://www.flickr.com/photos/prokudin-gorsky/ https://www.flickr.com/photos/prokudin-gorsky/ [0.1] https://www.loc.gov/exhibits/empire/making.html https://www.loc.gov/exhibits/empire/making.html [0.2] http://www.loc.gov/pictures/collection/prok/ http://www.loc.gov/pictures/collection/prok/ [1] http://www.npr.org/2017/02/21/515410087/an-attempt-to-save-south-carolinas-historical-documents-is-destroying-them http://www.npr.org/2017/02/21/515410087/an-attempt-to-save-s... [2] https://www.youtube.com/watch?v=f2r4Nffrc6Y https://www.youtube.com/watch?v=f2r4Nffrc6Y (there is also a Deleted Magic Revisited DVD with better quality and some more footage - finding it is left as an exercise to the reader)
- ghaff 10y ago
  >That's really the core problem with digital in a lot of respects - the bits are what they are, you can't go back and try again when technology is better. Heh. I've been digitizing a couple handfuls of old VHS tapes that have nostalgia or sentimental value to myself or a friend of mine. You really forget just how poor video from a few decades back was compared to what's available today.
  - paulmd 10y ago
    Yes. Even with high-end studio equipment the difference is astounding. Let alone with random consumer stuff that was poorly stored or whatever. My favorite example here is Star Trek TNG. It was shot on film but immediately transferred to VHS for editing and transmission. The BluRay releases were remastered from the film and the difference in quality is just astounding. Immensely greater color range and resolution. And since everything was done with motion-capture and practical special effects, all that looks amaze-balls too. There are like 2 actual CGI sequences in the entire series (the Crystalline Entity and something else). It was a massive effort to re-master the whole thing because again, it was cut on VHS. They had a map to the general area where a reel of film was stored but nothing specific ("3rd row of shelves, 4th from the top, halfway across"), and no real list of which takes or shots from the reels were actually used. Only a list of what shots were taken on what day, in most instances, and the shot lists weren't always followed. They literally re-cut the entire thing from scratch. Pretty amazing. On the flip side - it's going to be a lot tougher to re-master DS9 or Babylon 5, because they were actually shot on tape (VHS?) instead of film, and the effects were done digitally. The model files and stuff are long gone. You could re-create it from scratch, but nobody wants that (see: the Star Trek TOS remasters). Practically speaking, DVD quality is the best it's gonna get there.
- spookyuser 10y ago
  Fascinating, thank you for highlighting the problems of digital so succinctly.
  - paulmd 10y ago
jwarren 10y ago
Edit: Skip this and go read paulmd's comment, it's far superior: https://news.ycombinator.com/item?id=13886276 https://news.ycombinator.com/item?id=13886276 One reason for film's superiority as an archival medium is that you can hold a piece of film up to the light and see what it contains, whereas to see a digital archive you need to invent all the technology that leads up to decoding that particular file type. Disclaimer: I'm not a film archivist, but my partner is, so this is mostly second-hand knowledge backed up with quick google searches. There have been several different major types of film used throughout the year. Nitrate film was used from the late 1800s up until about the 1950s. It's highly flammable, and nowadays there are very few cinemas which are equipped to screen it. You need a very secure projection booth which can be well sealed in case of fire. It also burns without needing air, as it produces oxygen as part of it's reaction, which means that it's extremely difficult to douse with water. As it ages, the combustion temperature also lowers, and badly stored films may spontaneously combust. Despite all that, a properly preserved nitrate film looks phenomenal when projected. That was replaced with acetate film, initially known as "safety stock". It was initially thought to be pretty stable, but within a few years the decay (known as "vinegaring" due to the smell) started to set in. As with nitrate, proper storage can slow down the decay dramatically. This was used from the 50s until the 80s/90s. So there are these two historic film stocks, which were used in the production, archival, and distribution of every film for most of the last century, and they're both not fit for long term purpose. In the 80s, a more stable polyester film started to be used. This is the best film archival method available to my knowledge. It's shelf stable, and has proven itself in lab tests. This is what people are comparing digital archives to. Apart from the glib example at the beginning, I'm not going to try to argue the relative virtues of preserving the content of a film on film vs digital. However, the field appears to have a general acknowledgement of the importance of the medium of creation. So a digitally shot film would probably be preserved digitally (as well as possible) alongside any other medium.
- cm2187 10y ago
  > One reason for film's superiority as an archival medium is that you can hold a piece of film up to the light and see what it contains, whereas to see a digital archive you need to invent all the technology that leads up to decoding that particular file type. Realistically, unless these nukes are put to intensive use, we are not going to disinvent computers. But your point is certainly valid for codecs. The videos will require to be transcoded multiple times as old codecs become obsolete (who can read real media's .rm files today?)
a3n 10y ago
Yes, digital is probably orders of magnitude safer than film in a can. However, digital is not 100%. There's still data corruption, and loss of systems and programs that will play the data. It's still not set it and forget it, you have to actively preserve the digital media, and now, the surrounding playing mechanisms.

jl6 10y ago

It's a different set of risks. Digital is easier to make fireproof (store copies offsite) but vulnerable to new threats such as automated bulk deletion (e.g. a backup script gone wrong).

ghaff 10y ago
It really goes beyond that. Th proverbial prints in the shoebox are subject to fire, flood, analog deteriorations, and just being lost in a move or when cleaning out the attic while digital can be backed up six ways from Sunday. Of course, in practice how many people are one head crash away from losing most of their photos stored on a consumer-grade USB drive? And of course, many people are also subject to cloud companies going out of business, computer equipment being stolen, losing access to a site online, etc. And the situation with gaining access to photos and other things if a family member dies or whatever is another whole issue.

paulmd 10y ago

Well, like anything it totally depends on the specifics. A modern B+W film, properly fixed and washed, is essentially forever. Metallic silver doesn't really degrade by itself (50+ year lifespan) - and you can use a toner like selenium to convert the metallic silver into silver selenide which is even more stable, easily 100-200 years+ (note that this is generally done with prints but it also works with film). Color films are generally considerably less permanent because you can't capture color using metallic silver unless you have one negative each for red, green, and blue. You can do that though, the TechniColor "3-strip" process did this and it's actually extremely stable for its time. It was also used for the color series of Magic Lantern glass plates from the Prokudin-Gorsky survey of 1905 [0]. Modern negative films are decent if stored properly, but early color negative films were pretty disastrous in terms of long-term stability. Color slide films in particular are actually quite decent as long as you store them properly (cold, and out of light). Kodachrome (K-14 process), in particular, was remarkably stable because there is actually no dye in the film itself - it's created by reaction during the (incredibly complex) processing, so (again, if properly washed) there is nothing left to continue reacting and "fade" the image out. Properly processed, Kodachrome does actually approach the stability of B+W film. The only thing that really degrades it is exposure to light (i.e. projecting it or leaving it in a window). Many early film bases were problematic though, because it was essentially a parallel effort with basic materials-science research in plastics. Nitrocellulose is super flammable and many early theaters burned down as a result, it was replaced with acetate "safety film" which tends to go vinegary, finally everyone settled on polyester. Nowadays you can have a pretty good expectation of the storage properties of our plastics. Anyway, to make a short story long - you are assuming the question by adding "a well-thought-out digital archive". You can't pick a random instance of film storage and pretend that's representative of a modern, well-thought-out effort at archival. People did all kinds of dumb things in the past - South Carolina's constitution is falling apart because they laminated them back in the 60s and now they're turning to vinegar too [1]. The equivalent in digital terms is a random paper tape from the 60s pulled from someone's attic, or a DVD that was scribbled on with an alcohol-based marker and left on a spindle somewhere. Or someone's VCR bootleg of the first rough-cuts and rotoscope layers of Star Wars (see "Deleted Magic" [2], it's amazing but wow is the quality terrible in some of them). It's always going to be possible to store some relatively stable physical media, that's the easy part. The problem with digital is you then add additional tasks of loading and interpretation on top. Even if it's stable, how do you get that Quadruplex video tape into your computer? Is there a driver that interprets the filesystem? A codec (ideally open-source)? Is there a sync signal in the medium that might degrade (like VHS)? etc etc. Yes, these are not insoluble problems, but they do add a huge threshold to get over. It might be worth doing to save Star Wars but it's not going to happen for Uncle John's photo collection. Consider something like the task of encoding a message to an alien race. You have to provide something like a Rosetta Stone to help make sense of your format. Or you could just send a phonograph which can be read by many different methods (laser measurement is the coolest). It's a pretty solid assumption that there will always be basic instruments for physical measurement like lasers and reflectometers, and that's all you need to interpret a basic medium like B+W film or a phonograph. Although I suppose how dare I assume the visual wavelength of an alien species! /s On the other hand - if you are going at it thoughtfully, i.e. periodically rotating your digital storage medium to whatever the standard is at the time, and you retain sufficient backups - the chances of there ever being a total discontinuity in migration path, or a total loss of knowledge on how to interpret the format is relatively low. The going at it thoughtfully part is the key though. Again, nobody apart from a handful of archival specialists or enthusiasts can really read a random minicomputer tape from the 80s, and that's only 30 years. I personally started noticing bit-rot on a significant fraction of DVDs that I burned within 5 years (at least 20% of discs). Now I use DVDisaster to encode extra parity/recovery data on a rolling basis - disc X contains a 20% parity file for disc X-1, with the most recent kept on my HDDs, so that I can "chain" backwards through them. Still though - imagine the kind of digital files that were made in 1995 with MOV files or whatever. Now compare them to the high-resolution 4K scans that are made from the same films. That's really another problem with digital, not just archival - the bits are what they are, you can't go back and try again when technology is better. [0.0] https://www.flickr.com/photos/prokudin-gorsky/ https://www.flickr.com/photos/prokudin-gorsky/ [0.1] https://www.loc.gov/exhibits/empire/making.html https://www.loc.gov/exhibits/empire/making.html [0.2] http://www.loc.gov/pictures/collection/prok/ http://www.loc.gov/pictures/collection/prok/ [1] http://www.npr.org/2017/02/21/515410087/an-attempt-to-save-south-carolinas-historical-documents-is-destroying-them http://www.npr.org/2017/02/21/515410087/an-attempt-to-save-s... [2] https://www.youtube.com/watch?v=f2r4Nffrc6Y https://www.youtube.com/watch?v=f2r4Nffrc6Y (there is also a Deleted Magic Revisited DVD with better quality and some more footage - finding it is left as an exercise to the reader)

ghaff 10y ago
>That's really the core problem with digital in a lot of respects - the bits are what they are, you can't go back and try again when technology is better. Heh. I've been digitizing a couple handfuls of old VHS tapes that have nostalgia or sentimental value to myself or a friend of mine. You really forget just how poor video from a few decades back was compared to what's available today.
- paulmd 10y ago
  Yes. Even with high-end studio equipment the difference is astounding. Let alone with random consumer stuff that was poorly stored or whatever. My favorite example here is Star Trek TNG. It was shot on film but immediately transferred to VHS for editing and transmission. The BluRay releases were remastered from the film and the difference in quality is just astounding. Immensely greater color range and resolution. And since everything was done with motion-capture and practical special effects, all that looks amaze-balls too. There are like 2 actual CGI sequences in the entire series (the Crystalline Entity and something else). It was a massive effort to re-master the whole thing because again, it was cut on VHS. They had a map to the general area where a reel of film was stored but nothing specific ("3rd row of shelves, 4th from the top, halfway across"), and no real list of which takes or shots from the reels were actually used. Only a list of what shots were taken on what day, in most instances, and the shot lists weren't always followed. They literally re-cut the entire thing from scratch. Pretty amazing. On the flip side - it's going to be a lot tougher to re-master DS9 or Babylon 5, because they were actually shot on tape (VHS?) instead of film, and the effects were done digitally. The model files and stuff are long gone. You could re-create it from scratch, but nobody wants that (see: the Star Trek TOS remasters). Practically speaking, DVD quality is the best it's gonna get there.
spookyuser 10y ago
Fascinating, thank you for highlighting the problems of digital so succinctly.
- paulmd 10y ago

jwarren 10y ago

Edit: Skip this and go read paulmd's comment, it's far superior: https://news.ycombinator.com/item?id=13886276 https://news.ycombinator.com/item?id=13886276 One reason for film's superiority as an archival medium is that you can hold a piece of film up to the light and see what it contains, whereas to see a digital archive you need to invent all the technology that leads up to decoding that particular file type. Disclaimer: I'm not a film archivist, but my partner is, so this is mostly second-hand knowledge backed up with quick google searches. There have been several different major types of film used throughout the year. Nitrate film was used from the late 1800s up until about the 1950s. It's highly flammable, and nowadays there are very few cinemas which are equipped to screen it. You need a very secure projection booth which can be well sealed in case of fire. It also burns without needing air, as it produces oxygen as part of it's reaction, which means that it's extremely difficult to douse with water. As it ages, the combustion temperature also lowers, and badly stored films may spontaneously combust. Despite all that, a properly preserved nitrate film looks phenomenal when projected. That was replaced with acetate film, initially known as "safety stock". It was initially thought to be pretty stable, but within a few years the decay (known as "vinegaring" due to the smell) started to set in. As with nitrate, proper storage can slow down the decay dramatically. This was used from the 50s until the 80s/90s. So there are these two historic film stocks, which were used in the production, archival, and distribution of every film for most of the last century, and they're both not fit for long term purpose. In the 80s, a more stable polyester film started to be used. This is the best film archival method available to my knowledge. It's shelf stable, and has proven itself in lab tests. This is what people are comparing digital archives to. Apart from the glib example at the beginning, I'm not going to try to argue the relative virtues of preserving the content of a film on film vs digital. However, the field appears to have a general acknowledgement of the importance of the medium of creation. So a digitally shot film would probably be preserved digitally (as well as possible) alongside any other medium.

cm2187 10y ago
> One reason for film's superiority as an archival medium is that you can hold a piece of film up to the light and see what it contains, whereas to see a digital archive you need to invent all the technology that leads up to decoding that particular file type. Realistically, unless these nukes are put to intensive use, we are not going to disinvent computers. But your point is certainly valid for codecs. The videos will require to be transcoded multiple times as old codecs become obsolete (who can read real media's .rm files today?)

a3n 10y ago

Yes, digital is probably orders of magnitude safer than film in a can. However, digital is not 100%. There's still data corruption, and loss of systems and programs that will play the data. It's still not set it and forget it, you have to actively preserve the digital media, and now, the surrounding playing mechanisms.