3 ms·

Given IO errors and degradation of storage media, how long can one realistically preserve digital data? Is it possible with multiple disks and an extant error

Given IO errors and degradation of storage media, how long can one realistically preserve digital data? Is it possible with multiple disks and an extant error correction algorithm to prevent data corruption indefinitely, assuming some maximum error rate?

derefr 6y ago
> Is it possible with multiple disks and an extant error correction algorithm to prevent data corruption indefinitely, assuming some maximum error rate? This is the main thrust of the Dynamo paper, yes. You can solve all the technical problems. Put something in a "living system" like Amazon S3 (where data has 17+ clones; and where ops people are there to both replace individual hard disks as they fail, and to gradually port the system over to new storage layers as new technology displaces old), and then—presuming the company providing the service is still there—it'll be vanishingly unlikely that your data will have degraded on a human time-scale. What's left in the way of "indefinitely", though, are political problems. "How do I preserve data indefinitely" gets into the same sort of hairy problems as "how do I enforce my will on my descendants after I die?" kind of questions, where you start to see people putting forward ideas like immutable autonomous trusts. The key question being: how do I ensure that the stuff I want to stick around, sticks around, after I'm dead, during a period when literally nobody else cares about it, until a later time when somebody else does care about it?
agent86 6y ago
For consumers, M-DISC media for DVD/Blu-ray might be about the best there is for long term storage.[1] In the enterprise/experimental world there are things like Microsoft Project Silica[2][3] where they are storing data on quartz slides. [1]https://en.wikipedia.org/wiki/M-DISC https://en.wikipedia.org/wiki/M-DISC [2]https://www.microsoft.com/en-us/research/project/project-silica/ https://www.microsoft.com/en-us/research/project/project-sil... [3]https://news.microsoft.com/innovation-stories/ignite-project-silica-superman/ https://news.microsoft.com/innovation-stories/ignite-project...
xeromal 6y ago
Have you ever seen those laser glass sculptures you can have customized for a person? I've always though those would be a good way to store data permanently

derefr 6y ago

> Is it possible with multiple disks and an extant error correction algorithm to prevent data corruption indefinitely, assuming some maximum error rate? This is the main thrust of the Dynamo paper, yes. You can solve all the technical problems. Put something in a "living system" like Amazon S3 (where data has 17+ clones; and where ops people are there to both replace individual hard disks as they fail, and to gradually port the system over to new storage layers as new technology displaces old), and then—presuming the company providing the service is still there—it'll be vanishingly unlikely that your data will have degraded on a human time-scale. What's left in the way of "indefinitely", though, are political problems. "How do I preserve data indefinitely" gets into the same sort of hairy problems as "how do I enforce my will on my descendants after I die?" kind of questions, where you start to see people putting forward ideas like immutable autonomous trusts. The key question being: how do I ensure that the stuff I want to stick around, sticks around, after I'm dead, during a period when literally nobody else cares about it, until a later time when somebody else does care about it?

agent86 6y ago

For consumers, M-DISC media for DVD/Blu-ray might be about the best there is for long term storage.[1] In the enterprise/experimental world there are things like Microsoft Project Silica[2][3] where they are storing data on quartz slides. [1]https://en.wikipedia.org/wiki/M-DISC https://en.wikipedia.org/wiki/M-DISC [2]https://www.microsoft.com/en-us/research/project/project-silica/ https://www.microsoft.com/en-us/research/project/project-sil... [3]https://news.microsoft.com/innovation-stories/ignite-project-silica-superman/ https://news.microsoft.com/innovation-stories/ignite-project...

xeromal 6y ago

Have you ever seen those laser glass sculptures you can have customized for a person? I've always though those would be a good way to store data permanently