3 ms·

Digital data doesn't wear out, but it also doesn't degrade gracefully. Usually if a printed page starts to degrade there is at least the opportunity for someon

Digital data doesn't wear out, but it also doesn't degrade gracefully. Usually if a printed page starts to degrade there is at least the opportunity for someone to notice and take corrective action before the content is lost completely (scanning, processing and re-printing, perhaps). Digital data tends to go from "fine" to "gone" with no middle ground.

ivanhoe 4y ago
That's what backups and RAID arrays are for. We certainly know by now how to build a redundancy that will guaranty the data safety. For most of the libraries it's really a budget problem - they've been given huge expensive buildings with controlled air conditions for storing paper books and that's sort of taken for granted, but then governments take cuts on providing them with proper IT infrastructure and knowledgeable staff for maintaining the digital data copies.
- peoplefromibiza 4y ago
  > That's what backups and RAID arrays are for. the point is exactly that: if you have to rely on very complex setups to keep the data fresh, it means that the data is not safe. can you imagine an hard disk surviving from the ancient Egypt to us?
  - ketzu 4y ago
    Wasn't the point of the original comment that the setup to keep paper safe is also very complex? Another question I wonder about: How hard is noticing the failures?
    - robinsonb5 4y ago
      The cold hard truth here is that curating information is actually a very difficult problem - but paper records will survive longer than digital records if no-one's actively looking after them.
      - ivanhoe 4y ago
        It's 100% true for a book forgotten on your attic, but if discussing organized effort to preserve information I argue it's an illusion - the libraries have to be actively looked after too. There is a fire protection, regular deratization and disinsection, if roof starts leaking it has to be fixed. If you just leave a book in the middle of a field unprotected it won't last a year. When libraries are left without funds their books get destroyed fairly quickly, I've seen that first hand in my home country. Digital data requires some maintenance, but with cloud services that's offloaded to service providers. I haven't logged in into Glacier or the related AWS account in years, and yet it still backups my hdd regularly. Now, if a comet strikes the Earth and destroys all the civilization, yes, paper books will be a lot more useful than HDDs or dead cloud servers, but in more realistic scenarios data copies give you more flexibility. Just look at Ukraine, they moved all their official digital archives out of the country at the very beginning of the war. You couldn't ever pull it off with paper archives, it would take months and cost a fortune.
    - peoplefromibiza 4y ago
      yes, it's hard, but papyrus kept inside a very strong, possibly waterproof, possibly not affected by rust (alluminium, stainless steel etc), metal box will last much longer than hard drives kept inside the same strong metal box.
      - ivanhoe 4y ago
        true, but we could always print encoded digital data to that papyrus and call it a data-storage method :)
        anthk 4y ago
        And now you need a parser and a machine :)
- moffkalast 4y ago
  Bit rot: bonjour
moffkalast 4y ago
Well the thing about digital data is that it's all represented in an uneven compressed way. Take your average piece of paper with text on it, it's essentially a 2D array of white and black dots. Remove half of all black dots and you can still mostly see what the text was. Same is true with raw digital representations. Take on the other hand a jpeg which is a bunch of coefficients for sine waves, where every tiny change ripples out to ruin the entire thing. Even the basic 3 byte RGB per pixel representation is already compressed with only 8 bits being used to save 256 values. Flip the MSB and your value is already completely screwed. An equal encoding to real paper would be using 256 bits to represent each colour, summing up the bits to get integer values. That way you can flip 20 bits and you'll still easily tell what the colour was, without any extra processing. This of course comes at an exponential increase in storage requirements, plus we can be smarter than just using brute force. Taking a compressed representation and then adding on a few layers of error correction would likely be far more reliable while preserving most of the spatial efficiency. Of course then you need the software for unpacking and correcting to survive as well...
remram 4y ago
This completely ignores https://en.wikipedia.org/wiki/Error_correction_code https://en.wikipedia.org/wiki/Error_correction_code Those have been known for a long time and they are used in practice for both transmission and long-term storage. Big storage arrays come with health metrics and auto-rebalancing around broken drives. The idea that digital storage media degrades without "opportunity for someone to notice" is outright false.
- anthk 4y ago
  Devices themselves can fail too, not just media.
  - remram 4y ago
    Sure? We were talking about data though.

ivanhoe 4y ago

That's what backups and RAID arrays are for. We certainly know by now how to build a redundancy that will guaranty the data safety. For most of the libraries it's really a budget problem - they've been given huge expensive buildings with controlled air conditions for storing paper books and that's sort of taken for granted, but then governments take cuts on providing them with proper IT infrastructure and knowledgeable staff for maintaining the digital data copies.

peoplefromibiza 4y ago
> That's what backups and RAID arrays are for. the point is exactly that: if you have to rely on very complex setups to keep the data fresh, it means that the data is not safe. can you imagine an hard disk surviving from the ancient Egypt to us?
- ketzu 4y ago
  Wasn't the point of the original comment that the setup to keep paper safe is also very complex? Another question I wonder about: How hard is noticing the failures?
  - robinsonb5 4y ago
    The cold hard truth here is that curating information is actually a very difficult problem - but paper records will survive longer than digital records if no-one's actively looking after them.
    - ivanhoe 4y ago
      It's 100% true for a book forgotten on your attic, but if discussing organized effort to preserve information I argue it's an illusion - the libraries have to be actively looked after too. There is a fire protection, regular deratization and disinsection, if roof starts leaking it has to be fixed. If you just leave a book in the middle of a field unprotected it won't last a year. When libraries are left without funds their books get destroyed fairly quickly, I've seen that first hand in my home country. Digital data requires some maintenance, but with cloud services that's offloaded to service providers. I haven't logged in into Glacier or the related AWS account in years, and yet it still backups my hdd regularly. Now, if a comet strikes the Earth and destroys all the civilization, yes, paper books will be a lot more useful than HDDs or dead cloud servers, but in more realistic scenarios data copies give you more flexibility. Just look at Ukraine, they moved all their official digital archives out of the country at the very beginning of the war. You couldn't ever pull it off with paper archives, it would take months and cost a fortune.
  - peoplefromibiza 4y ago
    yes, it's hard, but papyrus kept inside a very strong, possibly waterproof, possibly not affected by rust (alluminium, stainless steel etc), metal box will last much longer than hard drives kept inside the same strong metal box.
    - ivanhoe 4y ago
      true, but we could always print encoded digital data to that papyrus and call it a data-storage method :)
      - anthk 4y ago
        And now you need a parser and a machine :)
moffkalast 4y ago
Bit rot: bonjour

moffkalast 4y ago

Well the thing about digital data is that it's all represented in an uneven compressed way. Take your average piece of paper with text on it, it's essentially a 2D array of white and black dots. Remove half of all black dots and you can still mostly see what the text was. Same is true with raw digital representations. Take on the other hand a jpeg which is a bunch of coefficients for sine waves, where every tiny change ripples out to ruin the entire thing. Even the basic 3 byte RGB per pixel representation is already compressed with only 8 bits being used to save 256 values. Flip the MSB and your value is already completely screwed. An equal encoding to real paper would be using 256 bits to represent each colour, summing up the bits to get integer values. That way you can flip 20 bits and you'll still easily tell what the colour was, without any extra processing. This of course comes at an exponential increase in storage requirements, plus we can be smarter than just using brute force. Taking a compressed representation and then adding on a few layers of error correction would likely be far more reliable while preserving most of the spatial efficiency. Of course then you need the software for unpacking and correcting to survive as well...

remram 4y ago

This completely ignores https://en.wikipedia.org/wiki/Error_correction_code https://en.wikipedia.org/wiki/Error_correction_code Those have been known for a long time and they are used in practice for both transmission and long-term storage. Big storage arrays come with health metrics and auto-rebalancing around broken drives. The idea that digital storage media degrades without "opportunity for someone to notice" is outright false.

anthk 4y ago
Devices themselves can fail too, not just media.
- remram 4y ago
  Sure? We were talking about data though.