5 ms·

> But how often one checks checksum hash on copy? Your operating system does this.

by altcognito 8y ago

imhoguy 8y ago
Only if you use ECC RAM and ZFS/Btrfs/ReFS. CRC e.g. in TCP is too weak.
- altcognito 8y ago
  Ah this is right, I was thinking of removable media, cdrom/dvd usually do this.
  - tajen 8y ago
    USBs don’t do this. My md5 checksums fail all the time for big files on USBs. And USBs dont have SMART features.
- zamadatix 8y ago
  Yes, if one was going to reliably archive information they wouldn't use FAT32 on a single spinning disk. Does this really need to be pointed out? The point is digital gives you ways to have perfect copies, correct loss, and the ability to transfer to the next storage medium with no decay. Things analog could never achieve.
  - jacobush 8y ago
    Hybrid analog could achieve this. Film could have a picture printed on it, but instead of going for maximum resolution (down to grain level) it could have tiny tiny pixels, not visible to the eye, but readable by a computer. A kind of steganography, if you will. The data encoded in the steganography, could be any data of course, but for our purposes, it would be the image itself. Such a picture could be copied, with loss, in an analog fashion, but could also be copied perfectly in a digital fashion, by a digital reader capable of reading out the steganography information. Even the analog copy could be digitally read if it was of high enough quality. Another, more straightforward, way would be to print on the film, first an analog rendition of the picture, then a digitally encoded pattern of the same image, rinse repeat. But from a future safe perspective, it could be smart to encode the data itself inside the image, as per the first paragraph. Someone copying the images could unwittingly copy the digital data too!
    - adrianN 8y ago
      You might be interested in Optar http://ronja.twibright.com/optar/ http://ronja.twibright.com/optar/
      - jacobush 8y ago
        Indeed, thanks.
    - yourapostasy 8y ago
      > The data encoded in the steganography, could be any data of course, but for our purposes, it would be the image itself. This was an intriguing idea, but after I did some back of the envelope rough numbers, I tentatively concluded microform might not encode densely enough to accomplish that. It already has poor fidelity characteristics for photographic data. Right off the bat you are losing lots of information during the reducing process, and the film itself does not have the kind of qualities photographers look for. Color microform is also much more expensive than black and white. So the color film works well for newsprint photos, but for archival grade purposes, we're likely better off with the original film negatives. For deep storage archives, I could see a use case for microform, and with the encoding you suggested, but not quite at one picture per frame (page). Steganography relies upon imperceptibly embedding information into the picture, so there is a very low encoding density limit because we're working with analog pictures. Optar at 600 ppi uses an entire page to encode 200 Kb, but microfilm is closer to 300 ppi, so at 100 percent density with no picture, we can expect around 100 Kb per page of Optar-encoded binary data. An average smartphone picture size these days is around 5 MB, closer to 15-50 MB for DSLR pictures. Optar-encoding the smartphone picture, we are around 50 pages before we scatter it into the picture to steganographically encode it. Already we can see the picture has to be blown up across more than 50 pages and stitched back together. At imperceptible levels of encoding density, I'm guessing we're looking at 500+ pages, maybe. A standard microfiche holds about 98 pages, so we're talking about 6'ish fiches per picture at cellphone grade, much more for DSLR grade picture encoding. For high quality archival photograph storage, I think current librarians have it right: lots of duplicates of negatives, geographically separated, then service copy negatives off those masters, digitally scan in the most-used pictures off the service copy, and let patrons use the digital copies. Migrate the digital copies until we establish a universally-accepted digital bootstrap mechanism and process for Long Now-span time scales.

> But how often one checks checksum hash on copy? Your operating system does this.

imhoguy 8y ago

Only if you use ECC RAM and ZFS/Btrfs/ReFS. CRC e.g. in TCP is too weak.

altcognito 8y ago
Ah this is right, I was thinking of removable media, cdrom/dvd usually do this.
- tajen 8y ago
  USBs don’t do this. My md5 checksums fail all the time for big files on USBs. And USBs dont have SMART features.
zamadatix 8y ago
Yes, if one was going to reliably archive information they wouldn't use FAT32 on a single spinning disk. Does this really need to be pointed out? The point is digital gives you ways to have perfect copies, correct loss, and the ability to transfer to the next storage medium with no decay. Things analog could never achieve.
- jacobush 8y ago
  Hybrid analog could achieve this. Film could have a picture printed on it, but instead of going for maximum resolution (down to grain level) it could have tiny tiny pixels, not visible to the eye, but readable by a computer. A kind of steganography, if you will. The data encoded in the steganography, could be any data of course, but for our purposes, it would be the image itself. Such a picture could be copied, with loss, in an analog fashion, but could also be copied perfectly in a digital fashion, by a digital reader capable of reading out the steganography information. Even the analog copy could be digitally read if it was of high enough quality. Another, more straightforward, way would be to print on the film, first an analog rendition of the picture, then a digitally encoded pattern of the same image, rinse repeat. But from a future safe perspective, it could be smart to encode the data itself inside the image, as per the first paragraph. Someone copying the images could unwittingly copy the digital data too!
  - adrianN 8y ago
    You might be interested in Optar http://ronja.twibright.com/optar/ http://ronja.twibright.com/optar/
    - jacobush 8y ago
      Indeed, thanks.
  - yourapostasy 8y ago
    > The data encoded in the steganography, could be any data of course, but for our purposes, it would be the image itself. This was an intriguing idea, but after I did some back of the envelope rough numbers, I tentatively concluded microform might not encode densely enough to accomplish that. It already has poor fidelity characteristics for photographic data. Right off the bat you are losing lots of information during the reducing process, and the film itself does not have the kind of qualities photographers look for. Color microform is also much more expensive than black and white. So the color film works well for newsprint photos, but for archival grade purposes, we're likely better off with the original film negatives. For deep storage archives, I could see a use case for microform, and with the encoding you suggested, but not quite at one picture per frame (page). Steganography relies upon imperceptibly embedding information into the picture, so there is a very low encoding density limit because we're working with analog pictures. Optar at 600 ppi uses an entire page to encode 200 Kb, but microfilm is closer to 300 ppi, so at 100 percent density with no picture, we can expect around 100 Kb per page of Optar-encoded binary data. An average smartphone picture size these days is around 5 MB, closer to 15-50 MB for DSLR pictures. Optar-encoding the smartphone picture, we are around 50 pages before we scatter it into the picture to steganographically encode it. Already we can see the picture has to be blown up across more than 50 pages and stitched back together. At imperceptible levels of encoding density, I'm guessing we're looking at 500+ pages, maybe. A standard microfiche holds about 98 pages, so we're talking about 6'ish fiches per picture at cellphone grade, much more for DSLR grade picture encoding. For high quality archival photograph storage, I think current librarians have it right: lots of duplicates of negatives, geographically separated, then service copy negatives off those masters, digitally scan in the most-used pictures off the service copy, and let patrons use the digital copies. Migrate the digital copies until we establish a universally-accepted digital bootstrap mechanism and process for Long Now-span time scales.