6 ms·
Twibright Optar – OPTical ARchiver – a codec for encoding data on paper
- rahimnathwani 5y agoThis reminds me of the Danmere Backer, which was for backing up data to VHS tapes.
- EvanAnderson 5y agoPrevious discussion of PaperBack (which, in turn, references Optar): https://news.ycombinator.com/item?id=10245836 https://news.ycombinator.com/item?id=10245836
- webmaven 5y agoThis reminds me of Xerox DataGlyphs: https://microglyphs.com/english/html/dataglyphs.shtml https://microglyphs.com/english/html/dataglyphs.shtml
- rubicks 5y ago> Reducing the space necessary to keep accounting records that are mandatory to be kept on paper I can't wait to see the ensuing hilarity when this is challenged in court.
- mmastrac 5y agoI realize that courts are looking at the spirit of the law and don't look kindly on loopholes, but this seems like it would be considered a reasonable option by a reasonable person, assuming you go through the diligence of ensuring the software remains available. Generally the paper retention laws I've worked with have more required that documents be produceable on demand on paper (and associated fined for failure).
- wazoox 5y agoThere are systems now to archive masses of data on film. Contrary to ordinary printed paper, film can easily last several decades and even centuries.
- AlotOfReading 5y agoPaper can also easily last centuries. I have some century old paper in my house, just sitting on the bookshelf. There are archives with examples that are upwards of a thousand years old. If we include parchments, we have surviving documents predating the pyramids.
- deleted 5y ago[deleted]
- zozbot234 5y agoI disagree, microfilms from only a few decades ago have degraded to the point of being near-illegible. Even low-quality paper has been far more reliable than that.
- mburee 5y agoOn that note, "Double Fold" by Nicholson Baker
- wazoox 5y agoI'm talking about 35mm film infrequently accessed, not microfilm which is mostly degraded by repeated reading. Typical laser-printed paper doesn't last more than a few decades AFAIK. Color 35mm film is stable for at least 60 years in ordinary conditions and B&W film 100 years and more.
- 8bitsrule 5y ago> Typical laser-printed paper doesn't last Many options there. I'm more concerned about printer-ink longevity. Tough to chase that down (recent tech). This page about preserving photos looks relevant; recommends coated halide or ink-jet for 100+ years) [https://www.shutterbug.com/content/how-long-will-your-digital-prints-lastbryou-may-be-surprised-page-2 https://www.shutterbug.com/content/how-long-will-your-digita...]. Paper's been around forever, so have long-lasting inks ... but printer-ink is too new. Typewriters have been around over a century ... haven't seen studies!
- WinterMount223 5y agoHow does this compare to printed base 64, then OCR? It seems more robust to print and recognize text than binary patterns. You can always type it out if you are desperate.
- rubicks 5y ago3750 characters/page at 6 bits/character gets you 22.5 kbits/page or ~2.8kB/page. Optar apparently does 200kB/page.
- 41b696ef1113 5y agoAt minimum, I would want to use one of the encodings that avoids oO0il1 ambiguity. Maybe settle for hex in a pinch.
- 41b696ef1113 5y agoI am reminded of NanoRosetta[0] which will engrave your data onto a physical ~coin sized object. [0] https://nanorosetta.com/ https://nanorosetta.com/
- throwaway81523 5y agoThis is pretty cool and I remember seeing something similar some years back. It would be nice to add a long range erasure code on top of the FEC, in case there is an ink splotch or the like.
- dragontamer 5y agoGolay code? I feel like with 200kb sizes, Reed Solomon would be better? Definitely a fun idea. 200KB per page is reasonable and much more than I thought was possible (I never did the math though). Maybe for actual pragmatic use, 50% Reed Solomon code for ~100Kb per page is still good? 100KB of data and 100KB of Reed Solomon code could correct up to 50KB of errors. Error detection (CRC32?) to reliably throw away errors (Pararchive style) also could help. -------- Or maybe just packaging the data in Pararchive before writing to paper would work, lol
- idealmedtech 5y agoIf you're using the standard RS(223,32) code you actually can get much more data than just half! Only 32 bytes in a 255 byte block are used for parity. We used it extensively when designing a comms protocol for LEO usage!
- dragontamer 5y agoWell, very large codes like RS(150000, 50000) might be a bit expensive computationally, but computers today are really really fast lol. I think codes of that size are some kind of low density parity check or something these days. Not something I understand, but RS codes are a big matrix multiplication: O(N^3) so maybe not practical lol. Don't mind me, I'm basically talking out of my ass on this one. ------------- EDIT: I'm remembering college a bit more. Code interleaving and 2 dimensional codes are used for CD ROMs. Reed Solomon works best if errors are spread out over many frames, so there are patterns you can do to make this sort of 2D printing more resilient vs 2 dimensional burst errors (ex: spilled coffee, or page tears, streaking printers)
- namibj 5y agoOne of the three exponents is due to data length, though. So "only" quadratic cost in the spreading distance of the error correction code.
- eternityforest 5y agoI think CDs use two layers of codes interleaved in some non-obvious pattern. Errors in the smaller blocks that take out a whole block are actually spread amongst multiple large blocks, and you know what bits are bad so you're correcting erasures instead of errors, which is easier.
- canadaduane 5y agoAre there any error correction systems that would make a 3cm x 3cm hole anywhere on the page still allow the full page to be completely recoverable? Or "holographic" error correction, where even if 90% of the page is gone, you could still get the "gist" of it somehow? (I'm not sure what I'm asking for mathematically, but probing for ideas that are out there.)
- deoxxa 5y agoErasure coding (particularly fountain codes) are what you'd want to use.
- mbreese 5y agoYou’d have to be clever about the physical distance between overlapping codes. You’d hate to see a 3cm artifact (cut, stain, spill, etc) take out a whole unit/block. (Assuming you’d print multiple blocks on a page.)
- bawolff 5y ago> Sheet music is sound storage which is very space inefficient and allows only MIDI capability. Twibright Optar stores digital music with full digital sound capability. Recommended format is 32kbps AoTuV Ogg Vorbis which allows storage of around 45 seconds of music per page with acceptable quality. Musical skill is not necessary for playback anymore. And music album can now literally be an album. Lol, i wonder with opus if you could have an audiobook thats thinner than the book.