2 ms·

I want to use images to encode HTML or other arbitrary binary data, in a way that I call Fondant. https://news.ycombinator.com/item?id=17461955 https://news.yc

I want to use images to encode HTML or other arbitrary binary data, in a way that I call Fondant. https://news.ycombinator.com/item?id=17461955 https://news.ycombinator.com/item?id=17461955 I made a checkerboard pattern of 8x8 pixels, but I still lose data to colour bleeding when re-encoding (e.g. upload to Facebook or Save to Camera Roll on the iPhone). Is there some way I can use the "lossless" entropy coding to avoid this problem?

userbinator 8y ago
You can perturb the input of the DCT so that a specific IDCT implementation will generate the desired output. This technique applied to JPEG has been known for a few years: https://www.virusbulletin.com/virusbulletin/2015/03/script-lossy-stream https://www.virusbulletin.com/virusbulletin/2015/03/script-l... In general signal-processing terms, this is known as preemphasis/equalisation --- since the communications channel will distort the signal in a known way, by sending a signal distorted appropriately in the opposite direction, it will arrive "undistorted" at the destination. You can also consider using ECC codes, so that any small errors introduced can be corrected.
- peterburkimsher 8y ago
  Will that work for redistribution? I hope that users could save and re-share the photo many times and let it still be loadable.
  - deleted 8y ago
    [deleted]
  - anyfoo 8y ago
    To be truly resilient against any kind of (potentially unknown, even) lossy image compression, you would need to add a lot of redundancy. QR Codes are pretty much that, but their data needs to survive being “transmitted” through a blurry photo with arbitrary lighting, rotation and perspective, so they likely have a lot more redundancy than you realistically need. (EDIT: You might hit the limit where you need QR Code levels of efficiency relatively quickly, depending on the number of steps and how aggressive each recompression is.) If saving and sharing can mean changing the image dimensions however, you of course need redundancy by that factor. But yeah, that’s why we don’t choose lossy compression algorithms for binary data. If every bit counts, there is no entropy you can lose (though a lossless algorithm might find a more efficient representation, so to speak).
    - peterburkimsher 8y ago
      QR codes have a maximum size of 7,089 numerical digits - not enough to transmit a program or song. I'd be hoping to move about 5 MB.
      - snaily 8y ago
        Facebook, which was the use case you mentioned upthread, doesn't store 5MB images. You'd have to design for way less than 5MB, I should imagine.
      - anyfoo 8y ago
        That was kind of my point, was it not? QR Code might or might not have too much redundancy (depending on how resilient you want to be against what amount of sharing), but the general principle stands. Lossy compression removes information by design, you want to keep all bits of information, therefore you have to represent your information in a redundant (and therefore inefficient, by density) way.

userbinator 8y ago

You can perturb the input of the DCT so that a specific IDCT implementation will generate the desired output. This technique applied to JPEG has been known for a few years: https://www.virusbulletin.com/virusbulletin/2015/03/script-lossy-stream https://www.virusbulletin.com/virusbulletin/2015/03/script-l... In general signal-processing terms, this is known as preemphasis/equalisation --- since the communications channel will distort the signal in a known way, by sending a signal distorted appropriately in the opposite direction, it will arrive "undistorted" at the destination. You can also consider using ECC codes, so that any small errors introduced can be corrected.

peterburkimsher 8y ago
Will that work for redistribution? I hope that users could save and re-share the photo many times and let it still be loadable.
- deleted 8y ago
  [deleted]
- anyfoo 8y ago
  To be truly resilient against any kind of (potentially unknown, even) lossy image compression, you would need to add a lot of redundancy. QR Codes are pretty much that, but their data needs to survive being “transmitted” through a blurry photo with arbitrary lighting, rotation and perspective, so they likely have a lot more redundancy than you realistically need. (EDIT: You might hit the limit where you need QR Code levels of efficiency relatively quickly, depending on the number of steps and how aggressive each recompression is.) If saving and sharing can mean changing the image dimensions however, you of course need redundancy by that factor. But yeah, that’s why we don’t choose lossy compression algorithms for binary data. If every bit counts, there is no entropy you can lose (though a lossless algorithm might find a more efficient representation, so to speak).
  - peterburkimsher 8y ago
    QR codes have a maximum size of 7,089 numerical digits - not enough to transmit a program or song. I'd be hoping to move about 5 MB.
    - snaily 8y ago
      Facebook, which was the use case you mentioned upthread, doesn't store 5MB images. You'd have to design for way less than 5MB, I should imagine.
    - anyfoo 8y ago
      That was kind of my point, was it not? QR Code might or might not have too much redundancy (depending on how resilient you want to be against what amount of sharing), but the general principle stands. Lossy compression removes information by design, you want to keep all bits of information, therefore you have to represent your information in a redundant (and therefore inefficient, by density) way.