3 ms·

I understand that the goal is to make the data survive video compression, but wouldn't it make sense to use at least some color information instead of entirely

I understand that the goal is to make the data survive video compression, but wouldn't it make sense to use at least some color information instead of entirely black and white pixels?

trklausss 4y ago
Seems that you did not read the README: There are two encoding modes, RGB and B/W. It uses a pixel-to-data width of 2x2, but says YouTube's compression algorithm is brutal, and one corrupted pixel already renders the whole thing corrupted.
- Wowfunhappy 4y ago
  Using full RGB clearly wouldn't work, but I do wonder if you could use the color channel for something, possibly redundancy information.
  - einr 4y ago
    Very likely you could get away with at least 4 bits (16 distinct colors) per pixel, which is 4x more efficient than pure black and white.
    - LocalH 4y ago
      I think the size of the effective chroma metapixels is more important than the range of values. You need to make them larger in order to keep the decoder from blending them together when upscaling the 4:2:0 chroma. Now, if you're using a 4:4:4 format to do this, then you should be able to use smaller chroma metapixels (I still wouldn't use the full chroma resolution, though, unless you're using a high bitrate or a lossless codec). However, that risks data corruption if passed through a pipeline that downsamples the chroma.
Oxidation 4y ago
Seems like it could benefit from forward error correction to defend against bit errors (this is how QR codes survive big chunks being partially obscured or replaced by logos, and also how CDs survive being scratched within certain limits).
- naikrovek 4y ago
  that error correction greatly inflates the final size of the QR code, too. there should be some error correction in a system like this, though.
  - Oxidation 4y ago
    You can choose how much correction you get, in terms of how many bit errors you can correct per 'n' bits. And you need surprisingly few bits to get pretty great performance under "reasonable" bit-error rate channel (like under 10% overhead). You can wind up the strength of the error correction if you anticipate a noisier channel. QR codes have 4 levels of correction you can use depending on how robust you wish them to be. CDs and DVDs use two chained, fixed, levels to keep the decoders simple. CDs have 25% overhead, but their correction is very strong: they can correct 4000 bits in a row.
LocalH 4y ago
Chroma is lossier than luma in most common video codecs. AVC is 4:2:0 on YouTube. 4:2:0, quite confusingly, means that chroma is halved in both dimensions compared to luma (so one chroma pixel is congruent with four luma pixels). As well, most decoders will apply filtering on the chroma to upsample it to match the luma, meaning that your color boundaries are going to be indistinct at best, and you might even lose the original chroma values entirely in the process. You'd have to use multiple chroma pixels as one metapixel in order to increase resilience, which would diminish the capacity. With modern codecs, a monochrome signal seems better to use for actual data, although I could see it being useful to use chroma for metadata.

I understand that the goal is to make the data survive video compression, but wouldn't it make sense to use at least some color information instead of entirely black and white pixels?

trklausss 4y ago

Seems that you did not read the README: There are two encoding modes, RGB and B/W. It uses a pixel-to-data width of 2x2, but says YouTube's compression algorithm is brutal, and one corrupted pixel already renders the whole thing corrupted.

Wowfunhappy 4y ago
Using full RGB clearly wouldn't work, but I do wonder if you could use the color channel for something, possibly redundancy information.
- einr 4y ago
  Very likely you could get away with at least 4 bits (16 distinct colors) per pixel, which is 4x more efficient than pure black and white.
  - LocalH 4y ago
    I think the size of the effective chroma metapixels is more important than the range of values. You need to make them larger in order to keep the decoder from blending them together when upscaling the 4:2:0 chroma. Now, if you're using a 4:4:4 format to do this, then you should be able to use smaller chroma metapixels (I still wouldn't use the full chroma resolution, though, unless you're using a high bitrate or a lossless codec). However, that risks data corruption if passed through a pipeline that downsamples the chroma.

Oxidation 4y ago

Seems like it could benefit from forward error correction to defend against bit errors (this is how QR codes survive big chunks being partially obscured or replaced by logos, and also how CDs survive being scratched within certain limits).

naikrovek 4y ago
that error correction greatly inflates the final size of the QR code, too. there should be some error correction in a system like this, though.
- Oxidation 4y ago
  You can choose how much correction you get, in terms of how many bit errors you can correct per 'n' bits. And you need surprisingly few bits to get pretty great performance under "reasonable" bit-error rate channel (like under 10% overhead). You can wind up the strength of the error correction if you anticipate a noisier channel. QR codes have 4 levels of correction you can use depending on how robust you wish them to be. CDs and DVDs use two chained, fixed, levels to keep the decoders simple. CDs have 25% overhead, but their correction is very strong: they can correct 4000 bits in a row.

LocalH 4y ago

Chroma is lossier than luma in most common video codecs. AVC is 4:2:0 on YouTube. 4:2:0, quite confusingly, means that chroma is halved in both dimensions compared to luma (so one chroma pixel is congruent with four luma pixels). As well, most decoders will apply filtering on the chroma to upsample it to match the luma, meaning that your color boundaries are going to be indistinct at best, and you might even lose the original chroma values entirely in the process. You'd have to use multiple chroma pixels as one metapixel in order to increase resilience, which would diminish the capacity. With modern codecs, a monochrome signal seems better to use for actual data, although I could see it being useful to use chroma for metadata.