5 ms·

Show HN: Cimbar – File transfer via color barcodes and the Android camera

87 points by sz3 6y ago

sz3 6y ago
Hello all, You've seen QR codes, maybe Microsoft HCCB [0], maybe jabcode [1] – well, here's a prototype of something new for the pile. :) I saw txqr [2] a while back and was impressed, but also curious about how much throughput was possible with animated bar codes. I may have gotten carried away in my research into the question. cimbar is a single and multi-frame color barcode format using reed solomon (for now) and wirehair [3] for error correction. Files are encoded into an animated series of bar codes drawn to the display. Files are decoded by an Android app [4] with its camera pointed at the cimbar code. This works with all antennas off, e.g. in airplane mode, because it's only using the visual data channel. I ported the encoder to wasm, because I could: https://cimbar.org https://cimbar.org Sustained transfer speed is currently on the order of 800 kilobits/second. So it's not very practical for files larger than a couple of MB, unless you have a lot of free time. :) [0] https://en.wikipedia.org/wiki/High_Capacity_Color_Barcode https://en.wikipedia.org/wiki/High_Capacity_Color_Barcode [1] https://github.com/jabcode/jabcode https://github.com/jabcode/jabcode [2] https://github.com/divan/txqr https://github.com/divan/txqr [3] https://github.com/catid/wirehair https://github.com/catid/wirehair [4] https://github.com/sz3/cfc/releases/latest https://github.com/sz3/cfc/releases/latest
- ignoramous 6y ago
  As a possible tangent, convert those images to inaudible audio to transfer files over the air: https://github.com/ganny26/awesome-audioqr https://github.com/ganny26/awesome-audioqr (air-gapped computers be damned)
  - sz3 6y ago
    Well, there's diminishing returns in format shifting. The encoded barcode contains various types of quasi-redundant visual information (e.g. error correction codes) to allow decoding to happen, so for audio-based transfer it'd be better to skip the image encode and blast out the file directly. That said... given the somewhat remarkable way fountain codes work, there's nothing stopping us from having a protocol that uses the audio and the video channels simultaneously for better throughput...
    - rmetzler 6y ago
      Would it be beneficial to use one channel (either audio or visual) to transmit the information and the other one for responses like acknowledge? So kind of like TCP over two different channels?
      - sz3 6y ago
        Well, as for acks specifically -- cimbar itself doesn't really need them, thanks to fountain codes [0]. But I can imagine a reverse (request?) channel being useful, if it had enough bandwidth for the desired application. :) As /u/ggerganov notes elsewhere in this thread (with some expertise on the audio side -- I can't claim any), the bandwidth of any audio channel is probably going to be pretty bad. edit: Notwithstanding how viable of an idea it might be, HTTP over audio+video would be pretty neat. :) [0] https://en.wikipedia.org/wiki/Fountain_code https://en.wikipedia.org/wiki/Fountain_code
  - ggerganov 6y ago
    I don't think air-gapped audio transmission could ever reach a fast and reliable transmission so that it allows transferring files in reasonable amount of time over reasonable transmitter-receiver distances. It's just too many hardware and physical limitations for this approach. Having said that, I am actually working on a small library for data-over-sound which can be used for small data chunk transmissions across the room [0]. [0] https://github.com/ggerganov/ggwave https://github.com/ggerganov/ggwave
- ktpsns 6y ago
  Amazing! Do you know any variants of these which squeeze out the maximum transfer rate over a high quality connection? Thinking about data transfer from a VPN/RDP remote desktop where I want to grab the screen output on the client with very little loss in quality (basically only limited by the chosen video encoder).
  - mleonhard 6y ago
    Using it to transfer files over a remote desktop connection is like a graphical version of zmodem [0]. [0] https://en.wikipedia.org/wiki/ZMODEM https://en.wikipedia.org/wiki/ZMODEM
  - sz3 6y ago
    I don't think I do? I'm not 100% sure I understand your example. Is it RDP server (remote) -> RDP client (local) -> cell phone?
donclark 6y ago
Is there a video of this in action?
- learn_more 6y ago
  It's simple to try out by uploading a file via the web page at https://cimbar.org https://cimbar.org
  - 6510 6y ago
    I have no android device.
    - desdiv 6y ago
      I just dragged a file into the site on my desktop computer and it worked. OP said the site uses the wasm implementation so any browser with javascript enabled should be able to encode files. Decoding needs the Android app.
- sz3 6y ago
  Hmm... no, but that's a good idea!
VikingCoder 6y ago
I'm a HUGE fan of air-gap data transfer, and this is very nearly what I want. Curious though, can't you make a web app to record the video, too?
- sz3 6y ago
  Definitely. I had to draw a line somewhere for an initial "does anyone care about this?" release, so there's some rough edges here. There are a number of loose ends that I'd like to look into down the road, and more flexibility for different use cases is on the list. edit: I should mention: the main reason the decoder is a native Android app and not a WebAssembly app is that decoding performance is a throughput bottleneck. I wasn't too eager to pay the wasm tax, when I wasn't sure if even native performance would be good enough. As it happens, I now think decode performance is going to be ok -- but it's still a bit of a weak spot in the scheme.
ggerganov 6y ago
I tried the browser encoder and here is what it looks like: https://youtu.be/RkpVhW7SuRY https://youtu.be/RkpVhW7SuRY Wonder if someone can decode the .mp4 file from the video. It's not rickroll, I promise ;-)
- sz3 6y ago
  I like the way you think. :) Though I think 720p is towards the low end of resolutions this'll work on -- the cimbar code itself is 1024x1024, and while there's some wiggle room (hamming distance between the symbols), in my experience it gets pretty iffy below 900x900. Also... I should probably update the overlay to auto-hide when a file is selected -- it helps a lot to have the full color range.
anonytrary 6y ago
You might want to have a picture on the main README as in https://github.com/sz3/libcimbar/blob/master/DETAILS.md https://github.com/sz3/libcimbar/blob/master/DETAILS.md. Had to fish for a picture...
- sz3 6y ago
  Probably a good idea. Noted.
cors-fls 6y ago
Very interesting but couldn't get it to work with my smartphone. Even with a 250Ko image file, I got bored before it finished ! My camera may be too low quality and/or my monitor too small. A barcode size parameter may be useful (even though it reduces throughput) so that it is easier to scan on all devices.
- sz3 6y ago
  Multiple sizes isn't a bad idea. I'm not sure I know enough about what sizes would be appropriate though. :) Transfers getting stuck like that usually is an indication that the decoder can't reliably find the 3+1 corner pattern. I might add a toggle to the decoder app that trades speed for more reliability. That said, there are some failure modes that are harder to fix: I had a mysteriously slow transfer that was driving me nuts, until I noticed that my mouse cursor was on top of one of the corners.
max_ 6y ago
I heard ordinary QR codes can hold up to 3KB of data. How much data can this one hold?
- sz3 6y ago
  The simple answer is: 4-color (standard): 7500 bytes 8-color: 8750 bytes monochrome: 5000 bytes Those numbers are with the standard, fairly high ECC setting (~20% of the image) that I settled on for video-based transfer. If we want to be more aggressive and use half the error correction: 4-color (standard): 8400 bytes 8-color: 9800 bytes monochrome: 5600 bytes
mleonhard 6y ago
Is it really phone CPU performance that limits the bandwidth? I expected it would be the camera.
- sz3 6y ago
  Right now, it's both. The decoder has a lot of image processing work to do (intriguingly well-suited for the GPU), and also lots of popcnts. I've optimized it a fair bit, but there's probably some tricks I still need to learn. It turns out that mobile processors don't like heat very much, and blow out their cache much quicker than you'd hope. :) In the long run, I think your intuition is correct. The hard physics of the camera constraints (exposure time, etc) will put a hard upper bound on FPS+fidelity, and thus bandwidth.

87 points by sz3 6y ago

sz3 6y ago

Hello all, You've seen QR codes, maybe Microsoft HCCB [0], maybe jabcode [1] – well, here's a prototype of something new for the pile. :) I saw txqr [2] a while back and was impressed, but also curious about how much throughput was possible with animated bar codes. I may have gotten carried away in my research into the question. cimbar is a single and multi-frame color barcode format using reed solomon (for now) and wirehair [3] for error correction. Files are encoded into an animated series of bar codes drawn to the display. Files are decoded by an Android app [4] with its camera pointed at the cimbar code. This works with all antennas off, e.g. in airplane mode, because it's only using the visual data channel. I ported the encoder to wasm, because I could: https://cimbar.org https://cimbar.org Sustained transfer speed is currently on the order of 800 kilobits/second. So it's not very practical for files larger than a couple of MB, unless you have a lot of free time. :) [0] https://en.wikipedia.org/wiki/High_Capacity_Color_Barcode https://en.wikipedia.org/wiki/High_Capacity_Color_Barcode [1] https://github.com/jabcode/jabcode https://github.com/jabcode/jabcode [2] https://github.com/divan/txqr https://github.com/divan/txqr [3] https://github.com/catid/wirehair https://github.com/catid/wirehair [4] https://github.com/sz3/cfc/releases/latest https://github.com/sz3/cfc/releases/latest

ignoramous 6y ago
As a possible tangent, convert those images to inaudible audio to transfer files over the air: https://github.com/ganny26/awesome-audioqr https://github.com/ganny26/awesome-audioqr (air-gapped computers be damned)
- sz3 6y ago
  Well, there's diminishing returns in format shifting. The encoded barcode contains various types of quasi-redundant visual information (e.g. error correction codes) to allow decoding to happen, so for audio-based transfer it'd be better to skip the image encode and blast out the file directly. That said... given the somewhat remarkable way fountain codes work, there's nothing stopping us from having a protocol that uses the audio and the video channels simultaneously for better throughput...
  - rmetzler 6y ago
    Would it be beneficial to use one channel (either audio or visual) to transmit the information and the other one for responses like acknowledge? So kind of like TCP over two different channels?
    - sz3 6y ago
      Well, as for acks specifically -- cimbar itself doesn't really need them, thanks to fountain codes [0]. But I can imagine a reverse (request?) channel being useful, if it had enough bandwidth for the desired application. :) As /u/ggerganov notes elsewhere in this thread (with some expertise on the audio side -- I can't claim any), the bandwidth of any audio channel is probably going to be pretty bad. edit: Notwithstanding how viable of an idea it might be, HTTP over audio+video would be pretty neat. :) [0] https://en.wikipedia.org/wiki/Fountain_code https://en.wikipedia.org/wiki/Fountain_code
- ggerganov 6y ago
  I don't think air-gapped audio transmission could ever reach a fast and reliable transmission so that it allows transferring files in reasonable amount of time over reasonable transmitter-receiver distances. It's just too many hardware and physical limitations for this approach. Having said that, I am actually working on a small library for data-over-sound which can be used for small data chunk transmissions across the room [0]. [0] https://github.com/ggerganov/ggwave https://github.com/ggerganov/ggwave
ktpsns 6y ago
Amazing! Do you know any variants of these which squeeze out the maximum transfer rate over a high quality connection? Thinking about data transfer from a VPN/RDP remote desktop where I want to grab the screen output on the client with very little loss in quality (basically only limited by the chosen video encoder).
- mleonhard 6y ago
  Using it to transfer files over a remote desktop connection is like a graphical version of zmodem [0]. [0] https://en.wikipedia.org/wiki/ZMODEM https://en.wikipedia.org/wiki/ZMODEM
- sz3 6y ago
  I don't think I do? I'm not 100% sure I understand your example. Is it RDP server (remote) -> RDP client (local) -> cell phone?

donclark 6y ago

Is there a video of this in action?

learn_more 6y ago
It's simple to try out by uploading a file via the web page at https://cimbar.org https://cimbar.org
- 6510 6y ago
  I have no android device.
  - desdiv 6y ago
    I just dragged a file into the site on my desktop computer and it worked. OP said the site uses the wasm implementation so any browser with javascript enabled should be able to encode files. Decoding needs the Android app.
sz3 6y ago
Hmm... no, but that's a good idea!

VikingCoder 6y ago

I'm a HUGE fan of air-gap data transfer, and this is very nearly what I want. Curious though, can't you make a web app to record the video, too?

sz3 6y ago
Definitely. I had to draw a line somewhere for an initial "does anyone care about this?" release, so there's some rough edges here. There are a number of loose ends that I'd like to look into down the road, and more flexibility for different use cases is on the list. edit: I should mention: the main reason the decoder is a native Android app and not a WebAssembly app is that decoding performance is a throughput bottleneck. I wasn't too eager to pay the wasm tax, when I wasn't sure if even native performance would be good enough. As it happens, I now think decode performance is going to be ok -- but it's still a bit of a weak spot in the scheme.

ggerganov 6y ago

I tried the browser encoder and here is what it looks like: https://youtu.be/RkpVhW7SuRY https://youtu.be/RkpVhW7SuRY Wonder if someone can decode the .mp4 file from the video. It's not rickroll, I promise ;-)

sz3 6y ago
I like the way you think. :) Though I think 720p is towards the low end of resolutions this'll work on -- the cimbar code itself is 1024x1024, and while there's some wiggle room (hamming distance between the symbols), in my experience it gets pretty iffy below 900x900. Also... I should probably update the overlay to auto-hide when a file is selected -- it helps a lot to have the full color range.

anonytrary 6y ago

You might want to have a picture on the main README as in https://github.com/sz3/libcimbar/blob/master/DETAILS.md https://github.com/sz3/libcimbar/blob/master/DETAILS.md. Had to fish for a picture...

sz3 6y ago
Probably a good idea. Noted.

cors-fls 6y ago

Very interesting but couldn't get it to work with my smartphone. Even with a 250Ko image file, I got bored before it finished ! My camera may be too low quality and/or my monitor too small. A barcode size parameter may be useful (even though it reduces throughput) so that it is easier to scan on all devices.

sz3 6y ago
Multiple sizes isn't a bad idea. I'm not sure I know enough about what sizes would be appropriate though. :) Transfers getting stuck like that usually is an indication that the decoder can't reliably find the 3+1 corner pattern. I might add a toggle to the decoder app that trades speed for more reliability. That said, there are some failure modes that are harder to fix: I had a mysteriously slow transfer that was driving me nuts, until I noticed that my mouse cursor was on top of one of the corners.

max_ 6y ago

I heard ordinary QR codes can hold up to 3KB of data. How much data can this one hold?

sz3 6y ago
The simple answer is: 4-color (standard): 7500 bytes 8-color: 8750 bytes monochrome: 5000 bytes Those numbers are with the standard, fairly high ECC setting (~20% of the image) that I settled on for video-based transfer. If we want to be more aggressive and use half the error correction: 4-color (standard): 8400 bytes 8-color: 9800 bytes monochrome: 5600 bytes

mleonhard 6y ago

Is it really phone CPU performance that limits the bandwidth? I expected it would be the camera.

sz3 6y ago
Right now, it's both. The decoder has a lot of image processing work to do (intriguingly well-suited for the GPU), and also lots of popcnts. I've optimized it a fair bit, but there's probably some tricks I still need to learn. It turns out that mobile processors don't like heat very much, and blow out their cache much quicker than you'd hope. :) In the long run, I think your intuition is correct. The hard physics of the camera constraints (exposure time, etc) will put a hard upper bound on FPS+fidelity, and thus bandwidth.