5 ms·

Random question, but related: Like this example, I see lots of other applications that require a QR code storing binary data and chose to encoded this data as B

Random question, but related: Like this example, I see lots of other applications that require a QR code storing binary data and chose to encoded this data as Base64 (or others) and then add it to a ASCII-only QR code format. Why don't they use a binary-mode QR code? Compatibility?

sgtnoodle 5y ago
Perhaps so that generic QR code scanners can parse it and then know to forward it along to the appropriate app?
dmurray 5y ago
Almost always technical incompetence, I think. At some level "compatibility" is correct. System 1 outputs binary data, system 2 takes that and turns it into JSON, system 3 encodes that in base64, system 4 turns that into QR codes, and system 4 was what was mandated for producing all of the organisation's QR codes.
umurkontaci 5y ago
> The reason for representing Health Cards using Numeric Mode QRs instead of Binary Mode (Latin-1) QRs is information density: with Numeric Mode, 20% more data can fit in a given QR, vs Binary Mode. This is because the JWS character set conveys only log_2(65) bits per character (~6 bits); binary encoding requires log_2(256) bits per character (8 bits), which means ~2 wasted bits per character https://spec.smarthealth.cards/#encoding-chunks-as-qr-codes https://spec.smarthealth.cards/#encoding-chunks-as-qr-codes
- Dylan16807 5y ago
  Whoever wrote this makes my head hurt. If you were using binary storage, you wouldn't use "the JWS character set" (aka base64). You'd store it directly and have 0 wasted bits. Also, they're not acknowledging the bits wasted by their current system. Numeric mode in QR codes spends 3.33 bits per digit. At two digits per character, they're spending 6.67 bits to store only 6 bits of information.
zirror 5y ago
The EU's Digital Green Certificate [1] uses Base45, which I think they basically invented. They do this because "even in Byte mode a typical QR-code reader tries to interpret a byte sequence as a UTF-8 or ISO/IEC 8859-1 encoded text."[2] They use the 45 chars allowed by the alphanumeric-mode IIRC. [1]: https://github.com/eu-digital-green-certificates/dgc-overview https://github.com/eu-digital-green-certificates/dgc-overvie... [2]: https://datatracker.ietf.org/doc/draft-faltstrom-base45/ https://datatracker.ietf.org/doc/draft-faltstrom-base45/
- Dylan16807 5y ago
  Wait, why are they using 45 characters to encode 16 bits when you only need 41? Especially when they could have avoided % and space.
  - zirror 5y ago
    Well, tbh, I don't know the reason, I can only speculate. It is meant to encode pure binary data (cbor) and not to be human readable or URI or anything like that. It is specifically designed for QR Codes. I read through some Github issue [1] just now and as far as I can tell, it is more or less a design oversight, which might be remedied by a later version. [1]: https://github.com/ehn-dcc-development/hcert-spec/issues/64 https://github.com/ehn-dcc-development/hcert-spec/issues/64
    - Dylan16807 5y ago
      > It is meant to encode pure binary data (cbor) and not to be human readable or URI or anything like that. Well the reason to care is to avoid QR decoder quirkiness, otherwise you should probably just use the binary encoding. > I read through some Github issue [1] According to a comment near they end they were originally going to try to pack everything as a single base 45 bignum, so that half explains it. But not why they'd stick with 45 characters when changing that.
gnabgib 5y ago
It's a weird situation, but it's very obvious with HC1 and SHC cards. It starts with JWT. JSON is a human readable format (in utf-8), if humans don't need to read, the data could be binary, and the format could be exact. JSON isn't an exact spec, which is mostly inherited from JavaScript (there's no such thing as an integer only floating point, so 1e3==1000==1000.0==1.00e3 in human-readable form, as a stored number they are identical). Then there's differences in white-space (new lines, indentation) - although this could likely be overcome with convention. Because of this the JWT creators said instead of signing the data, we'll sign the exact representation in the payload - but of course with white-space and formatting variance (including a deserialize/serialize loops changing representations, or - in the case of bearer tokens, the HTTP spec allowing newlines/white space to be inserted at the protocol level) they had to encode it as non-human readable (base64). Now everyone agrees you're signing that exact Base64 representation of the JSON object. But! We've build a(n arguably verbose) human readable format that isn't readable by humans. The SHC spec (common in North America) actually holds a JWT that's signed by an elliptic curve private key. You can validate the signature with a public key. The public/private choice here is great, the JWT is terrible.. they've doubled down on the mistakes. Further to keep the QR smallish, they zipped the payload portion (which is supported by JWT - this is done before the base64 stage), and use only the minimum QR resilience setting (which is fine if it's on a screen, if it's printed this may lead to reading problems). Now we have human readable (JSON) compressed in machine readable (deflate) in machine readable (base 64) in machine readable (QR) - for machine reading purposes. They didn't even trim the fluff (every SHC begins with 56 because.. you guessed it, the `{` character), or use sensible choices (they don't use IssuedAt/iat, but NotBefore/nbf to indicate the generation date). Anyway, SHC (reasonably) noticed because of the (mostly) base64 encoding the character set is only 64 characters (6 bits) which doesn't use the ASCII space (7 bits) very well, so they store the first 'shc://' in ASCII and the rest is a number (there are three modes in QR: ASCII, binary, numeric - the density loosely matches binary representations - a numeric digit (0-9) takes 4 bits, ascii char takes 7 and binary takes 8). ASCII doesn't support the world very well, UTF8 isn't supported by QR (except as binary). In the SHC case, because it's signing a specific format/output of the JSON data, it doesn't have the white-space formatting concerns that JWTs have to overcome. If they wanted to stick with a JWT like format (JSON object), they could have skipped the base64 before sign step, at which point they might as well get rid of the header (we're no longer to JWT spec), deflate the message to be signed, and put the signature after the deflated message. All the same data, less of the overhead, and better use of the binary space.
TechBro8615 5y ago
The most common reason is probably because it’s easier to develop, debug and test code that outputs ASCII strings rather than binary blobs.

sgtnoodle 5y ago

Perhaps so that generic QR code scanners can parse it and then know to forward it along to the appropriate app?

dmurray 5y ago

Almost always technical incompetence, I think. At some level "compatibility" is correct. System 1 outputs binary data, system 2 takes that and turns it into JSON, system 3 encodes that in base64, system 4 turns that into QR codes, and system 4 was what was mandated for producing all of the organisation's QR codes.

umurkontaci 5y ago

> The reason for representing Health Cards using Numeric Mode QRs instead of Binary Mode (Latin-1) QRs is information density: with Numeric Mode, 20% more data can fit in a given QR, vs Binary Mode. This is because the JWS character set conveys only log_2(65) bits per character (~6 bits); binary encoding requires log_2(256) bits per character (8 bits), which means ~2 wasted bits per character https://spec.smarthealth.cards/#encoding-chunks-as-qr-codes https://spec.smarthealth.cards/#encoding-chunks-as-qr-codes

Dylan16807 5y ago
Whoever wrote this makes my head hurt. If you were using binary storage, you wouldn't use "the JWS character set" (aka base64). You'd store it directly and have 0 wasted bits. Also, they're not acknowledging the bits wasted by their current system. Numeric mode in QR codes spends 3.33 bits per digit. At two digits per character, they're spending 6.67 bits to store only 6 bits of information.

zirror 5y ago

The EU's Digital Green Certificate [1] uses Base45, which I think they basically invented. They do this because "even in Byte mode a typical QR-code reader tries to interpret a byte sequence as a UTF-8 or ISO/IEC 8859-1 encoded text."[2] They use the 45 chars allowed by the alphanumeric-mode IIRC. [1]: https://github.com/eu-digital-green-certificates/dgc-overview https://github.com/eu-digital-green-certificates/dgc-overvie... [2]: https://datatracker.ietf.org/doc/draft-faltstrom-base45/ https://datatracker.ietf.org/doc/draft-faltstrom-base45/

Dylan16807 5y ago
Wait, why are they using 45 characters to encode 16 bits when you only need 41? Especially when they could have avoided % and space.
- zirror 5y ago
  Well, tbh, I don't know the reason, I can only speculate. It is meant to encode pure binary data (cbor) and not to be human readable or URI or anything like that. It is specifically designed for QR Codes. I read through some Github issue [1] just now and as far as I can tell, it is more or less a design oversight, which might be remedied by a later version. [1]: https://github.com/ehn-dcc-development/hcert-spec/issues/64 https://github.com/ehn-dcc-development/hcert-spec/issues/64
  - Dylan16807 5y ago
    > It is meant to encode pure binary data (cbor) and not to be human readable or URI or anything like that. Well the reason to care is to avoid QR decoder quirkiness, otherwise you should probably just use the binary encoding. > I read through some Github issue [1] According to a comment near they end they were originally going to try to pack everything as a single base 45 bignum, so that half explains it. But not why they'd stick with 45 characters when changing that.

gnabgib 5y ago

It's a weird situation, but it's very obvious with HC1 and SHC cards. It starts with JWT. JSON is a human readable format (in utf-8), if humans don't need to read, the data could be binary, and the format could be exact. JSON isn't an exact spec, which is mostly inherited from JavaScript (there's no such thing as an integer only floating point, so 1e3==1000==1000.0==1.00e3 in human-readable form, as a stored number they are identical). Then there's differences in white-space (new lines, indentation) - although this could likely be overcome with convention. Because of this the JWT creators said instead of signing the data, we'll sign the exact representation in the payload - but of course with white-space and formatting variance (including a deserialize/serialize loops changing representations, or - in the case of bearer tokens, the HTTP spec allowing newlines/white space to be inserted at the protocol level) they had to encode it as non-human readable (base64). Now everyone agrees you're signing that exact Base64 representation of the JSON object. But! We've build a(n arguably verbose) human readable format that isn't readable by humans. The SHC spec (common in North America) actually holds a JWT that's signed by an elliptic curve private key. You can validate the signature with a public key. The public/private choice here is great, the JWT is terrible.. they've doubled down on the mistakes. Further to keep the QR smallish, they zipped the payload portion (which is supported by JWT - this is done before the base64 stage), and use only the minimum QR resilience setting (which is fine if it's on a screen, if it's printed this may lead to reading problems). Now we have human readable (JSON) compressed in machine readable (deflate) in machine readable (base 64) in machine readable (QR) - for machine reading purposes. They didn't even trim the fluff (every SHC begins with 56 because.. you guessed it, the `{` character), or use sensible choices (they don't use IssuedAt/iat, but NotBefore/nbf to indicate the generation date). Anyway, SHC (reasonably) noticed because of the (mostly) base64 encoding the character set is only 64 characters (6 bits) which doesn't use the ASCII space (7 bits) very well, so they store the first 'shc://' in ASCII and the rest is a number (there are three modes in QR: ASCII, binary, numeric - the density loosely matches binary representations - a numeric digit (0-9) takes 4 bits, ascii char takes 7 and binary takes 8). ASCII doesn't support the world very well, UTF8 isn't supported by QR (except as binary). In the SHC case, because it's signing a specific format/output of the JSON data, it doesn't have the white-space formatting concerns that JWTs have to overcome. If they wanted to stick with a JWT like format (JSON object), they could have skipped the base64 before sign step, at which point they might as well get rid of the header (we're no longer to JWT spec), deflate the message to be signed, and put the signature after the deflated message. All the same data, less of the overhead, and better use of the binary space.

TechBro8615 5y ago

The most common reason is probably because it’s easier to develop, debug and test code that outputs ASCII strings rather than binary blobs.