10 ms·
I recall this plaguing the Secure Scuttlebutt project -- they were signing JSON objects that were encoded however Node saw fit to encode them, which meant that
by nemo1618 7y ago
I recall this plaguing the Secure Scuttlebutt project -- they were signing JSON objects that were encoded however Node saw fit to encode them, which meant that in order to write an alternative implementation of the protocol, you had to encode all the various objects exactly how Node encodes them, which is (unsurprisingly) non-trivial. I wonder if they addressed this in a later version.
- Vendan 7y agoAs one of the people that built a alternative implementation, this is completely true, but also much worse, cause there were 2 things around crypto, signing and hashing, and one encoded it one way, and the other encoded it a different way (encoding as far as utf16 and such) which meant that for ascii, everything just worked fine, but as soon as you started using non-ascii, everything got 2x as complex.
- tracker1 7y agoIf you're serializing the JSON, that is UTF8. You're signing UTF8 bytes... don't mess with them, and the signature will be the same... also, for JWT, the UTF8 is then converted to base64 representation and tethered to the signature. You're signing the UTF8 bytes, not JSON. It doesn't matter how it's serialized, if it isn't UTF8, you're doing it wrong. The order or properties doesn't matter, the signature is on the bytes.
- deleted 7y ago[deleted]
- Vendan 7y agoSSB signs objects like so: { "previous": "%XphMUkWQtomKjXQvFGfsGYpt69sgEY7Y4Vou9cEuJho=.sha256", "author": "@FCX/tsDLpubCPKKfIrw4gc+SQkHcaD17s7GI6i/ziWY=.ed25519", "sequence": 2, "timestamp": 1514517078157, "hash": "sha256", "content": { "type": "post", "text": "Second post!" } } gets signed like { "previous": "%XphMUkWQtomKjXQvFGfsGYpt69sgEY7Y4Vou9cEuJho=.sha256", "author": "@FCX/tsDLpubCPKKfIrw4gc+SQkHcaD17s7GI6i/ziWY=.ed25519", "sequence": 2, "timestamp": 1514517078157, "hash": "sha256", "content": { "type": "post", "text": "Second post!" }, "signature": "z7W1ERg9UYZjNfE72ZwEuJF79khG+eOHWFp6iF+KLuSrw8Lqa6 IousK4cCn9T5qFa8E14GVek4cAMmMbjqDnAg==.sig.ed25519" } which means that you have to be careful about how you remove the signature in order to verify the original. The "main" node implementation does this by parsing the json, removing the field, and then re serializing, forcing any alternate implementation to exactly match the node serialization in order to be compatible
- wyldfire 7y agoGee, I would just naively wonder whether the scope of the signature includes the outermost closing brace. Does it?
- Vendan 7y agoAt least when I implemented it, it was the full message, as a complete valid json object. The signature would get added as another field, and then the object reserialized.
- tracker1 7y agoThis is why JWT is better... JWT signs "header.body" with each part as base64 from the JSON, and joined with a period. The content in the body and header are immaterial.
- lvh 7y agoIt's not "better", it solves a materially different problem. The article already acknowledges that if you can afford to, you should just stick a tag on the outside. (That's what JWT does, but the actual thing recommended, just HMAC, is better still, for reasons mentioned elsewhere in the comments.)
- juliusmusseau 7y agoThe one approach you missed on that blog post: put an envelope inside an envelope. Does anyone ever do this? { "original":"json", "goes":"here" } Signed: { "original_json_contents_base64":"ewogICJvcmlnaW5hbCI6Impzb24iLAogICJnb2VzIjoiaGVyZSIKfQo=", "hmac_sha256_of_base64":"bf1f4cb95ce8633aff46888e1717873e32bb2a770b3d4b5b74a59e5e9adefeda" } This way you have full control over the raw bytes you want to sign (by forcing them into Base64 where other systems can't get their dirty paws on them). I guess the problem here is if intermediate systems want to do stuff based on the payload (but without validating it), they won't like this. But if the problem is just intermediate systems barfing on non-json, this might work! p.s. enjoyable blog post - as they always are! ;-)
- lvh 7y agoThat's true if you have the luxury of a traditional tag on the outside, but falls apart when you have systems where signatures are in-band, like SAML, which is where canonicalization shows up. (Both for UTF-8 itself, which isn't canonical, but especially for your serialization format e.g. XML/JSON, which is usually far hairier.)
- tracker1 7y agoIf you treat the original UTF8 as bytes, then you still have that as bytes... this is part of why JWT uses base64 around the JSON string's bytes. In any case, you don't create a signature on a string per se, you create it on bytes, even if those bytes are derived from a string. Also, you don't validate a signature by re-creating the bytes in question... that's a flawed approach, and not the approach for example JWT takes.
- lvh 7y ago> In any case, you don't create a signature on a string per se, you create it on bytes, even if those bytes are derived from a string. Everyone is on the same page that at some point bytes go into a hash function. That's not the problem. You mention JWT, but JWT only does external signing, which doesn't trigger the problematic case several people are describing to you. Perhaps an example would be more useful. If you start with a JSON like: {"a": 1} how do you build a JSON like: {"a": 1, "tag": "deadbeefdeadbeefdeadbeef"} with a signing and verification algorithm that works? > Also, you don't validate a signature by re-creating the bytes in question... that's a flawed approach, and not the approach for example JWT takes. Can you describe an HMAC validation process that doesn't involve recreating the bytes in the HMAC tag?
- tracker1 7y agoYou don't... you could create one like... {"body":"base64-json","tag":"hash"} It seems to me that trying to do what you're saying is a flawed approach.
- 7y ago
- steveklabnik 7y agoJSON is not always guaranteed to be UTF-8. It wasn't updated to mandate that until 2017 https://www.tbray.org/ongoing/When/201x/2017/12/14/RFC-8259-STD-90 https://www.tbray.org/ongoing/When/201x/2017/12/14/RFC-8259-... That said, I certainly hope that it is, generally speaking.
- tracker1 7y agoThe original spec said UTF-8/16/32, but unaware of any reference implementation that used anything other than UTF-8 ... though, who knows with hand rolled crap, and windows in the mix.
- Vendan 7y agohttps://tc39.es/ecma262/#sec-json.stringify https://tc39.es/ecma262/#sec-json.stringify The stringify function returns a String in UTF-16 encoded JSON format representing an ECMAScript value, or undefined.
- masklinn 7y agoPython's `json` returns text, which you can encode however you want.