3 ms·

Ask HN: JSON RFC 8785 Serialization

2 points by jzombie 3y ago

This seems like such a trivial question to ask, but a project at my work uses a custom HTTP message request signer implemented in JavaScript, which they're making an API for, and I wrote a Python implementation of this and was surprised how much I had to tweak json.dumps to produce a JSON string to match the same signature. I went ahead and duplicated this to PHP and Ruby, writing some unit tests, which all seem to pass, but I'm curious if anyone else has experienced anything of the like and what they did. ChatGPT recommends using JSON Canonicalization Algorithm (JCA) RFC 8785, which seems legit at first glance. While I'm sure this will work all just well and good, I'm curious to hear anyone what kinds of trials and errors anyone else has gone through on such a seemingly simple task.

cratermoon 3y ago
if the signed JSON string is { "widget" :{ "dataA": "aaa", "dataB": "€30" } } and your code produces {"widget":{"dataB":"\u20ac30","dataA":"aaa"}} The signatures will be different. The JSON spec doesn't require any ordering of fields or say much about how strings must be represented. This is all discussed in sec 3.2 of RFC8785 https://www.rfc-editor.org/rfc/rfc8785#name-generation-of-canonical-jso https://www.rfc-editor.org/rfc/rfc8785#name-generation-of-ca... You don't say how your work's custom message request signer works, but yes, you have to have the exact same json on both ends to match signatures.
- jzombie 3y ago
  Thanks for this feedback.
  - cratermoon 3y ago
    I ran across and had to fix something similar in a previous job. The organization's SAML assertions were being rejected for invalid signatures. What I found in the code that created the XML and signed it, someone had changed or removed a tag after the signature was created but before the XML was written, thus the signature didn't match what was in the actual assertion. Reverting that change fixed the immediate problem, and made login work again for most cases. It took a bit longer to address the underlying issue. On the whole were trying to support SAML using a library written at a fairly low level, one intended for people trying to implement the SAML protocol itself, not use SAML for authentication. This lead to ongoing issues that they never really solved.

2 points by jzombie 3y ago

cratermoon 3y ago

if the signed JSON string is { "widget" :{ "dataA": "aaa", "dataB": "€30" } } and your code produces {"widget":{"dataB":"\u20ac30","dataA":"aaa"}} The signatures will be different. The JSON spec doesn't require any ordering of fields or say much about how strings must be represented. This is all discussed in sec 3.2 of RFC8785 https://www.rfc-editor.org/rfc/rfc8785#name-generation-of-canonical-jso https://www.rfc-editor.org/rfc/rfc8785#name-generation-of-ca... You don't say how your work's custom message request signer works, but yes, you have to have the exact same json on both ends to match signatures.

jzombie 3y ago
Thanks for this feedback.
- cratermoon 3y ago
  I ran across and had to fix something similar in a previous job. The organization's SAML assertions were being rejected for invalid signatures. What I found in the code that created the XML and signed it, someone had changed or removed a tag after the signature was created but before the XML was written, thus the signature didn't match what was in the actual assertion. Reverting that change fixed the immediate problem, and made login work again for most cases. It took a bit longer to address the underlying issue. On the whole were trying to support SAML using a library written at a fairly low level, one intended for people trying to implement the SAML protocol itself, not use SAML for authentication. This lead to ongoing issues that they never really solved.