4 ms·

> Message signatures are signatures of stringified JSON. Given that JSON is not particularly well defined to guarantee representation stability are implementati

> Message signatures are signatures of stringified JSON. Given that JSON is not particularly well defined to guarantee representation stability are implementation differences handled? Systems I've seen (and written) that do this use deterministic serialization algorithms that sort keys and do other things standard, general-purpose implementations don't. The implementations in core libraries, browsers, and the like tend to be faster, but the payloads being signed in the apps usually aren't that large.

leesalminen 4y ago
Taking a look at NIP-01, the serialized data that gets hashed and signed is a flat array, not an object. So no sorting of keys here. I don’t think any json serializer would change the order of items in an array, right? https://github.com/nostr-protocol/nips/blob/master/01.md https://github.com/nostr-protocol/nips/blob/master/01.md
- pointlessone 4y ago
  That's true but there's still plenty of room for incompatibility. JSON doesn't put any requirements on floats precision, for instance. Another example is that any character may be escaped (that's in the RFC, literally, verbatim) so while it's recommended to use UTF-8 for serialization a random lib can decide that ASCII is the way and escape all your emojis. To be fair, NIP-01 specifies that it has to be UTF-8 but no mention of float precision can be found. I'm almost certain these two are not the only ways a JSON can be divergent while still being valid. And I have to point out that we're doomed to repeat our mistakes. XML is also very flexible. And people wanted to sign XML documents and they found this to be a problem. So they came up with Canonical XML form — a way to remove some of that flexibility to make sure it's possible to reliably derive a stable variant that can be signed and verified. Unfortunately, we haven't came up with Canonical JSON yet. But maybe we will soon.
  - toomim 4y ago
    Canonical JSON has a IETF RFC: https://datatracker.ietf.org/doc/rfc8785/ https://datatracker.ietf.org/doc/rfc8785/

leesalminen 4y ago

Taking a look at NIP-01, the serialized data that gets hashed and signed is a flat array, not an object. So no sorting of keys here. I don’t think any json serializer would change the order of items in an array, right? https://github.com/nostr-protocol/nips/blob/master/01.md https://github.com/nostr-protocol/nips/blob/master/01.md

pointlessone 4y ago
That's true but there's still plenty of room for incompatibility. JSON doesn't put any requirements on floats precision, for instance. Another example is that any character may be escaped (that's in the RFC, literally, verbatim) so while it's recommended to use UTF-8 for serialization a random lib can decide that ASCII is the way and escape all your emojis. To be fair, NIP-01 specifies that it has to be UTF-8 but no mention of float precision can be found. I'm almost certain these two are not the only ways a JSON can be divergent while still being valid. And I have to point out that we're doomed to repeat our mistakes. XML is also very flexible. And people wanted to sign XML documents and they found this to be a problem. So they came up with Canonical XML form — a way to remove some of that flexibility to make sure it's possible to reliably derive a stable variant that can be signed and verified. Unfortunately, we haven't came up with Canonical JSON yet. But maybe we will soon.
- toomim 4y ago
  Canonical JSON has a IETF RFC: https://datatracker.ietf.org/doc/rfc8785/ https://datatracker.ietf.org/doc/rfc8785/