7 ms·

Hmm ok I'm definitely not communicating properly what it is then... I'm not sure what nomenclature I could use besides "data format" to describe what this is,

Hmm ok I'm definitely not communicating properly what it is then... I'm not sure what nomenclature I could use besides "data format" to describe what this is, any more than one could describe JSON as anything other than a data format... The security aspect is where I'm having the most trouble communicating, because it's just not a well known issue. The problem comes from the different ways that codecs deal with the data they receive (for example Java Codec A vs Golang codec B vs Python codec C vs Python codec D, etc). If the spec isn't strict enough, a conformant codec could behave in ways that are insecure when coupled with other systems. Security really does start at the specification, in that the spec has to define what a codec can and cannot do. For example: "A codec MUST reject invalid characters. It MUST NOT truncate them or replace them with the Unicode replacement character". A rule like this would prevent the admin\U+D800 vulnerability because a codec is not allowed to behave this way (truncating or replacing). With JSON, this isn't specified, so systems where the same data is parsed by multiple subsystems are vulnerable. Specifically regarding the admin\U+D800 problem: Any sufficiently complex system will have multiple subsytems. So say you have a user-facing system that accepts a "new account" request in JSON. The attacker passes in "admin\U+D800" as the group (meaning the 5 character "admin" plus the unicode character D800, which is invalid). Your account creator's JSON decoder ignores bad characters, so when it checks the group vs its blacklist, it finds that "admin\U+D800" != "admin". Validation passed, it sends the new user info to the account storage subsystem (as a JSON message). Your account storage subsystem's JSON decoder silently strips bad characters, so your account subsystem only sees "admin", not "admin\U+D800". Now this new user is an admin, thanks to a privilege escalation vulnerability brought on by a lax JSON spec (all codecs involved were conformant). If the JSON spec had specified that "A codec MUST reject invalid characters. It MUST NOT truncate them or replace them with the Unicode replacement character", this kind of vulnerability could not happen with a conformant codec. Thanks for the feedback!

abainbridge 5y ago
> I'm not sure what nomenclature I could use besides "data format" Yeah, I don't know either. I see that the Wikipedia page for JSON describes it as a "data interchange format". Maybe that helps. I'd probably side-step the problem by having your home page start with "Like Protocol Buffers but more secure", followed immediately by the example you just gave me.
- kstenerud 5y ago
  Either way, the responses here give me a lot to go on! I guess I need to revamp what is emphasized and in what order. It's going to have to hammer on security all the way. Everything else is incidental and a "nice-to-have". First, the need for tight and precise specs, not allowing optional or under-defined behaviors or too many features, and what goes wrong if a data format doesn't do that (I'll see if I can simplify the admin example or maybe do a $0 cost purchase exploit example). Next, versioned documents and why that's important: Without it you're limited in how you can update the format to deal with emerging threats as they come along. Otherwise you get deprecations, loss of code space, and the possibility to get permanently stuck with an unfixable problem. Then talk about fundamental type support and why we need so many: If you don't do that, everyone has to make their own encodings for common types (like dates, media etc), which won't be compatible or as carefully thought out, opening up security holes again. The trick is how to make security sound sexy enough for people to take notice...

abainbridge 5y ago

> I'm not sure what nomenclature I could use besides "data format" Yeah, I don't know either. I see that the Wikipedia page for JSON describes it as a "data interchange format". Maybe that helps. I'd probably side-step the problem by having your home page start with "Like Protocol Buffers but more secure", followed immediately by the example you just gave me.

kstenerud 5y ago
Either way, the responses here give me a lot to go on! I guess I need to revamp what is emphasized and in what order. It's going to have to hammer on security all the way. Everything else is incidental and a "nice-to-have". First, the need for tight and precise specs, not allowing optional or under-defined behaviors or too many features, and what goes wrong if a data format doesn't do that (I'll see if I can simplify the admin example or maybe do a $0 cost purchase exploit example). Next, versioned documents and why that's important: Without it you're limited in how you can update the format to deal with emerging threats as they come along. Otherwise you get deprecations, loss of code space, and the possibility to get permanently stuck with an unfixable problem. Then talk about fundamental type support and why we need so many: If you don't do that, everyone has to make their own encodings for common types (like dates, media etc), which won't be compatible or as carefully thought out, opening up security holes again. The trick is how to make security sound sexy enough for people to take notice...