3 ms·
I skimmed your site and didn't really get it. Maybe that's what everyone else does. In case it helps, here are the thoughts that occurred to me, in the order th
by abainbridge 5y ago
I skimmed your site and didn't really get it. Maybe that's what everyone else does. In case it helps, here are the thoughts that occurred to me, in the order they occurred.
0. To me "data format" makes me think of things like PNG, or the DWARF debug symbols format, or the MPEG transport stream, not general purpose text formats like JSON and XML (I'm not a web person).
1. A secure data format. OK, weird. I thought it was always the programs/libraries that dealt with the data formats that were guilty of the security bugs, not the format itself.
2. All the bullet points under simple and efficient are already true for all the data formats I know/use/care about. So already I've dismissed your project as interesting - ie the "simple" and "efficient" are already solved and I don't believe "secure" is a real problem. But I'm aware I might be jumping to conclusions, so I keep reading.
3. I skipped straight to "Security - Protecting your data". I thought the sentence explaining why security matters was superfluous - your audience already knows why security matters.
4. "The existing ad-hoc data formats are too loosely defined to be secure, and can't be fixed because they're not versioned". OK, this looks like the meat. I click the link.
5. "There are many vectors that attackers could take advantage of when they control the data your system is receiving, the most common of which are induced data loss, field omission, key collisions, and exploitation of algorithmic complexity". If the data is from an attacker, data loss and field omission sound like good things - I don't want their data or fields because they are an attacker.
6. '"change user" command with a group of admin\U+D800'. I'm still confused. It still sounds like the "admin\U+D800" string is processed by my program. Your data format doesn't know whether that is a valid string or not. Telling your data format is no easier than telling my program. Oh! Maybe it is. Because you only need to specify it in the data format, not in every program/library that implements the format. Is this the point of the system?
- kstenerud 5y agoHmm ok I'm definitely not communicating properly what it is then... I'm not sure what nomenclature I could use besides "data format" to describe what this is, any more than one could describe JSON as anything other than a data format... The security aspect is where I'm having the most trouble communicating, because it's just not a well known issue. The problem comes from the different ways that codecs deal with the data they receive (for example Java Codec A vs Golang codec B vs Python codec C vs Python codec D, etc). If the spec isn't strict enough, a conformant codec could behave in ways that are insecure when coupled with other systems. Security really does start at the specification, in that the spec has to define what a codec can and cannot do. For example: "A codec MUST reject invalid characters. It MUST NOT truncate them or replace them with the Unicode replacement character". A rule like this would prevent the admin\U+D800 vulnerability because a codec is not allowed to behave this way (truncating or replacing). With JSON, this isn't specified, so systems where the same data is parsed by multiple subsystems are vulnerable. Specifically regarding the admin\U+D800 problem: Any sufficiently complex system will have multiple subsytems. So say you have a user-facing system that accepts a "new account" request in JSON. The attacker passes in "admin\U+D800" as the group (meaning the 5 character "admin" plus the unicode character D800, which is invalid). Your account creator's JSON decoder ignores bad characters, so when it checks the group vs its blacklist, it finds that "admin\U+D800" != "admin". Validation passed, it sends the new user info to the account storage subsystem (as a JSON message). Your account storage subsystem's JSON decoder silently strips bad characters, so your account subsystem only sees "admin", not "admin\U+D800". Now this new user is an admin, thanks to a privilege escalation vulnerability brought on by a lax JSON spec (all codecs involved were conformant). If the JSON spec had specified that "A codec MUST reject invalid characters. It MUST NOT truncate them or replace them with the Unicode replacement character", this kind of vulnerability could not happen with a conformant codec. Thanks for the feedback!
- abainbridge 5y ago> I'm not sure what nomenclature I could use besides "data format" Yeah, I don't know either. I see that the Wikipedia page for JSON describes it as a "data interchange format". Maybe that helps. I'd probably side-step the problem by having your home page start with "Like Protocol Buffers but more secure", followed immediately by the example you just gave me.
- kstenerud 5y agoEither way, the responses here give me a lot to go on! I guess I need to revamp what is emphasized and in what order. It's going to have to hammer on security all the way. Everything else is incidental and a "nice-to-have". First, the need for tight and precise specs, not allowing optional or under-defined behaviors or too many features, and what goes wrong if a data format doesn't do that (I'll see if I can simplify the admin example or maybe do a $0 cost purchase exploit example). Next, versioned documents and why that's important: Without it you're limited in how you can update the format to deal with emerging threats as they come along. Otherwise you get deprecations, loss of code space, and the possibility to get permanently stuck with an unfixable problem. Then talk about fundamental type support and why we need so many: If you don't do that, everyone has to make their own encodings for common types (like dates, media etc), which won't be compatible or as carefully thought out, opening up security holes again. The trick is how to make security sound sexy enough for people to take notice...