15 ms·

While this is true of JSON, it's also true of any other non-trivial serialization and/or encoding format. The main lessons to learn here are that: 1) implement

While this is true of JSON, it's also true of any other non-trivial serialization and/or encoding format. The main lessons to learn here are that: 1) implementation matters 2) "simple" specs never really are It's definitely important to have documents like this one that explore the edge cases and the differences between implementations, but you can replace "JSON" in the introductory paragraph with any other serialization format, encoding standard, or IPC protocol and it would remain true: "<format> is not the easy, idealised format as many do believe. [There are not] two libraries that exhibit the very same behaviour. Moreover, [...] edge cases and maliciously crafted payloads can cause bugs, crashes and denial of services, mainly because <format> libraries rely on specifications that have evolved over time and that left many details loosely specified or not specified at all."

izacus 9y ago
How do Protocol Buffers (which I see used quite alot in similar environments as JSON) compare? Anyone has experience in the format?
- toast0 9y ago
  My experience with json and similar formats is that most of the complexity arrises from using delimited strings instead of length prefixed strings, and the exciting escaping that results. If the strings are character strings instead of byte strings, you get to add an extra layer of character encoding excitement. PHP serialization is better here, everything is type:value or type:length:value, although strings do have quotes around them, because their byte length is known, internal quotes need not be escaped. You can still have issues with genrating and parsing the human readible numbers properly (floating point is always fun, and integers may have some bit size limit I don't recall), but you don't need to worry about quoting Unicode values properly. Protocol buffers have clear length indications, so that's easier, but it's not a 'self documenting' format, you need to have the description file to parse an encoded value. The end result is usually many fewer bits though.
  - ChrisSD 9y ago
    The problem is that formats like JSON are designed to be human readable and writable. Length prefixing is a non starter here. Protobuf and similar are binary formats so don't have this limitation.
    - zeveb 9y ago
      > The problem is that formats like JSON are designed to be human readable and writable. Length prefixing is a non starter here. Canonical S-expression are both human-readable & length-prefixed. They do this by have an advanced representation which is human-friendly: (data (looks "like this" |YWluJ3QgaXQgY29vbD8=|)) And a canonical representation which is length-prefixed: (4:data(5:looks9:like this14:ain't it cool?))
  - johannes1234321 9y ago
    The PHP serialisation format has many issues, especially since it allows all sorts of PHP data structures to be encoded. This allows defining references and serializing objects using custom routines into arbitrary binary blobs. Also PHP's unserialization can be used to trigger the autoloader as it tries to resolve unloaded classes, which can trigger unsafe routines in those. Certainly no data format for data exchange between systems, especially untrusted sources.
    - toast0 9y ago
      You shouldn't use PHP's unserialize implementation with untrusted sources; but my point was that its format makes it relatively simple to parse vs json or xml where you have to do a lot of work to parse strings. If you're writing your own parser (including a parser for another language), you could decide to only parse basic types (bool, int, float, array); if you're designing your own format, you could take the lesson of length prefixed strings are much easier to use for computers than delimited strings.
- pjscott 9y ago
  I wrote a protobuf decoder once and found it to be remarkably pleasant. Getting the decoder working only took a few hours. The format was obviously designed to be straightforward -- no escaping, no backtracking, no ambiguity. I believe the grammar is LL(0), which is a nice touch. And because it's not meant to be human-readable, there's no incentive for people to make their parsers deviate from the strict grammar; e.g. there's no protobuf quirk analogous to JSON's parser-dependent handling of trailing commas, because why would anyone bother?
  - iainmerrick 9y ago
    There is no such quirk. Trailing commas are simply not standard JSON (sadly). Even if they were, it would still be easy to parse!
    - pjscott 9y ago
      You're right in the same technically-correct sense that parsing HTML 4 is easy: it's just SGML! And as long as you don't have to support any of the crazy deviations from the spec that some people have come to rely on, that's fine. There's a whole spectrum of unofficial parser "helpfulness" here, with HTML 4 being an extreme case of parsers filled with hacks to deal with existing broken data, protobufs being an extreme case of parsers doing the One and Only True Thing, and JSON mostly toward the same end of the spectrum as protobufs, but a bit less so.
      - iainmerrick 9y ago
        Yeah, that spectrum is a good way to think about it. JSON hits a sweet spot of being very easy for computers to deal with almost all the time, while also being reasonably easy for humans to read and write. I was going to add “if you started with that as the spec, it wouldn’t be hard to design something better than JSON” but real examples like YAML are pretty awkward, so probably it’s a harder problem than it seems.
        Mikhail_Edoshin 9y ago
- ori_b 9y ago
  Protocol buffers sidestep the issue of independent implementations behaving differently by simply not having widely used independent implementations. That said, I've still been bitten by the Python implementation on the Mac acting differently from the C++ implementation on Linux, although I can't remember exactly what the issue was right now.
  - agrafix 9y ago
    One of the things that has bitten me before was the maximum allowed nesting depth - this was different in two implementations, so one rejected the payload while the other one parsed it fine :(
SamReidHughes 9y ago
No, this is not true of many reasonable formats. You don't have to make an obtusely nontrivial format to encode the data JSON does.
- IshKebab 9y ago
  Most of the things they test are true of any text-based format, and many of them are true of any serialisation format. E.g. 100000 opening brackets. You could do the same in XML for example and I expect many parsers would fail.
  - jandrese 9y ago
    Maybe the difference is that nobody ever thought that XML was easy to parse.
  - SamReidHughes 9y ago
    XML is far worse than JSON, it is known. It's a lot easier to screw up a text based format than a binary format. But it would also be possible to make a better text based format than JSON, mainly by defining down what numerical values are allowed to be represented and exactly how they get parsed, and making it harder to screw up strings. That's where most of the problems are.
    - jandrese 9y ago
      I think the reason JSON doesn't take well to strong typing is that it is designed for Javascript. When you pass some Javascript object to the JSON encoder how is it supposed to decide which numeric representation to use? Does it always go for the smallest one that works? The Javascript decoder at the other end is just going to throw away that type information anyway, so all it is good for is verifying the data while you are parsing it. Maybe not a totally useless thing to do but it's a lot of work to get that modest benefit.
      - SamReidHughes 9y ago
        Making an encoding tailored to JavaScript documents that is a binary format is easy. And I don't think strong typing has anything to do with it. Making it handle Python and Ruby objects at the same time is harder, because they have different opinions about what a string is, what a number can be.
arghwhat 9y ago
It really isn't true for JSON either. If you read the rant, most of it is simply about JSON parsers accepting additional, non-JSON syntaxes. Looking at the matrix, all green, yellow, light blue and dark blue are OK outcomes. Red are crashes (stack overflow with 10000 nested arrays, for example), and dark brown are valid JSON that didn't parse (things like UTF-8 mishandling). The issues aren't really JSON-specific.
- dagenix 9y ago
  So if JSON parsers can't agree on what JSON actually is, that isn't a problem?
  - benzoate 9y ago
    I consider it more like behaviour outside of the spec is undefined, and some parsers have bugs.
    - dagenix 9y ago
      A good spec should define all those corner cases. With Json, it's possible for two bug free libraries to take the same input and produce a different output.
      - arghwhat 9y ago
        Only if the input is invalid JSON.
  - arghwhat 9y ago
    JSON parsers all agree what JSON actually is (except for the brown colored fields in the compat matrix, which are actual JSON compat bugs). JSON parsers may also permit various additional variations which are very explicitly not JSON. This means that they may accept a handcrafted, technically invalid JSON document. However, a JSON encoder may never generate a document containing such "extensions", as this would not be JSON. This concept of having parsers accept more than necessary follows best practices of robustness. As the robustness principle goes: "Be conservative in what you do, be liberal in what you accept from others".
ccvannorman 9y ago
Indeed, this is the principal point behind Godel Escher Bach, Hofstadter's work which explores (by various proofs, theories and parables) how no absolutes exist, particularly with respect to structuring/interpreting data. "It doesn't make sense to formalize a system absolutely"
ospider 9y ago
Paring "csv" and "obj" format is also difficult. They both have "simple" specs, but neither of them has a standard spec.
- jstimpfle 9y ago
  They both have after-the fact specifications that (try to) codify various flavours that had already been released into the wild. (I assume you mean Wavefront Obj). It's a pity that they aren't better specified. The same holds for ini. Maybe that's just the curse of good engineering: If it's simple, it's easy to make a mess out of it.

izacus 9y ago

How do Protocol Buffers (which I see used quite alot in similar environments as JSON) compare? Anyone has experience in the format?

toast0 9y ago
My experience with json and similar formats is that most of the complexity arrises from using delimited strings instead of length prefixed strings, and the exciting escaping that results. If the strings are character strings instead of byte strings, you get to add an extra layer of character encoding excitement. PHP serialization is better here, everything is type:value or type:length:value, although strings do have quotes around them, because their byte length is known, internal quotes need not be escaped. You can still have issues with genrating and parsing the human readible numbers properly (floating point is always fun, and integers may have some bit size limit I don't recall), but you don't need to worry about quoting Unicode values properly. Protocol buffers have clear length indications, so that's easier, but it's not a 'self documenting' format, you need to have the description file to parse an encoded value. The end result is usually many fewer bits though.
- ChrisSD 9y ago
  The problem is that formats like JSON are designed to be human readable and writable. Length prefixing is a non starter here. Protobuf and similar are binary formats so don't have this limitation.
  - zeveb 9y ago
    > The problem is that formats like JSON are designed to be human readable and writable. Length prefixing is a non starter here. Canonical S-expression are both human-readable & length-prefixed. They do this by have an advanced representation which is human-friendly: (data (looks "like this" |YWluJ3QgaXQgY29vbD8=|)) And a canonical representation which is length-prefixed: (4:data(5:looks9:like this14:ain't it cool?))
- johannes1234321 9y ago
  The PHP serialisation format has many issues, especially since it allows all sorts of PHP data structures to be encoded. This allows defining references and serializing objects using custom routines into arbitrary binary blobs. Also PHP's unserialization can be used to trigger the autoloader as it tries to resolve unloaded classes, which can trigger unsafe routines in those. Certainly no data format for data exchange between systems, especially untrusted sources.
  - toast0 9y ago
    You shouldn't use PHP's unserialize implementation with untrusted sources; but my point was that its format makes it relatively simple to parse vs json or xml where you have to do a lot of work to parse strings. If you're writing your own parser (including a parser for another language), you could decide to only parse basic types (bool, int, float, array); if you're designing your own format, you could take the lesson of length prefixed strings are much easier to use for computers than delimited strings.
pjscott 9y ago
I wrote a protobuf decoder once and found it to be remarkably pleasant. Getting the decoder working only took a few hours. The format was obviously designed to be straightforward -- no escaping, no backtracking, no ambiguity. I believe the grammar is LL(0), which is a nice touch. And because it's not meant to be human-readable, there's no incentive for people to make their parsers deviate from the strict grammar; e.g. there's no protobuf quirk analogous to JSON's parser-dependent handling of trailing commas, because why would anyone bother?
- iainmerrick 9y ago
  There is no such quirk. Trailing commas are simply not standard JSON (sadly). Even if they were, it would still be easy to parse!
  - pjscott 9y ago
    You're right in the same technically-correct sense that parsing HTML 4 is easy: it's just SGML! And as long as you don't have to support any of the crazy deviations from the spec that some people have come to rely on, that's fine. There's a whole spectrum of unofficial parser "helpfulness" here, with HTML 4 being an extreme case of parsers filled with hacks to deal with existing broken data, protobufs being an extreme case of parsers doing the One and Only True Thing, and JSON mostly toward the same end of the spectrum as protobufs, but a bit less so.
    - iainmerrick 9y ago
      Yeah, that spectrum is a good way to think about it. JSON hits a sweet spot of being very easy for computers to deal with almost all the time, while also being reasonably easy for humans to read and write. I was going to add “if you started with that as the spec, it wouldn’t be hard to design something better than JSON” but real examples like YAML are pretty awkward, so probably it’s a harder problem than it seems.
      - Mikhail_Edoshin 9y ago
ori_b 9y ago
Protocol buffers sidestep the issue of independent implementations behaving differently by simply not having widely used independent implementations. That said, I've still been bitten by the Python implementation on the Mac acting differently from the C++ implementation on Linux, although I can't remember exactly what the issue was right now.
- agrafix 9y ago
  One of the things that has bitten me before was the maximum allowed nesting depth - this was different in two implementations, so one rejected the payload while the other one parsed it fine :(

SamReidHughes 9y ago

No, this is not true of many reasonable formats. You don't have to make an obtusely nontrivial format to encode the data JSON does.

IshKebab 9y ago
Most of the things they test are true of any text-based format, and many of them are true of any serialisation format. E.g. 100000 opening brackets. You could do the same in XML for example and I expect many parsers would fail.
- jandrese 9y ago
  Maybe the difference is that nobody ever thought that XML was easy to parse.
- SamReidHughes 9y ago
  XML is far worse than JSON, it is known. It's a lot easier to screw up a text based format than a binary format. But it would also be possible to make a better text based format than JSON, mainly by defining down what numerical values are allowed to be represented and exactly how they get parsed, and making it harder to screw up strings. That's where most of the problems are.
  - jandrese 9y ago
    I think the reason JSON doesn't take well to strong typing is that it is designed for Javascript. When you pass some Javascript object to the JSON encoder how is it supposed to decide which numeric representation to use? Does it always go for the smallest one that works? The Javascript decoder at the other end is just going to throw away that type information anyway, so all it is good for is verifying the data while you are parsing it. Maybe not a totally useless thing to do but it's a lot of work to get that modest benefit.
    - SamReidHughes 9y ago
      Making an encoding tailored to JavaScript documents that is a binary format is easy. And I don't think strong typing has anything to do with it. Making it handle Python and Ruby objects at the same time is harder, because they have different opinions about what a string is, what a number can be.

arghwhat 9y ago

It really isn't true for JSON either. If you read the rant, most of it is simply about JSON parsers accepting additional, non-JSON syntaxes. Looking at the matrix, all green, yellow, light blue and dark blue are OK outcomes. Red are crashes (stack overflow with 10000 nested arrays, for example), and dark brown are valid JSON that didn't parse (things like UTF-8 mishandling). The issues aren't really JSON-specific.

dagenix 9y ago
So if JSON parsers can't agree on what JSON actually is, that isn't a problem?
- benzoate 9y ago
  I consider it more like behaviour outside of the spec is undefined, and some parsers have bugs.
  - dagenix 9y ago
    A good spec should define all those corner cases. With Json, it's possible for two bug free libraries to take the same input and produce a different output.
    - arghwhat 9y ago
      Only if the input is invalid JSON.
- arghwhat 9y ago
  JSON parsers all agree what JSON actually is (except for the brown colored fields in the compat matrix, which are actual JSON compat bugs). JSON parsers may also permit various additional variations which are very explicitly not JSON. This means that they may accept a handcrafted, technically invalid JSON document. However, a JSON encoder may never generate a document containing such "extensions", as this would not be JSON. This concept of having parsers accept more than necessary follows best practices of robustness. As the robustness principle goes: "Be conservative in what you do, be liberal in what you accept from others".

ccvannorman 9y ago

Indeed, this is the principal point behind Godel Escher Bach, Hofstadter's work which explores (by various proofs, theories and parables) how no absolutes exist, particularly with respect to structuring/interpreting data. "It doesn't make sense to formalize a system absolutely"

ospider 9y ago

Paring "csv" and "obj" format is also difficult. They both have "simple" specs, but neither of them has a standard spec.

jstimpfle 9y ago
They both have after-the fact specifications that (try to) codify various flavours that had already been released into the wild. (I assume you mean Wavefront Obj). It's a pity that they aren't better specified. The same holds for ini. Maybe that's just the curse of good engineering: If it's simple, it's easy to make a mess out of it.