3 ms·

Interestingly, I seem to have the opposite complaint about XML parsers. People generate crappy, broken XML all the time, but many of the services that digest it

Interestingly, I seem to have the opposite complaint about XML parsers. People generate crappy, broken XML all the time, but many of the services that digest it are horribly picky and unwilling to take even valid inputs. Some refuse empty fields and demand they not be included as tags. Some demand a specific list of tags, and insist all be included even if empty. Some clearly have non-recursive parsing, only honor certain depths of tree. Some have completely hard-coded parsers, and only accept certain tags in certain orders. The list goes on. Is this just Sturgeon's Law, where things are broken in both directions? Or is there some predictable reason that some tools become overly permissive, and others overly restrictive?

captainmuon 9y ago
I have the same with JSON. Sure, when I'm using JSON/REST APIs everything works fine, but I deal a lot with JSON on disk, and it happens a lot that you have sloppy JSON because people hand-edit it. Trailing commas and comments are the most common, but sometimes you just have js/python object literals dumped into a file. In every language I work in, I find myself writing a lenient JSON parser. So far, Python, C++, and Nim (for JS, there is already JSON5). I am also working on a whitespace and comment preserving parser that round-trips, so you can change a value in a configuration file without loosing comments. (Currently, I always produce standard JSON so compatibility is not an issue.) I have been thinking about publishing these parsers, but after seeing how much abuse and nasty comments the JSON5 guy got, I have been holding off...
- Zaak 9y ago
  I would love to see a space and comment preserving json parser. Being able to programmatically update a config file without losing formatting and such sounds like a big win.
- nly 9y ago
  > I am also working on a whitespace and comment preserving parser that round-trips, so you can change a value in a configuration file without loosing comments. I am 80% done writing such a parser for a over-engineered legacy config file format. It isn't that hard really, you just need a parser framework/library that can output a complete parse tree (a tree where a preorder traversal covers every byte in the input contiguously, exactly once). The remaining 20% of the work is keeping track of the line and column number (almost there), cleaning up the code, and being able to reserialize the AST to a binary format (for reloading in another process where I don't want to have to rerun the original PEG)
inimino 9y ago
I think it's just the "X" in XML. XML parsers are much more likely to be hand-rolled, and the standard tooling is heavyweight and mostly never caught on.

captainmuon 9y ago

I have the same with JSON. Sure, when I'm using JSON/REST APIs everything works fine, but I deal a lot with JSON on disk, and it happens a lot that you have sloppy JSON because people hand-edit it. Trailing commas and comments are the most common, but sometimes you just have js/python object literals dumped into a file. In every language I work in, I find myself writing a lenient JSON parser. So far, Python, C++, and Nim (for JS, there is already JSON5). I am also working on a whitespace and comment preserving parser that round-trips, so you can change a value in a configuration file without loosing comments. (Currently, I always produce standard JSON so compatibility is not an issue.) I have been thinking about publishing these parsers, but after seeing how much abuse and nasty comments the JSON5 guy got, I have been holding off...

Zaak 9y ago
I would love to see a space and comment preserving json parser. Being able to programmatically update a config file without losing formatting and such sounds like a big win.
nly 9y ago
> I am also working on a whitespace and comment preserving parser that round-trips, so you can change a value in a configuration file without loosing comments. I am 80% done writing such a parser for a over-engineered legacy config file format. It isn't that hard really, you just need a parser framework/library that can output a complete parse tree (a tree where a preorder traversal covers every byte in the input contiguously, exactly once). The remaining 20% of the work is keeping track of the line and column number (almost there), cleaning up the code, and being able to reserialize the AST to a binary format (for reloading in another process where I don't want to have to rerun the original PEG)

inimino 9y ago

I think it's just the "X" in XML. XML parsers are much more likely to be hand-rolled, and the standard tooling is heavyweight and mostly never caught on.