3 ms·

In my case the parse tree had nodes that corresponded with grammar rules (mapped to XML elements) and nodes that corresponded to the preserved but otherwise uni

by nly 5y ago

In my case the parse tree had nodes that corresponded with grammar rules (mapped to XML elements) and nodes that corresponded to the preserved but otherwise uninteresting fluff that surrounded them (mapped to XML PCDATA). We wanted a parse tree that could be serialized out to produce the original document byte for byte, but still reflected the grammar. Things get uglier for JSON in the real world case: "5+7 #comment" becomes: <sum><integer>5</integer><operation>+</operation><integer>7</integer> #comment</sum> The XML output is essentially the original input + grammar markup. Your solution also works but is a data structure representation with a lot more line noise for humans to read.

dnautics 5y ago
It just looks bad because I made it one line; by comparison if I hadn't seen the pre-parsed form 5+7 I would not be able to immediately grok what is going on in your parsed one-line XML, either; there's a lot of "line noise" there too. Moreover it feels like the XML you have has lost the structure of the meaning, by having the tokens 5, +, and 7 be linearized. You probably want a tree with + at the root and 5 and 7 as leaves. That XML encourages these sorts of "oddly flat" representations is one of its weaknesses.
- nly 5y ago
  + at the root would indicate an abstract syntax tree rather than a concrete or "full" syntax tree (terminology varies slightly) This was primarily a mechanism to load parsed documents in to code that didn't have access to the recursive descent parser that parsed the document, for either security or performance reasons. Loading the XML with the appropriate parse flags, doing a pre-order traversal on all the leaf PCDATA, and then printing it out, would regenerate the original document byte for byte. Perhaps I should go back and implement JSON output and see how I feel about it... thought I'm pretty sure once you introduce a bunch of "text" and "type" fields to the JSON tree it's going to look a lot less readable than XML markup. There are other fields as well like line and column numbers (output as XML attributes)

by nly 5y ago

dnautics 5y ago

It just looks bad because I made it one line; by comparison if I hadn't seen the pre-parsed form 5+7 I would not be able to immediately grok what is going on in your parsed one-line XML, either; there's a lot of "line noise" there too. Moreover it feels like the XML you have has lost the structure of the meaning, by having the tokens 5, +, and 7 be linearized. You probably want a tree with + at the root and 5 and 7 as leaves. That XML encourages these sorts of "oddly flat" representations is one of its weaknesses.

nly 5y ago
+ at the root would indicate an abstract syntax tree rather than a concrete or "full" syntax tree (terminology varies slightly) This was primarily a mechanism to load parsed documents in to code that didn't have access to the recursive descent parser that parsed the document, for either security or performance reasons. Loading the XML with the appropriate parse flags, doing a pre-order traversal on all the leaf PCDATA, and then printing it out, would regenerate the original document byte for byte. Perhaps I should go back and implement JSON output and see how I feel about it... thought I'm pretty sure once you introduce a bunch of "text" and "type" fields to the JSON tree it's going to look a lot less readable than XML markup. There are other fields as well like line and column numbers (output as XML attributes)