3 ms·
The paper author understates the importance of composability. > While having a tool that supports combining grammars would be handy, I don’t see it as a must-h
by austincheney 8y ago
The paper author understates the importance of composability.
> While having a tool that supports combining grammars would be handy, I don’t see it as a must-have. In practice, languages are not combined very often.
That was once true, but not any more. Intermixing of grammars within a single code instance is becoming more normal, and in many cases it is now the norm. The inability to accept this new norm leaves you holding a bag of multiple parsers and a large amount of processing overhead with a loss of flexibility on how to manage the entirety of the code. It is a horrid mess to deal with a collection of unrelated and nested ASTs to represent a single code instance.
> If the languages being combined are radically different, then for the sake of the programmer, there will probably need to be unambiguous delimiters at the transitions between the languages.
A faulty assumption that often does not reflect reality.
By focusing on composability first you remove many design barriers that limit or prevent design decisions down the line.
---
The fault of many parsers, from a design perspective, are a heavy concern for goals outside of parsing, frequently either a compile step or the shape of the AST. A compile operation is completely independent of a parse operation. Once an AST is created a consuming compiler will execute on it. Keep these concerns separate. Don't force your parser to act as a proxy for the limitations of your compiler.
A flexible and well reasoned parser can return an AST is a variety of shapes/formats. It doesn't really matter so long as the data structure is well reasoned and in a way the consuming application understands.
Also performance is not well understood in many cases of parsing, which is probably why parsers are often intermingled with the concerns of compilers. A parse operation takes time. It also takes different unrelated time for a consuming application to consume the AST. The speed with which a parse tree is generated is unrelated to the speed with which it is read. This is why composability is more important the paper gives credit for.
If anybody is interested I am working on addressing some of these concerns at https://sparser.io/ https://sparser.io/ which is a personal pet project. It doesn't cure cancer or solve world hunger yet (I am just one guy with a job), but the concept does solve some design problems and composability issues associated with parsing.
- tlb 8y agoI'm working on a language that embeds Javascript. Javascript syntax makes it quite easy to find the end of a complete block by considering just parens, brackets, braces, strings, and comments. Then I can just pass the string to a JS interpreter. So it's easy to allow javascript { ...any valid javascript code... } C++ syntax, on the other hand, makes it very hard to delimit a block. For instance, it's hard to guess whether a > is an operator or the end of a template parameter list. And preprocessor macros mean that the entire context of all the includes files before the current point can change what any token means. I think a good goal for a syntax is to make it easy to delimit blocks without a complete parser. It also makes syntax highlighting practical.
- austincheney 8y agoJavaScript related languages introduce the same problem. Is the `<` a comparison operator, the start of a type generics segment from something like TypeScript, a markup tag from something like React, or something else? JavaScript alone does not provide the context for that clarity. Boost templates in C++ are similar sort of problem, but there is less variability around boost templates, so they would be easier to discern from the surrounding syntax than the several JavaScript variances. > I think a good goal for a syntax is to... I manage a parser supporting many languages, not a language. I don't have the freedom to dictate what a language should look like. Instead my application must have the flexibility to tolerate other peoples' language design decisions, what ever they may be.
- a1369209993 8y agoI've discovered that python-style block syntax makes it trivial to embed arbitrary other languages, eg: while lang == nothing: lang = something() interpret(lang,:: ... whatever )) }] --] </script> __END__ {{({[(( /* ... ) # works fine
- jcranmer 8y agoThere's an easy way to handle composability: pick a quoting syntax that doesn't appear in the target language (or is unlikely to appear in it) and embed the inner language in that. Stuff like {{/}} (common in templating frameworks) or LLVM TableGen's [{/}] are decent examples.
- austincheney 8y agoI maintain a parser. I am not designing a language.