4 ms·

It's tempting to think this way, but it's a performance/complexity nightmare. If everything is a plugin, then you need ways for different parsers to talk to eac

It's tempting to think this way, but it's a performance/complexity nightmare. If everything is a plugin, then you need ways for different parsers to talk to each other to deal with nested syntax, plus ways for different syntactic transformers to correctly handle nesting of certain things. For example, if you desired to implement a plugin for "thin arrows", and a plugin for JSX, how would you parse this? const x = <foo x = {y -> <y/> } /> You'd have to have the JSX parser know to invoke the thin arrow parser, then the thin arrow parser has to know how to invoke the JSX parser, then when you emit, you have to do the whole dance over again. If you're willing to take some trade-offs in the parsing phase, it's OK. But the emit is where you get killed - either every plugin has to be written generically and correctly enough to deal with any arbitrary nesting of syntaxes (including the cases where the semantics of nesting aren't even clear, like where a 'fat' arrow here would capture 'this' from), or you do one tree pass per transformer, which gets very expensive. There's a reason you don't usually see architectures like that live very long.

patates 10y ago
Well, this is not the first time that a comment on HN not only made me change my opinion but also taught me something. Thank you for the detailed answer. Maybe the specific example you give can be solved in a very hacky way with something like a priority system but I can see how messy it can get even when tokenizing the possible expressions for such an open "playground", let alone transforming in the correct sequence. Thank you, Ryan. edit: Oh you are the person who implemented JSX support for Typescript! How do we buy you a beer, or in this case a six-pack?
pluma 10y ago
Aren't the Babel people trying to define a common standard format for the AST so different parsers and code generators can better work together? That said, if you look at TS as a superset of Babel plugins, there's actually not much to TS that is TS-specific. Adding that to Babel would be relatively simple. You seem to be arguing it's a slippery slope. We're not talking about arbitrary syntax. We're talking about a fairly well-established syntax extension backed by Google and Microsoft. It doesn't have to perfectly universal and generic to support TS. EDIT: If you were specifically addressing the parent: yes, if every syntax extension was created as a parser plugin, getting them to work together would be messy, but that's basically what we have in PostCSS isn't it? Plugins explicitly instructing users "this conflicts with X, please add it after Y but before Z" -- which is fine IMO as you only need to care about the most popular ones and everybody else knows their stack is experimental.
- phpnode 10y ago
  > Aren't the Babel people trying to define a common standard format for the AST so different parsers and code generators can better work together? Actually kind of the opposite - babel used to use the common ESTree AST format, but they've moved away to something similar but incompatible.
yortus 10y ago
> it's a performance/complexity nightmare Babel itself demonstrates the truth of this. If you look at the source code for any of Babel's syntax plugins, they are literally only a few lines long, because all they do is activate a feature that is actually part of the parser core.
jerf 10y ago
(This is not disagreement with RyanCavanaugh; it's a question for expansion by the community.) It seems like Perl 6 is trying to implement pluggable grammars that work like that. Can anyone here familiar with the Perl 6 ecosystem comment on how it's going over there? Can you feasibly write a module with 5 or 6 different grammar extensions? What's it like? My understanding of the situation has matched what RyanCavanaugh expressed here for a while, and I'm curious if people's experiences match it or not.
bpicolo 10y ago
PostCSS has this very problem, which is why I don't use PostCSS :)

patates 10y ago

Well, this is not the first time that a comment on HN not only made me change my opinion but also taught me something. Thank you for the detailed answer. Maybe the specific example you give can be solved in a very hacky way with something like a priority system but I can see how messy it can get even when tokenizing the possible expressions for such an open "playground", let alone transforming in the correct sequence. Thank you, Ryan. edit: Oh you are the person who implemented JSX support for Typescript! How do we buy you a beer, or in this case a six-pack?

pluma 10y ago

Aren't the Babel people trying to define a common standard format for the AST so different parsers and code generators can better work together? That said, if you look at TS as a superset of Babel plugins, there's actually not much to TS that is TS-specific. Adding that to Babel would be relatively simple. You seem to be arguing it's a slippery slope. We're not talking about arbitrary syntax. We're talking about a fairly well-established syntax extension backed by Google and Microsoft. It doesn't have to perfectly universal and generic to support TS. EDIT: If you were specifically addressing the parent: yes, if every syntax extension was created as a parser plugin, getting them to work together would be messy, but that's basically what we have in PostCSS isn't it? Plugins explicitly instructing users "this conflicts with X, please add it after Y but before Z" -- which is fine IMO as you only need to care about the most popular ones and everybody else knows their stack is experimental.

phpnode 10y ago
> Aren't the Babel people trying to define a common standard format for the AST so different parsers and code generators can better work together? Actually kind of the opposite - babel used to use the common ESTree AST format, but they've moved away to something similar but incompatible.

yortus 10y ago

> it's a performance/complexity nightmare Babel itself demonstrates the truth of this. If you look at the source code for any of Babel's syntax plugins, they are literally only a few lines long, because all they do is activate a feature that is actually part of the parser core.

jerf 10y ago

(This is not disagreement with RyanCavanaugh; it's a question for expansion by the community.) It seems like Perl 6 is trying to implement pluggable grammars that work like that. Can anyone here familiar with the Perl 6 ecosystem comment on how it's going over there? Can you feasibly write a module with 5 or 6 different grammar extensions? What's it like? My understanding of the situation has matched what RyanCavanaugh expressed here for a while, and I'm curious if people's experiences match it or not.

bpicolo 10y ago

PostCSS has this very problem, which is why I don't use PostCSS :)