3 ms·

`externals` seem the (only) way to mix arbitrary code into the parsing, so... Can `externals` call back into the parser? That is, is the parser reentrant? Ca

`externals` seem the (only) way to mix arbitrary code into the parsing, so... Can `externals` call back into the parser? That is, is the parser reentrant? Can `externals` be a zero-length assertions? That is, is there a progress requirement for success? And thus serve as arbitrary-computation parse-rejecting "semantic actions". Do `externals` have access to GLR parse state?

maxbrunsfeld 8y ago
The Externals API is designed to only let you do things that are compatible with incremental parsing. It’s good for things like indentation based delimiters and automatic semicolon insertion. You cannot call back into the parser; it’s just for tokenization. You can produce zero length tokens, and we do use this feature a lot. You dont have access to the parse state directly, but the external scanner is passed an array of Boolean values that indicates which external tokens are expected in the current state.