3 ms·

This seems more flexible to me somehow. Using the approach in TFA, can the laser handle tokens that are prefixes of other tokens? Or even tokens that share pre

This seems more flexible to me somehow. Using the approach in TFA, can the laser handle tokens that are prefixes of other tokens? Or even tokens that share prefixes? Lexing "truefalse" as two adjacent tokens "true" and "false" seems slightly crazier than just lexing it as one (meaningless) token.

ufo 7y ago
Usually, if there is a token that is a prefix of another the longer token wins. For example, in Javascript --x is parsed as the decrement operator instead of as two unary "-" operators.
- esfandia 7y ago
  With that rule, the "leading zero" problem in the article should have been caught properly as well. Why wasn't it? edit: I guess because "01" isn't a valid token.
  - y4mi 7y ago
    "--" is also invalid in json, as it doesn't deal with mutations. The parent was talking about js.