3 ms·

Oh fun, I wrote a similar library in 2015 for Haskell. There is an annoying gotcha to deal with: there are sequences of valid characters that can be parsed inco

Oh fun, I wrote a similar library in 2015 for Haskell. There is an annoying gotcha to deal with: there are sequences of valid characters that can be parsed incorrectly if you’re doing incremental chunks, namely if “0.0” is split across two input chunks you can get a token stream with two valid float literals rather than 1! Namely “0” and “.0”, which is just a really annoying wart of json float syntax.

rictic 1y ago
Yeah, getting numbers correct was one of the trickier wrinkles in the project. https://github.com/rictic/jsonriver/blob/5515be978bb564e9bdc6b13323b23090a899d0de/src/tokenize.ts#L152-L190 https://github.com/rictic/jsonriver/blob/5515be978bb564e9bdc...
tracnar 1y ago
Don't you need to wait for some kind of delimiter (like ",", "]", "}", newline, EOF) before parsing something else than a string?
- rictic 1y ago
  Only for numbers! Strings, objects, arrays, true, false, and null all have an unambiguous ending.
  - stefs 1y ago
    but you don't do this for strings either, as shown in the examples - partial strings are pushed even though they're not yet ended: {"name": "Ale"}
    - rictic 1y ago
      Oh this isn't about the public API, it's about the internal logic of the parser.
yonatan8070 1y ago
An "off the top of my head" solution to this would be not to yield tokens until a terminating character (comma, \n, }).