3 ms·

> Want to parse a string in JSON? okay, this is a starting quote, so let's iterate down the rest of the document, looking for unescaped end quotes, and then cop

> Want to parse a string in JSON? okay, this is a starting quote, so let's iterate down the rest of the document, looking for unescaped end quotes, and then copy that into a buffer. Yes, so what's the problem? Meanwhile keep in mind that the JSON document was already passed in a HTTP request body, or that its trivial to put string length checks on a parser, or even nesting limit checks.

hn_throwaway_99 7y ago
Delimited formats are, in general, slower to parse than formats where the record sizes are encoded in the message. That said, not something I want to prematurely optimize for.
- strbean 7y ago
  I don't think it's that premature. Making the (big) assumption that serialization format X has good stable support in your language, and has the same expressive capabilities as JSON, you should be able to use it as a drop in replacement (as long as you control both ends of the communication...). And doesn't a huge portion of what we do around here amount to "shuffle serialized data around as efficiently as we can"?
- rumanator 7y ago
  > Delimited formats are, in general, slower to parse than formats where the record sizes are encoded in the message. I don't see your point at all. The state machine required to parse a JSON string essentially has only 2 states: current token is either a string character or a string delimiter. That's it. Adding a record size increases the number of states because now you not only need to track if a token is a string character but also if the character makes sense to be there within that state. Moreover, when you parse a JSON string, once you hit the string's end delimiter you already are able to calculate the string size. Thus, if the string is short enough to fit the buffer then not only is the lexer simpler but it also requires the same memory allocations than strig formats with record sizes. If however a string doesn't fit the buffer then we are already in the territory of a ropes data structure in both cases, thus the number of memory allocations tend to be equivalent.

hn_throwaway_99 7y ago

Delimited formats are, in general, slower to parse than formats where the record sizes are encoded in the message. That said, not something I want to prematurely optimize for.

strbean 7y ago
I don't think it's that premature. Making the (big) assumption that serialization format X has good stable support in your language, and has the same expressive capabilities as JSON, you should be able to use it as a drop in replacement (as long as you control both ends of the communication...). And doesn't a huge portion of what we do around here amount to "shuffle serialized data around as efficiently as we can"?
rumanator 7y ago
> Delimited formats are, in general, slower to parse than formats where the record sizes are encoded in the message. I don't see your point at all. The state machine required to parse a JSON string essentially has only 2 states: current token is either a string character or a string delimiter. That's it. Adding a record size increases the number of states because now you not only need to track if a token is a string character but also if the character makes sense to be there within that state. Moreover, when you parse a JSON string, once you hit the string's end delimiter you already are able to calculate the string size. Thus, if the string is short enough to fit the buffer then not only is the lexer simpler but it also requires the same memory allocations than strig formats with record sizes. If however a string doesn't fit the buffer then we are already in the territory of a ropes data structure in both cases, thus the number of memory allocations tend to be equivalent.