2 ms·
> Delimited formats are, in general, slower to parse than formats where the record sizes are encoded in the message. I don't see your point at all. The state m
by rumanator 7y ago
> Delimited formats are, in general, slower to parse than formats where the record sizes are encoded in the message.
I don't see your point at all. The state machine required to parse a JSON string essentially has only 2 states: current token is either a string character or a string delimiter. That's it. Adding a record size increases the number of states because now you not only need to track if a token is a string character but also if the character makes sense to be there within that state.
Moreover, when you parse a JSON string, once you hit the string's end delimiter you already are able to calculate the string size. Thus, if the string is short enough to fit the buffer then not only is the lexer simpler but it also requires the same memory allocations than strig formats with record sizes. If however a string doesn't fit the buffer then we are already in the territory of a ropes data structure in both cases, thus the number of memory allocations tend to be equivalent.