3 ms·
Well, you have to define a buffer size for the lexer, for both avoiding dynamic allocations and avoid bowing your memory. A 512 bytes buffer can hold 509 ASCII
by marcosdumay 5y ago
Well, you have to define a buffer size for the lexer, for both avoiding dynamic allocations and avoid bowing your memory. A 512 bytes buffer can hold 509 ASCII characters, two quotes and a zero terminator.
I don't know why they did it, but the size makes a lot of sense.
- Someone 5y agoBut why would the lexer include the delimiting quotes in the token for a string literal?
- marcosdumay 5y agoYou probably wouldn't. But a standard should allow the most possible kinds of usage, and there are some situation where it makes sense (like on transpilers).