3 ms·
Not sure what you mean by this. In the C standard, numeric tokens include signs, floating point dots, type specifiers (U, L, etc.), and much more. For the gory
by eliben 13y ago
Not sure what you mean by this.
In the C standard, numeric tokens include signs, floating point dots, type specifiers (U, L, etc.), and much more. For the gory details see: https://github.com/eliben/pycparser/blob/master/pycparser/c_lexer.py https://github.com/eliben/pycparser/blob/master/pycparser/c_...
But in the end, it's all in the same token.
- marshray 13y agoI mean tokenizing things like "8-8" vs "8+-8" vs "8---8" present some special challenges for the lexer.