3 ms·
> the parser component is a context-free parser > The Python lexer ... is the canonical non-regular, context-free language! So we have a context-free lexer an
by perfunctory 7y ago
> the parser component is a context-free parser
> The Python lexer ... is the canonical non-regular, context-free language!
So we have a context-free lexer and a context-free parser. How come the total is not context free?
- nightcracker 7y agoThe lexer isn't context-free, and you should re-read the paragraph. The part that you left out in the dots is crucial: > The Python lexer keeps, in addition, a stack of counters for the indentation levels. Moreover, it keeps track of the nesting of parentheses; and the language of balanced parentheses is the canonical non-regular, context-free language! So in this design, there is no clear separation of the context-free part and the context-sensitive part, and the context-sensitive part goes well beyond what a typical lexer can do.
- deleted 7y ago[deleted]
- marmada 7y agoDid you ignore the next sentence: So in this design, there is no clear separation of the context-free part and the context-sensitive part, and the context-sensitive part goes well beyond what a typical lexer can do.