3 ms·
I like writing compilers, how do languages like Haskell and Python handle blocks-by-indentation? I figure you have some sort of INDENT token, but how do you kn
by gnuvince 12y ago
I like writing compilers, how do languages like Haskell and Python handle blocks-by-indentation? I figure you have some sort of INDENT token, but how do you know its width? Also, in specifying the grammar, with explicitly delimited blocks you just say `block := OPEN stmt_list CLOSE`. What about with indentation?
- TazeTSchnitzel 12y agoI think they specify "INDENT foo DEDENT" and the lexer tracks indentation and issues INDENT/DEDENT as appropriate.
- falcolas 12y agoThis is correct. A bit of state (an indentation width stack) in the lexer creates the block boundaries as clearly for the interpreter as normal braces does. Any problems in parsing are typically only encountered by stateless parsers.
- dochtman 12y agoI have one (probably quite naive) implementation here: https://github.com/djc/runa/blob/master/runac/parser.py#L54 https://github.com/djc/runa/blob/master/runac/parser.py#L54 (Runa uses tabs for indentation, not spaces, but it's basically the same thing.) Whitespace at the start of a line is a separate token, then I add a thin layer of processing that turns a change of indentation level (i.e. count chars in the token contents) into INDENT or DEDENT tokens.