There is more to translation of text to AST than parsing and lexing. There are semantic checks. In addition, most languages are not CFG and require nested cont

There is more to translation of text to AST than parsing and lexing. There are semantic checks. In addition, most languages are not CFG and require nested contexts. "#{"\tThis is some #{'damn'} string".to_upper_case + " hello"}\n" Also, there are lexerless parsers such as derivative parsing that can be combined together and don't produce a stream of tokens. https://dl.acm.org/doi/10.1145/2034773.2034801 https://dl.acm.org/doi/10.1145/2034773.2034801 https://arxiv.org/pdf/1604.04695.pdf https://arxiv.org/pdf/1604.04695.pdf