You can't do it in general, e.g. to tokenize Python you'd need a layer that'll count tabs (because INDENT and DEDENT are tokens). That it, unless your regex dia

You can't do it in general, e.g. to tokenize Python you'd need a layer that'll count tabs (because INDENT and DEDENT are tokens). That it, unless your regex dialect is Turing complete, and if my memory serves, some of them are? I can't really remember. In general though, you can't do it with "normal" regex.