This could be useful for writing lexers. Although I typically need to interleave some "decoding" with proper lexing (like decoding escape sequences in strings,

This could be useful for writing lexers. Although I typically need to interleave some "decoding" with proper lexing (like decoding escape sequences in strings, universal character names in identifiers etc.), which would make it harder, when you want to do it in a single pass.