3 ms·
Lexers run multiple regexes in tandem to determine the next token type. To do that with your method you'd have to run multiple DFAs on the same string. Lexers c
by jules 8y ago
Lexers run multiple regexes in tandem to determine the next token type. To do that with your method you'd have to run multiple DFAs on the same string. Lexers can do it with 1 DFA by having multiple different final states (non-final + 1 per token type vs non-final + final).
- ori_b 8y agoYou can do that from a regex api fairly easily. Something like: re = regex.compilemulti([ (".*@(.*)\\.(com|org|net)", TagEmail), ("http://.*, TagURL), ("[a-z]*", TagWord) ]) match regex.tagmatch(re, pat) | (TagEmail, submatches): domain=submatches[1] | (TagURL, submatches): url=submatches[0] | (TagWord, _): /* nothing */ ;; I haven't had time to do it in Myrddin's libregex (https://myrlang.org/doc/libregex/ https://myrlang.org/doc/libregex/) yet, but I've got plans to do something like this.
- jules 8y agoSure, and that's exactly what re2c does too. It lets you do a match on multiple regexes: regex1 { code that executes if regex1 matches } regex2 { code that executes if regex2 matches } regex3 { code that executes if regex3 matches }
- ori_b 8y agoYes, compiled into source code, which doesn't let you change the set of regexes at runtime.