all the "string theory" means is that you cannot parse html completely or check its validity with a SINGLE regex based on a standard nfa or dfa. the reason why

all the "string theory" means is that you cannot parse html completely or check its validity with a SINGLE regex based on a standard nfa or dfa. the reason why is that standard nfas and dfas cannot implement arbitrary counters for the purposes of keeping track of recursion depth or "how many open parens are currently open right now in the parsing?" there are a few subtle points here: 1) you CAN parse and validate non-regular languages by using regular expressions for subsets of them that ARE regular and then validating the whole. this is exactly what most parser generators generate. 2) you CAN use regular expressions to manipulate text in regular languages if you are careful. however, you should be extremely cautious, as non-regular parts of languages are often those that are used to support escaping or quoting or recursive sections of languages and subtle bugs in these sorts of things can sometimes have unintended consequences in terms of security. 3) you CAN use regular expressions for simple search and replace but see #2. if the language allows escaping or quoting, your regexes will not respect it on their own. this may have the unintended consequence of things you expect to have escaped not be escaped or vice versa. depending on other assertions in your project, this may or may not matter. in the worst case, it could result in a hard to track down or security critical bug. 4) regular expressions are implemented by simulating nfas and dfas. sometimes when implementing these simulations, programmers take advantage of the fact that they're programming and just slap counters in there giving their implementations the ability to support some non-regular languages. as a rule of thumb, if i'm dealing with untrusted input, i'll use a proper parser. so yes, you can use regular expressions, but you should understand their limitations if you do so. (which also means understanding your input language)