4 ms·

The full set of PCREs, as the author points out, are NP-complete. The problem is, just because you can do something doesn't mean you should. If you ever find y

The full set of PCREs, as the author points out, are NP-complete. The problem is, just because you can do something doesn't mean you should. If you ever find yourself constructing a recursively enumerable grammar (or even a CFG) using a regular expressions - whether PCREs or any other variant - you should ask yourself why you aren't using a parser generator or a proper tool for creating a compiler front-end. I hope people don't miss the author's closing point, which is the most important part: > But don’t forget: Just because you can, doesn’t mean that you should. Processing HTML with regular expressions is a really bad idea in some cases. In other cases it’s probably the best thing to do. I disagree that there are cases in which it's probably the best thing to do. Most languages support XPATH/CSS selectors/etc., which are much better tools for matching arbitrary HTML patterns. I'm guilty of conjuring up a regex to scrape image links every now and then, but you should really only do that when your domain of expected input data is far more restricted than the actual CFG that you're dealing with.

abecedarius 14y ago
I'm guilty of conjuring up a regex to scrape image links every now and then Any idea how to supersede that temptation? That is, what's wrong with the HTML-processing tool/sublanguage that makes regexes attractive instead? (For me, it's that I don't need to scrape HTML quite often enough to want to learn to use what's available. I'm probably irrationally lazy.)
- rogerbinns 14y ago
  Usually the problem is that the APIs are like typical XML apis which are a royal pain to use requiring various hoop jumping in order to say which tags are of interest in which container and relationships with siblings. If you are lucky you have to learn some sort of XPATH like syntax. Meanwhile you do a view source, and can trivially see exactly which lines you want, and say sod all that nonsense - I'll use a regex. My recommendation is to find a library that provides jQuery/CSS selector style syntax and semantics, and then suddenly it is a lot easier to deal with the document. For example for Python there is soupselect or cssselect. Amusingly the latter shows that the selector "div.content" translates into the XPATH "descendant-or-self::div[@class and contains(concat(' ', normalize-space(@class), ' '), ' content ')]".
meric 14y ago
Let's say you're correcting a 200 line HTML file, currently open in your text editor, to make it look neater. E.g. fixing tags like < p id='some_id'> It would be ideal to use regular expression find and replace to look for: < ([a-z]) and replace with: <$1 Of course, be sure to review every replacement to make sure it isn't part of javascript or something like that. IMO it'd be faster than writing a script and then running it against the file.

abecedarius 14y ago

I'm guilty of conjuring up a regex to scrape image links every now and then Any idea how to supersede that temptation? That is, what's wrong with the HTML-processing tool/sublanguage that makes regexes attractive instead? (For me, it's that I don't need to scrape HTML quite often enough to want to learn to use what's available. I'm probably irrationally lazy.)

rogerbinns 14y ago
Usually the problem is that the APIs are like typical XML apis which are a royal pain to use requiring various hoop jumping in order to say which tags are of interest in which container and relationships with siblings. If you are lucky you have to learn some sort of XPATH like syntax. Meanwhile you do a view source, and can trivially see exactly which lines you want, and say sod all that nonsense - I'll use a regex. My recommendation is to find a library that provides jQuery/CSS selector style syntax and semantics, and then suddenly it is a lot easier to deal with the document. For example for Python there is soupselect or cssselect. Amusingly the latter shows that the selector "div.content" translates into the XPATH "descendant-or-self::div[@class and contains(concat(' ', normalize-space(@class), ' '), ' content ')]".

meric 14y ago

Let's say you're correcting a 200 line HTML file, currently open in your text editor, to make it look neater. E.g. fixing tags like < p id='some_id'> It would be ideal to use regular expression find and replace to look for: < ([a-z]) and replace with: <$1 Of course, be sure to review every replacement to make sure it isn't part of javascript or something like that. IMO it'd be faster than writing a script and then running it against the file.