3 ms·
Yes, it's hopeless to try and parse arbitrary HTML with regexes (or with anything really. I believe, the full HTML5 parsing algorithm is not even type-2, it's m
by xg15 6y ago
Yes, it's hopeless to try and parse arbitrary HTML with regexes (or with anything really. I believe, the full HTML5 parsing algorithm is not even type-2, it's more or less turing-complete. In-the-wild HTML can also interact with scripting in all kinds of entertaining ways, to the point that a conforming HTML5 parser has to be able to execute javascript while parsing - and the javascript can inject additional tokens into the parser's input stream. It's possible to create a HTML5 document that only validates on every second thursday of the month.)
However, if we know the document is well-formed(!) XHTML, shouldn't it be possible? This would mean the document is valid XML and XML was specifically designed to be regex-friendly, I believe.
At least, out of my head, the only gotchas that have to be accounted for are comments and CDATA sections - those may contain arbitrary unescaped text, including angle brackets. However they also have unambigous start and end markers and can't be nested, so a regex could account for them.
Attribute values should not be a problem, as angle brackets must be escaped inside those to be valid XML.
I'm not sure about processing instructions and doctypes though.
- theandrewbailey 6y agoOn my blog, I write my posts in markdown. After it's converted to HTML, I use regex to search and replace images (for high-res and alternative formats), and get the first paragraph (for a 'preview'). I've been doing this for years, so the 'never use regex over HTML' advice isn't holding up for me.
- chubot 6y agoYes that's basically what I wrote about here: https://news.ycombinator.com/item?id=26359556 https://news.ycombinator.com/item?id=26359556 I listed 3 or 4 caveats. CDATA might be another one since I'm not handling those... I've never used it so I left it out. Actually I remember that regular languages weren't entirely enough; I think the .*? non-greedy matching was useful, e.g. for finding --> at the end of comments.