5 ms·

In the bad old days, parsing real-world HTML was a horrible task because every web-browser had a huge collection of undocumented corner-cases and hacks; some ac

In the bad old days, parsing real-world HTML was a horrible task because every web-browser had a huge collection of undocumented corner-cases and hacks; some accidental, some the result of reverse-engineering other vendors' corner-cases and hacks. Most standalone HTML parsers could generate some document tree from a given input file; whether or not it would match the one generated by an actual browser was another matter. These days, however, we have the HTML5 parsing algorithm, reverse-engineered from various vendors' web browsers but actually documented and implementable (still horribly complicated, but that's legacy content for you). Not only is the HTML5 parsing algorithm designed to be compatible with legacy browsers, modern browsers are replacing their old parsing code with new HTML5-compatible implementations, so parsing should be even more consistent (I know Firefox has switched to an HTML5 parser, I think IE has made a bunch of noise about it too; I don't follow WebKit all that closely, but I'd be surprised if they haven't moved towards an HTML5 parser).

justincormack 15y ago
Yes webkit has too http://www.webkit.org/blog/1273/the-html5-parsing-algorithm/ http://www.webkit.org/blog/1273/the-html5-parsing-algorithm/
justinschuh 15y ago
WebKit has been using the new algorithm since mid 2010. And I'm pretty sure Chrome 7 (Octoboer 2010) was the first major browser to ship an HTML5 compliant parser.
- hoppipolla 15y ago
  Right, WebKit shipped code based on the spec first but the spec itself underwent subsequent revision as the Gecko / Presto implementors found site compatibility issues and bugs. I think the WebKit implementation was recently updated to the spec, so we are at, or at least very close to, having very interoperable HTML parsing in Opera/Firefox/Safari/Chrome. I also believe that Microsoft are aiming to implement the new algorithm in IE 10. From an interoperability point of view the HTML parsing algorithm is the poster child for the success of the HTML effort; there is a testsuite of several thousand tests [1] (also submitted to the W3C [2]) that has contributions from multiple browser vendors and a number of unaffiliated individuals. Although parsing isn't sexy in the way that, say, <canvas> is, getting interoperable parsing makes it much easier to create cross-browser content (at Opera we closed a huge number of site-compatibilty bugs when we landed the new algorithm). There are also a few open-source implementations that are not tied to browsers e.g. for python (and kind of also PHP) [3], for java [4] (fun fact: the gecko C++ implementation is generated from that java implementation) and javascript [5] https://github.com/andreasgal/dom.js https://github.com/andreasgal/dom.js It would be great to see more conforming implementations for other languages, or to see libraries like libxml2 that have existing ad-hoc HTML parsers update their implementations to match the spec. [1] http://code.google.com/p/html5lib/source/browse/#hg%2Ftestdata http://code.google.com/p/html5lib/source/browse/#hg%2Ftestda... [2] http://w3c-test.org/html/tests/submission/Opera/html5lib/ http://w3c-test.org/html/tests/submission/Opera/html5lib/ [3] http://code.google.com/p/html5lib/ http://code.google.com/p/html5lib/ [4] http://about.validator.nu/htmlparser/ http://about.validator.nu/htmlparser/ [5] https://github.com/andreasgal/dom.js https://github.com/andreasgal/dom.js
masklinn 15y ago
> I know Firefox has switched to an HTML5 parser Yep, this was mainlined in Firefox 4 (with Gecko 2.0). > I think IE has made a bunch of noise about it too Support is being built, it's planned for IE10. > I don't follow WebKit all that closely, but I'd be surprised if they haven't moved towards an HTML5 parser The HTML5 parsing algorithm has been in Webkit since the second half of 2010. And you have not asked, but HTML5 parsing was officially released in Opera 11.6 last month.
- kkolev 15y ago
  > HTML5 parsing was officially released in Opera 11.6 last month. I hope that's not related to the annoying freezes the community's been complaining about since that release...
_delirium 15y ago
Theres a Perl implementation in CPAN as well, though I haven't made nontrivial use of it, so I'm not sure how fast/robust it is: http://search.cpan.org/~tobyink/Task-HTML5-0.103/lib/Task/HTML5.pm http://search.cpan.org/~tobyink/Task-HTML5-0.103/lib/Task/HT...

justincormack 15y ago

Yes webkit has too http://www.webkit.org/blog/1273/the-html5-parsing-algorithm/ http://www.webkit.org/blog/1273/the-html5-parsing-algorithm/

justinschuh 15y ago

WebKit has been using the new algorithm since mid 2010. And I'm pretty sure Chrome 7 (Octoboer 2010) was the first major browser to ship an HTML5 compliant parser.

hoppipolla 15y ago
Right, WebKit shipped code based on the spec first but the spec itself underwent subsequent revision as the Gecko / Presto implementors found site compatibility issues and bugs. I think the WebKit implementation was recently updated to the spec, so we are at, or at least very close to, having very interoperable HTML parsing in Opera/Firefox/Safari/Chrome. I also believe that Microsoft are aiming to implement the new algorithm in IE 10. From an interoperability point of view the HTML parsing algorithm is the poster child for the success of the HTML effort; there is a testsuite of several thousand tests [1] (also submitted to the W3C [2]) that has contributions from multiple browser vendors and a number of unaffiliated individuals. Although parsing isn't sexy in the way that, say, <canvas> is, getting interoperable parsing makes it much easier to create cross-browser content (at Opera we closed a huge number of site-compatibilty bugs when we landed the new algorithm). There are also a few open-source implementations that are not tied to browsers e.g. for python (and kind of also PHP) [3], for java [4] (fun fact: the gecko C++ implementation is generated from that java implementation) and javascript [5] https://github.com/andreasgal/dom.js https://github.com/andreasgal/dom.js It would be great to see more conforming implementations for other languages, or to see libraries like libxml2 that have existing ad-hoc HTML parsers update their implementations to match the spec. [1] http://code.google.com/p/html5lib/source/browse/#hg%2Ftestdata http://code.google.com/p/html5lib/source/browse/#hg%2Ftestda... [2] http://w3c-test.org/html/tests/submission/Opera/html5lib/ http://w3c-test.org/html/tests/submission/Opera/html5lib/ [3] http://code.google.com/p/html5lib/ http://code.google.com/p/html5lib/ [4] http://about.validator.nu/htmlparser/ http://about.validator.nu/htmlparser/ [5] https://github.com/andreasgal/dom.js https://github.com/andreasgal/dom.js

masklinn 15y ago

> I know Firefox has switched to an HTML5 parser Yep, this was mainlined in Firefox 4 (with Gecko 2.0). > I think IE has made a bunch of noise about it too Support is being built, it's planned for IE10. > I don't follow WebKit all that closely, but I'd be surprised if they haven't moved towards an HTML5 parser The HTML5 parsing algorithm has been in Webkit since the second half of 2010. And you have not asked, but HTML5 parsing was officially released in Opera 11.6 last month.

kkolev 15y ago
> HTML5 parsing was officially released in Opera 11.6 last month. I hope that's not related to the annoying freezes the community's been complaining about since that release...

_delirium 15y ago

Theres a Perl implementation in CPAN as well, though I haven't made nontrivial use of it, so I'm not sure how fast/robust it is: http://search.cpan.org/~tobyink/Task-HTML5-0.103/lib/Task/HTML5.pm http://search.cpan.org/~tobyink/Task-HTML5-0.103/lib/Task/HT...