6 ms·

> This is one reason why html5parser for Python is so slow. I presume you mean html5lib and not the new html5-parser; html5lib isn't slow because of error reco

> This is one reason why html5parser for Python is so slow. I presume you mean html5lib and not the new html5-parser; html5lib isn't slow because of error recovery: html5lib is slow because it's a parser written in Python. The parsing performance is largely taken up, last I benchmarked the VM, by allocation overhead (heck, reading s[0] of a string s causes an allocation) and super-simple VM instructions (primarily dispatch overhead). What I'd like to see is how much faster an HTML parser would be if it rejected any document with a parse error at the first parse error; I doubt it'd be much (you'd still need the branches to detect the parse error, though the code might be smaller and you'd get better cache locality).

kbenson 9y ago
Have a super strict engine that parses as fast as possible, and do a full fail on a parse error and fall back to a more lax engine and start parsing from scratch. People that care about speed will notice that they can get a speedy page by being strict, and those that don't care will see a slight slowdown in their page times. Automatic incentivization of good future behavior.
- zAy0LfpBZLC8mAC 9y ago
  That sounds like a path towards fixing this mess that could actually work! I mean, you even could add deliberate delays, simply to incentivise site operators, because they tend to care about even relatively small effects on conversion rates and the like, while for most end users some sluggishness on some bad websites probably would not be enough to cause switching to a different browser?!
- inimino 9y ago
  This idea comes up a lot, but it is based on faulty assumptions. First, time parsing HTML is irrelevant compared to network latency and even DOM and CSS work that browsers do. Second, a parser that handled only, say, strict XML would certainly be easier for the parser author to write, but it would be slower than the optimized parsers browsers already have for HTML. If optimized, it still would not be significantly faster than a full HTML5 parser. Your idea wouldn't work because it would never create any noticeable impact on page loading speed even in the best case scenario. That's assuming that browsers want to maintain and ship two totally different parsers, which (cf. XHTML) they don't.
  - kbenson 9y ago
    > time parsing HTML is irrelevant compared to network latency and even DOM and CSS work that browsers do. This is likely true for most sites on the internet. It is worth noting that on an overly large HTML file of ~1.4 MB, stuffing that into an HTML element object by assigning innerHTML with that content takes approximately 75 ms on my i7-6700HQ.[1] A much larger file of 10 MB took 656 ms to parse, which seems to indicate it scales linearly and processes ~16KB/ms (16MB/s). For the most heavy real sites I could find, it generally took no longer than 25 ms. It's worth noting that 16MB/s is actually only slightly faster than 100Mb/s, so some people may actually take longer parsing than receiving the HTML. That said, the parsing/processing tested here may be doing more than what we strictly care about for this diuscussion, so I'm not including it as a rebuttal, but as a point of reference that thought was interesting. 1: var el = document.createElement('html'); console.log((new Date()).getTime()); el.innerHTML = htmlstr; console.log((new Date()).getTime());

kbenson 9y ago

Have a super strict engine that parses as fast as possible, and do a full fail on a parse error and fall back to a more lax engine and start parsing from scratch. People that care about speed will notice that they can get a speedy page by being strict, and those that don't care will see a slight slowdown in their page times. Automatic incentivization of good future behavior.

zAy0LfpBZLC8mAC 9y ago
That sounds like a path towards fixing this mess that could actually work! I mean, you even could add deliberate delays, simply to incentivise site operators, because they tend to care about even relatively small effects on conversion rates and the like, while for most end users some sluggishness on some bad websites probably would not be enough to cause switching to a different browser?!
inimino 9y ago
This idea comes up a lot, but it is based on faulty assumptions. First, time parsing HTML is irrelevant compared to network latency and even DOM and CSS work that browsers do. Second, a parser that handled only, say, strict XML would certainly be easier for the parser author to write, but it would be slower than the optimized parsers browsers already have for HTML. If optimized, it still would not be significantly faster than a full HTML5 parser. Your idea wouldn't work because it would never create any noticeable impact on page loading speed even in the best case scenario. That's assuming that browsers want to maintain and ship two totally different parsers, which (cf. XHTML) they don't.
- kbenson 9y ago
  > time parsing HTML is irrelevant compared to network latency and even DOM and CSS work that browsers do. This is likely true for most sites on the internet. It is worth noting that on an overly large HTML file of ~1.4 MB, stuffing that into an HTML element object by assigning innerHTML with that content takes approximately 75 ms on my i7-6700HQ.[1] A much larger file of 10 MB took 656 ms to parse, which seems to indicate it scales linearly and processes ~16KB/ms (16MB/s). For the most heavy real sites I could find, it generally took no longer than 25 ms. It's worth noting that 16MB/s is actually only slightly faster than 100Mb/s, so some people may actually take longer parsing than receiving the HTML. That said, the parsing/processing tested here may be doing more than what we strictly care about for this diuscussion, so I'm not including it as a rebuttal, but as a point of reference that thought was interesting. 1: var el = document.createElement('html'); console.log((new Date()).getTime()); el.innerHTML = htmlstr; console.log((new Date()).getTime());