7 ms·

The HTML5 Parsing Algorithm

85 points by sant0sk1 16y ago

mbrubeck 16y ago
Yay! Also coming in Firefox 4: http://hacks.mozilla.org/2010/05/firefox-4-the-html5-parser-inline-svg-speed-and-more/ http://hacks.mozilla.org/2010/05/firefox-4-the-html5-parser-... and IE9: http://blogs.msdn.com/b/ie/archive/2010/03/16/html5-hardware-accelerated-first-ie9-platform-preview-available-for-developers.aspx http://blogs.msdn.com/b/ie/archive/2010/03/16/html5-hardware... Unfortunately the new parsing algorithm breaks my online banking site: http://bugzil.la/565689 http://bugzil.la/565689
- palish 16y ago
  Ugh, their response: This is an intentional change. Prior to the HTML5 parsing algorithm, browsers backtracked and reparsed when seeing an EOF inside a script to deal with </script> inside an inline script. This means that prior to HTML5, an accidental or maliciously forced premature end of file could change the executability properties of pieces of an HTML file. The current magic in the spec was carefully designed and researched to permit forward-only parsing in a maximally Web-compatible way. The solution in the spec was known to break a handful of pages among lots and lots of pages listed by dmoz, but the breakage was deemed negligible. This is probably the highest-risk change in the HTML5 parsing algorithm, and this is the first report of it breaking an "important" contemporary site. Considering that keeping the forward-only tokenization behavior is highly desirable, in the absence of evidence of more important breakage, I'm treating this as an evang issue realizing that further evidence may force us to revisit this part of the spec. But let's try to get away with forward-only parsing! What is the logic behind this? Just because we want "forward-only parsing", we're going to try to force people not to write </script> inside of a Javascript string? That seems rather silly. Are there any benefits to forward-only parsing besides speed? Because if this change was done in the name of performance, then we really should re-evaluate who exactly is proposing these types of changes and why.
  - cameronh90 16y ago
    Forward only parsing is not only faster, it's less complicated which should translate to fewer bugs. It's worth noting that the HTML parser isn't used only when initially rendering the page, it's also used when rewriting the contents with JavaScript. The bug report has a perfectly reasonable workaround you can use, and you can always put your JavaScript in a separate file or escape < >. It's a good idea to do this even with the current generation of browsers, because if you don't, you sometimes get strange bugs... often bugs that are not reproducible in all browsers. Honestly, allowing unescaped </script> inside inline javascript wasn't a good idea to begin with.
  - judofyr 16y ago
    browsers backtracked and reparsed when seeing an EOF inside a script to deal with </script> inside an inline script Correct me if I'm wrong, but it seems to me that in order to detect that the </script> is inside an inline script, you'll have to parse the JavaScript. Requiring every HTML5 parser to include a JavaScript parser in order to parse it correctly seems a little too much for me. Another reason: code complexity. Backtracking makes everything a lot more complex which is the exact opposite of what they're trying to achieve with the HTML5 parsing algorithm. Less code = fewer bugs, easier to understand and easier to implement.
  - pornel 16y ago
    ECMAScript has escape sequence just for this occasion: <\/script> Works everywhere (it's shame so few people know about it and use uglier and invalid "</sc"+"ript>").
    - pbiggar 16y ago
      Why is that invalid?
      - pornel 16y ago
        http://www.w3.org/TR/html4/types.html http://www.w3.org/TR/html4/types.html "The first occurrence of the character sequence "</" (end-tag open delimiter) is treated as terminating the end of the element's content." Of course browsers never implemented this correctly.
tedunangst 16y ago
"Together, these two algorithms form the core of the parser and consist of over 10,000 lines of code." Anybody else think that if your markup language requires a 10k line parser, somebody took a wrong turn at the complexity vs simplicity fork in the road?
- syncsynchalt 16y ago
  This must be your first time hearing about HTML, welcome! Just imagine how large "quirks mode" must be.
- j-g-faustus 16y ago
  I tend to think that very few things that see widespread real-world use are "simple" by any reasonable metric. The basic idea may be simple, but by the time you have covered a reasonable fraction of real-world use cases and error conditions for a sizeable population, the resulting code is anything but. And 10k lines still counts as "moderately simple" - 10k lines is well within scope for one developer working alone.
- btn 16y ago
  What's the magic number of LOC that any markup language should be implemented under?
  - tedunangst 16y ago
    To put 10k in perspective, that's substantially more than the pcc C compiler front end, and closing in on the compiler's total line count. When I consider what I can accomplish using C vs what I can accomplish using HTML, that doesn't appear to be the expected result.
    - plorkyeran 16y ago
      C compilers don't have to deal with users that expect purely random garbage thrown at the parsers to have a "valid" parsing that's consistent across every implementation. Unfortunately, making a language simple to use by not requiring that users get everything right makes it very hard to implement.
jimmyjazz14 16y ago
I would prefer that the parser just failed when invalid code was encountered, personally I saw this as a strong point of XHTML.
- benhoyt 16y ago
  Hmmm. I suspect that would break 90% of web pages on the web. On the other hand, people would make their pages valid pretty quick. :-)
  - wlievens 16y ago
    What about pages that are not maintained but still carry value?
    - deleted 16y ago
      [deleted]
  - wilhelm 16y ago
    No. Browsers that implemented this would just lose their market share instantly. Users prefer being able to use their favourite sites.
- ugh 16y ago
  One of the nice things about the web is that all websites still work. Here is the first webpage: http://www.w3.org/History/19921103-hypertext/hypertext/WWW/TheProject.html http://www.w3.org/History/19921103-hypertext/hypertext/WWW/T... That one wouldn’t work with a strict parser (look at the source: it uses the header tag instead of the head tag). Breaking half or more of all the websites out there wouldn’t exactly be what I would call the “spirit of the www”.
  - tedunangst 16y ago
    Use a new parser for new pages, the old parser for old pages. The introduction of H.264 didn't suddenly render all older formats unplayable. Why can't a new markup format coexist with older formats, too?

85 points by sant0sk1 16y ago

Yay! Also coming in Firefox 4: http://hacks.mozilla.org/2010/05/firefox-4-the-html5-parser-inline-svg-speed-and-more/ http://hacks.mozilla.org/2010/05/firefox-4-the-html5-parser-... and IE9: http://blogs.msdn.com/b/ie/archive/2010/03/16/html5-hardware-accelerated-first-ie9-platform-preview-available-for-developers.aspx http://blogs.msdn.com/b/ie/archive/2010/03/16/html5-hardware... Unfortunately the new parsing algorithm breaks my online banking site: http://bugzil.la/565689 http://bugzil.la/565689

palish 16y ago
Ugh, their response: This is an intentional change. Prior to the HTML5 parsing algorithm, browsers backtracked and reparsed when seeing an EOF inside a script to deal with </script> inside an inline script. This means that prior to HTML5, an accidental or maliciously forced premature end of file could change the executability properties of pieces of an HTML file. The current magic in the spec was carefully designed and researched to permit forward-only parsing in a maximally Web-compatible way. The solution in the spec was known to break a handful of pages among lots and lots of pages listed by dmoz, but the breakage was deemed negligible. This is probably the highest-risk change in the HTML5 parsing algorithm, and this is the first report of it breaking an "important" contemporary site. Considering that keeping the forward-only tokenization behavior is highly desirable, in the absence of evidence of more important breakage, I'm treating this as an evang issue realizing that further evidence may force us to revisit this part of the spec. But let's try to get away with forward-only parsing! What is the logic behind this? Just because we want "forward-only parsing", we're going to try to force people not to write </script> inside of a Javascript string? That seems rather silly. Are there any benefits to forward-only parsing besides speed? Because if this change was done in the name of performance, then we really should re-evaluate who exactly is proposing these types of changes and why.
- cameronh90 16y ago
  Forward only parsing is not only faster, it's less complicated which should translate to fewer bugs. It's worth noting that the HTML parser isn't used only when initially rendering the page, it's also used when rewriting the contents with JavaScript. The bug report has a perfectly reasonable workaround you can use, and you can always put your JavaScript in a separate file or escape < >. It's a good idea to do this even with the current generation of browsers, because if you don't, you sometimes get strange bugs... often bugs that are not reproducible in all browsers. Honestly, allowing unescaped </script> inside inline javascript wasn't a good idea to begin with.
- judofyr 16y ago
  browsers backtracked and reparsed when seeing an EOF inside a script to deal with </script> inside an inline script Correct me if I'm wrong, but it seems to me that in order to detect that the </script> is inside an inline script, you'll have to parse the JavaScript. Requiring every HTML5 parser to include a JavaScript parser in order to parse it correctly seems a little too much for me. Another reason: code complexity. Backtracking makes everything a lot more complex which is the exact opposite of what they're trying to achieve with the HTML5 parsing algorithm. Less code = fewer bugs, easier to understand and easier to implement.
- pornel 16y ago
  ECMAScript has escape sequence just for this occasion: <\/script> Works everywhere (it's shame so few people know about it and use uglier and invalid "</sc"+"ript>").
  - pbiggar 16y ago
    Why is that invalid?
    - pornel 16y ago
      http://www.w3.org/TR/html4/types.html http://www.w3.org/TR/html4/types.html "The first occurrence of the character sequence "</" (end-tag open delimiter) is treated as terminating the end of the element's content." Of course browsers never implemented this correctly.

tedunangst 16y ago

"Together, these two algorithms form the core of the parser and consist of over 10,000 lines of code." Anybody else think that if your markup language requires a 10k line parser, somebody took a wrong turn at the complexity vs simplicity fork in the road?

syncsynchalt 16y ago
This must be your first time hearing about HTML, welcome! Just imagine how large "quirks mode" must be.
j-g-faustus 16y ago
I tend to think that very few things that see widespread real-world use are "simple" by any reasonable metric. The basic idea may be simple, but by the time you have covered a reasonable fraction of real-world use cases and error conditions for a sizeable population, the resulting code is anything but. And 10k lines still counts as "moderately simple" - 10k lines is well within scope for one developer working alone.
btn 16y ago
What's the magic number of LOC that any markup language should be implemented under?
- tedunangst 16y ago
  To put 10k in perspective, that's substantially more than the pcc C compiler front end, and closing in on the compiler's total line count. When I consider what I can accomplish using C vs what I can accomplish using HTML, that doesn't appear to be the expected result.
  - plorkyeran 16y ago
    C compilers don't have to deal with users that expect purely random garbage thrown at the parsers to have a "valid" parsing that's consistent across every implementation. Unfortunately, making a language simple to use by not requiring that users get everything right makes it very hard to implement.

jimmyjazz14 16y ago

I would prefer that the parser just failed when invalid code was encountered, personally I saw this as a strong point of XHTML.

benhoyt 16y ago
Hmmm. I suspect that would break 90% of web pages on the web. On the other hand, people would make their pages valid pretty quick. :-)
- wlievens 16y ago
  What about pages that are not maintained but still carry value?
  - deleted 16y ago
    [deleted]
- wilhelm 16y ago
  No. Browsers that implemented this would just lose their market share instantly. Users prefer being able to use their favourite sites.
ugh 16y ago
One of the nice things about the web is that all websites still work. Here is the first webpage: http://www.w3.org/History/19921103-hypertext/hypertext/WWW/TheProject.html http://www.w3.org/History/19921103-hypertext/hypertext/WWW/T... That one wouldn’t work with a strict parser (look at the source: it uses the header tag instead of the head tag). Breaking half or more of all the websites out there wouldn’t exactly be what I would call the “spirit of the www”.
- tedunangst 16y ago
  Use a new parser for new pages, the old parser for old pages. The introduction of H.264 didn't suddenly render all older formats unplayable. Why can't a new markup format coexist with older formats, too?