4 ms·
"Together, these two algorithms form the core of the parser and consist of over 10,000 lines of code." Anybody else think that if your markup language requires
by tedunangst 16y ago
"Together, these two algorithms form the core of the parser and consist of over 10,000 lines of code."
Anybody else think that if your markup language requires a 10k line parser, somebody took a wrong turn at the complexity vs simplicity fork in the road?
- syncsynchalt 16y agoThis must be your first time hearing about HTML, welcome! Just imagine how large "quirks mode" must be.
- j-g-faustus 16y agoI tend to think that very few things that see widespread real-world use are "simple" by any reasonable metric. The basic idea may be simple, but by the time you have covered a reasonable fraction of real-world use cases and error conditions for a sizeable population, the resulting code is anything but. And 10k lines still counts as "moderately simple" - 10k lines is well within scope for one developer working alone.
- btn 16y agoWhat's the magic number of LOC that any markup language should be implemented under?
- tedunangst 16y agoTo put 10k in perspective, that's substantially more than the pcc C compiler front end, and closing in on the compiler's total line count. When I consider what I can accomplish using C vs what I can accomplish using HTML, that doesn't appear to be the expected result.
- plorkyeran 16y agoC compilers don't have to deal with users that expect purely random garbage thrown at the parsers to have a "valid" parsing that's consistent across every implementation. Unfortunately, making a language simple to use by not requiring that users get everything right makes it very hard to implement.
- j-g-faustus 16y agoConsidering the number of people that can accomplish anything useful with HTML vs the number of people that can accomplish anything useful with C (or even XML), it's well in line with my expectations. It's fairly easy to get a simpler implementation by shifting complexity from the parser to the user. But from a "total society productivity" standpoint, having a handful of developers spend a few months to implement a resilient parser is a cheap price to pay to have umpteen million people save hours of debugging for every page they create.
- sethg 16y agoBack when HTML was invented, it was based on SGML. In retrospect, this was a fundamental design flaw. SGML has all sorts of fillips that were intended to make documents easy to type (back in the pre-GUI era) but which in practice make them hard to parse. SGML parsers, even parsers built into expensive commercial software, were famous for not reliably interoperating. The HTML5 spec abandons SGML entirely because, as it turned out, even those fillips of SGML did not make it possible for browsers to consistently agree on how to parse HTML in the wild. It would have been better for everyone if HTML had used a bog-simple Lisp-like format from day one, but without a time machine we’re stuck with these kludges of history.
- deleted 16y ago[deleted]
- othermaciej 16y agoYou make it sound like it was a conscious decision. Most of the complexity comes from the need to tolerate the janky markup that older browsers processed in a reasonable way.
- rimantas 16y agoPay attention to this part: All browsers that implement the HTML5 parsing algorithm should parse HTML the same way, which means your web page should parse the same way in Firefox 4 and the WebKit nightly, even if it contains invalid markup. Parsing perfect HTML5 would be easy, but one of the features of HTML5 spec is that it does that no spec did before: it defines how parsing should work exactly, even in the case of invalid markup. Also, the parser hast to deal with deprecated elements no longer in the spec (such as infamous <font>). I assums most work went into this "how to parse tag soup" part.
- tedunangst 16y agoDeciding that invalid markup should work would be the wrong turn I alluded to. It would have been very easy to drop all the back compat bs by saying "An HTML5 document is one that begins with the 6 bytes '<html5' and if the document is invalid, reject it. Anything else, parse however you want." Browsers that support HTML5 add text/html5 to the Accept header.
- ori_b 16y agoSo, how many different parsers did you want to see in the browser again? You'd still need to support HTML4 somewhere. Supporting HTML5 separately just means duplicating the common parts of the parser. The simplicity boat has already sailed.
- tedunangst 16y agoAddition is simpler than combination. HTML4 + HTML5 < HTML4 * HTML5
- ori_b 16y agoThey're extremely similar. You don't get an explosion of code size. Unless you're suggesting that HTML5 should also be radically different syntactically as well?
- murrayh 16y agoJust for kicks, I am going to try and implement these two algorithms using using the State Machine Compiler (http://smc.sourceforge.net/ http://smc.sourceforge.net/). I expect that my "code" will be much less than 10,000 lines, although I don't really know how long the code SMC itself generates will be.
- astrange 16y agoIs there any advantage to SMC vs. Ragel for parsing? The code looks similar but a bit larger.
- murrayh 16y agoHeh, just what I need, a new toy to play with! I spent a few hours looking into Ragel over the weekend, and, honestly, I can't envisage how a html5 parser mostly written in Ragel would look. But I'm going to give it a crack.
- masklinn 16y agoMost markup languages (indeed most languages) don't have error discovery and recovery built in and specified. HTML5 does, and most of the spec (and probably the parser) is about dealing with error cases. Plus it's not like XML parsers are small. And HTML parsing is far more complex an affair (due to not just blowing up on error)
- gorm 16y agoHaving this complexity ensures that nobody will ever endeavor into writing any more browser engines, those ensuring that there will be no more browser bugs. The rest of the job is fixing the behavior of the already existing browsers and then the web will be perfect.
- pornel 16y agoHTML already tried the "let's switch to a simple parser" route, and failed: http://dig.csail.mit.edu/breadcrumbs/node/166 http://dig.csail.mit.edu/breadcrumbs/node/166 Now HTML5 has to deal with two parsers.