4 ms·

Worth noting the discussion at the link. Given that he omitted huge specs like WebGL etc. I wouldn't say it's wildly wrong. But I'd love to somehow arrive at a

Worth noting the discussion at the link. Given that he omitted huge specs like WebGL etc. I wouldn't say it's wildly wrong. But I'd love to somehow arrive at a better estimate.

cxr 4y ago
It's not hard. Start with the WHATWG's spec, then incorporate the other specs it references using a reasonable heuristic to determine if a given item should be included or not. If you don't think the estimate from Reckless, Infinite Scope is wildly off, then you either didn't read the methodology and do a spot-check of the dataset, or you really don't understand the scope of what gets published by W3C and how little much of it has to do with Web browsers or how many revisions of them there are.
- illiarian 4y ago
  It may not be possible to come up with a "reasonable heuristic": https://news.ycombinator.com/item?id=35524018 https://news.ycombinator.com/item?id=35524018
  - cxr 4y ago
    It is possible, and that sentence is verging on nonsense. A heuristic is not by definition perfect or optimal.
    - illiarian 4y ago
      > It is possible, and that sentence is verging on nonsense. Define "reasonable" then, when talking about the web.
      - cxr 4y ago
        The only bar that the heuristic has to pass here is "delivers a result that doesn't suck as bad as the analysis in Reckless, Infinite Scope". The analysis in that article is so bad, however, that your heuristic can literally be, "if you encounter an item that was also in Drew DeVault's input set, then assign an arbitrary probability 0.9 (or whatever) of whether the item should be counted", and it would still give you a more realistic result than what the article says (and that people are actually relying on in their arguments—and that you are defending) here. Aside from that, given how many logical errors and weird counterconclusions[1] you've managed to stuff into this discussion, though (and to have been able to do so economically[2]), I'm going to go ahead and say this is my last response to you that I spend more than 10 seconds writing out. 1. e.g. <https://news.ycombinator.com/item?id=35521704#35524952 https://news.ycombinator.com/item?id=35521704#35524952> 2. wrt number of words, fittingly
chrismorgan 4y ago
I believe it’s still wildly wrong. Had arp242 not spoken up at that time, I’d have been saying something similar, because the numbers were to me blindingly obviously extremely unrealistic. The entire HTML Standard (which is somewhat of a misnomer now, it covers much more than just HTML, quite a bit of CSS interactions, other web platform functionality, JavaScript APIs, and the likes) is now about half a million words by the most generous of counting methods (it’s written in a fairly verbose style, which is a really really really good thing when you compare it to the average IETF RFC), and I suspect it’s bigger than everything else put together, apart from ECMAScript (around 270,000 words¹ and probably growing at a faster rate than all other specs: it’s written in an even more verbose style, most of which is effectively straight code in prose form, whereas in the HTML Standard “straight code” is only a decent chunk of it). As for WebGL, the WebGL parts are actually quite little. https://registry.khronos.org/webgl/specs/latest/1.0/ https://registry.khronos.org/webgl/specs/latest/1.0/ is only about 20,000 words. I gather it defers significantly to GLES20 (PDF, 204 pages, ~60,000 words), and GLES20GLSL (PDF, 119 pages, ~30,000 words), and it has GL32CORE in its references (PDF, 404 pages, ~125,000 words), but doesn’t actually use cite it in the text and I don’t know if it’s relevant. There doesn’t look to be anything else significant that wouldn’t already be included. But really, WebGL is a fairly thin layer atop OpenGL ES 2.0, just removing some functionality and applying some restrictions. I believe you would reasonably expect a browser to use an existing OpenGL ES 2.0 implementation, so I’d be quite content to exclude the 90,000 (or perhaps it’s ~215,000?) words of that, just like it’s common to reuse an existing JavaScript engine (though you also don’t have to). Yet note this: it seems that even if we include it all (and presuming I haven’t missed anything, which I admit I could easily have done, I’m not conversant with these specs like I am with HTML/CSS/JS specs), it’s still under 0.2% of Drew’s massively-inflated figure. —⁂— ¹ Whew, https://262.ecma-international.org/ https://262.ecma-international.org/ took me several minutes to download, despite being only 7MB. Sigh; the trials of being in Australia, where things hosted in the USA are often inexplicably painfully slow—like, sub-256kbps. When already downloaded, it renders in under four seconds, which is really fairly impressive when it’s doing all that layout on a document a million pixels tall—this ain’t a PDF where you can only render one page at a time. The HTML Standard is almost two million pixels tall, and also loads completely in under four seconds—simpler styles, perhaps? I refer to it often enough that I build it locally so I don’t have to download its 13MB all the time, or compromise with the multipage version that you can’t search through as easily.
arp242 4y ago
The question is what exactly is needed for a useful and functional browser. You certainly don't need all features from Chrome, but you do need more than, say, Lynx or Dillo. Is WebGL needed? I've browsed the web for years with it disabled and have not suffered any inconvenience. I'd probably say it's not needed, but I'm a bit on the fence about it and can understand if people would disagree. All browsers implement XSLT, but is that actually needed for a functional modern browser? Maybe not? I can't remember the last time I've seen it used, but perhaps it is. And do you include HTTP? Or is that too low-level? Do you include PNG and SVG or just PNG? If you include SVG then why not PNG? There are some obvious "we need this", some obvious "we don't need this", and a lot of unclear and somewhat subjective area. I do know that you can't really say "yes there's bad data, but it probably cancels out against stuff omitted"; if anything, it only underscored my point that the list is not good. An uncurated or minimally curated document dump is not the correct approach in the first place, if you do that for SMTP you'd end up with a lot of irrelevant documents too simply because the specification is a few decades old and stuff gets superseded, some things never sees real-world implementations, things no one uses any more, etc. I started making a better list when the article was originally posted, starting from "okay, let's just check what you need for a useful browser normal people can use every day" and ended up with a few dozen things, but I never really posted it as I wasn't quite sure that was fully correct either and because I never really figured out some of the questions above. I think most of the complexity stem not just from the word count, but rather that everything interacts with everything else. Consider the relatively new "position: sticky" in CSS. Okay, great. But it doesn't work well with flexboxes, or RTL, or negative margins, or z-index, etc. etc. [1] Adding what seems like a fairly simple feature is quite complex because it interacts with so many things. It's not hard to imagine a fresh new HTML and CSS which allows all the features the current does but does so in a much simpler and orthogonal way, which would of course break backward compatibility and every website. [1]: In 2020 anyway; I'm not sure on the current state; here are some of the links of my post from 2020 which like most of my posts I never finished: https://bugzilla.mozilla.org/show_bug.cgi?id=1488080 https://bugzilla.mozilla.org/show_bug.cgi?id=1488080 https://bugzilla.mozilla.org/show_bug.cgi?id=1498772 https://bugzilla.mozilla.org/show_bug.cgi?id=1498772 https://bugzilla.mozilla.org/show_bug.cgi?id=1519600 https://bugzilla.mozilla.org/show_bug.cgi?id=1519600 https://bugzilla.mozilla.org/show_bug.cgi?id=1490487 https://bugzilla.mozilla.org/show_bug.cgi?id=1490487 https://bugzilla.mozilla.org/show_bug.cgi?id=1488950 https://bugzilla.mozilla.org/show_bug.cgi?id=1488950 https://bugzilla.mozilla.org/show_bug.cgi?id=1514291 https://bugzilla.mozilla.org/show_bug.cgi?id=1514291 https://bugzilla.mozilla.org/show_bug.cgi?id=1528957 https://bugzilla.mozilla.org/show_bug.cgi?id=1528957 https://bugzilla.mozilla.org/show_bug.cgi?id=1472602 https://bugzilla.mozilla.org/show_bug.cgi?id=1472602 https://bugzilla.mozilla.org/show_bug.cgi?id=1455660 https://bugzilla.mozilla.org/show_bug.cgi?id=1455660 https://bugzilla.mozilla.org/show_bug.cgi?id=1450601 https://bugzilla.mozilla.org/show_bug.cgi?id=1450601 https://bugzilla.mozilla.org/show_bug.cgi?id=1424384 https://bugzilla.mozilla.org/show_bug.cgi?id=1424384 https://bugzilla.mozilla.org/show_bug.cgi?id=1341643 https://bugzilla.mozilla.org/show_bug.cgi?id=1341643 https://bugzilla.mozilla.org/show_bug.cgi?id=1526342 https://bugzilla.mozilla.org/show_bug.cgi?id=1526342 https://bugzilla.mozilla.org/show_bug.cgi?id=1519073 https://bugzilla.mozilla.org/show_bug.cgi?id=1519073 https://bugzilla.mozilla.org/show_bug.cgi?id=1414874 https://bugzilla.mozilla.org/show_bug.cgi?id=1414874
hanselot 4y ago
Given that someone that needs to develop a browser probably needs to hunt and peck through all that trash to find the relevant bits of information, is it not actually more damning that such a vast quantity of irrelevant cruft exists? Is this not the corporate equivalent of creating a walled garden (perhaps not the right phrase here, gastric moat sounds more apt), by exhausting the resources of all that should choose to attempt to scale this mountain of junk? That being said, I can't make any suggestions as to how you could shortcut through that other than just having decades of experience in the field.
- arp242 4y ago
  Isn't that true for any system that's been around for a few decades? Try implementing XMPP; which XEPs do you pick? It's a long list.[1] Try implementing email: there's probably more RFCs to exclude than include at this point, and what do you need and what is optional? [1]: https://xmpp.org/extensions/ https://xmpp.org/extensions/
  - jssfr 4y ago
    The Compliance Suites (also linked on the top of the page you linked) are intended to provide some guidance about what's important. The current edition of those can be found in XEP-0459: https://xmpp.org/extensions/xep-0459.html https://xmpp.org/extensions/xep-0459.html
  - im3w1l 4y ago
    This is in fact one of the big issues with xmpp. Everything is sorta-kinda compatible but not really. And email is so getting so complicated that many people are scared of running their own server let alone programming one.
- nicoburns 4y ago
  One thing the web specs do incredibly well is cross-linking. I've found it quite easy to start with a high-level spec (e.g. flexbox) and drill down into the bits I need because anywhere another spec is referenced it's linked to directly.
- deleted 4y ago

Worth noting the discussion at the link. Given that he omitted huge specs like WebGL etc. I wouldn't say it's wildly wrong. But I'd love to somehow arrive at a better estimate.

cxr 4y ago

It's not hard. Start with the WHATWG's spec, then incorporate the other specs it references using a reasonable heuristic to determine if a given item should be included or not. If you don't think the estimate from Reckless, Infinite Scope is wildly off, then you either didn't read the methodology and do a spot-check of the dataset, or you really don't understand the scope of what gets published by W3C and how little much of it has to do with Web browsers or how many revisions of them there are.

illiarian 4y ago
It may not be possible to come up with a "reasonable heuristic": https://news.ycombinator.com/item?id=35524018 https://news.ycombinator.com/item?id=35524018
- cxr 4y ago
  It is possible, and that sentence is verging on nonsense. A heuristic is not by definition perfect or optimal.
  - illiarian 4y ago
    > It is possible, and that sentence is verging on nonsense. Define "reasonable" then, when talking about the web.
    - cxr 4y ago
      The only bar that the heuristic has to pass here is "delivers a result that doesn't suck as bad as the analysis in Reckless, Infinite Scope". The analysis in that article is so bad, however, that your heuristic can literally be, "if you encounter an item that was also in Drew DeVault's input set, then assign an arbitrary probability 0.9 (or whatever) of whether the item should be counted", and it would still give you a more realistic result than what the article says (and that people are actually relying on in their arguments—and that you are defending) here. Aside from that, given how many logical errors and weird counterconclusions[1] you've managed to stuff into this discussion, though (and to have been able to do so economically[2]), I'm going to go ahead and say this is my last response to you that I spend more than 10 seconds writing out. 1. e.g. <https://news.ycombinator.com/item?id=35521704#35524952 https://news.ycombinator.com/item?id=35521704#35524952> 2. wrt number of words, fittingly

chrismorgan 4y ago

I believe it’s still wildly wrong. Had arp242 not spoken up at that time, I’d have been saying something similar, because the numbers were to me blindingly obviously extremely unrealistic. The entire HTML Standard (which is somewhat of a misnomer now, it covers much more than just HTML, quite a bit of CSS interactions, other web platform functionality, JavaScript APIs, and the likes) is now about half a million words by the most generous of counting methods (it’s written in a fairly verbose style, which is a really really really good thing when you compare it to the average IETF RFC), and I suspect it’s bigger than everything else put together, apart from ECMAScript (around 270,000 words¹ and probably growing at a faster rate than all other specs: it’s written in an even more verbose style, most of which is effectively straight code in prose form, whereas in the HTML Standard “straight code” is only a decent chunk of it). As for WebGL, the WebGL parts are actually quite little. https://registry.khronos.org/webgl/specs/latest/1.0/ https://registry.khronos.org/webgl/specs/latest/1.0/ is only about 20,000 words. I gather it defers significantly to GLES20 (PDF, 204 pages, ~60,000 words), and GLES20GLSL (PDF, 119 pages, ~30,000 words), and it has GL32CORE in its references (PDF, 404 pages, ~125,000 words), but doesn’t actually use cite it in the text and I don’t know if it’s relevant. There doesn’t look to be anything else significant that wouldn’t already be included. But really, WebGL is a fairly thin layer atop OpenGL ES 2.0, just removing some functionality and applying some restrictions. I believe you would reasonably expect a browser to use an existing OpenGL ES 2.0 implementation, so I’d be quite content to exclude the 90,000 (or perhaps it’s ~215,000?) words of that, just like it’s common to reuse an existing JavaScript engine (though you also don’t have to). Yet note this: it seems that even if we include it all (and presuming I haven’t missed anything, which I admit I could easily have done, I’m not conversant with these specs like I am with HTML/CSS/JS specs), it’s still under 0.2% of Drew’s massively-inflated figure. —⁂— ¹ Whew, https://262.ecma-international.org/ https://262.ecma-international.org/ took me several minutes to download, despite being only 7MB. Sigh; the trials of being in Australia, where things hosted in the USA are often inexplicably painfully slow—like, sub-256kbps. When already downloaded, it renders in under four seconds, which is really fairly impressive when it’s doing all that layout on a document a million pixels tall—this ain’t a PDF where you can only render one page at a time. The HTML Standard is almost two million pixels tall, and also loads completely in under four seconds—simpler styles, perhaps? I refer to it often enough that I build it locally so I don’t have to download its 13MB all the time, or compromise with the multipage version that you can’t search through as easily.

arp242 4y ago

The question is what exactly is needed for a useful and functional browser. You certainly don't need all features from Chrome, but you do need more than, say, Lynx or Dillo. Is WebGL needed? I've browsed the web for years with it disabled and have not suffered any inconvenience. I'd probably say it's not needed, but I'm a bit on the fence about it and can understand if people would disagree. All browsers implement XSLT, but is that actually needed for a functional modern browser? Maybe not? I can't remember the last time I've seen it used, but perhaps it is. And do you include HTTP? Or is that too low-level? Do you include PNG and SVG or just PNG? If you include SVG then why not PNG? There are some obvious "we need this", some obvious "we don't need this", and a lot of unclear and somewhat subjective area. I do know that you can't really say "yes there's bad data, but it probably cancels out against stuff omitted"; if anything, it only underscored my point that the list is not good. An uncurated or minimally curated document dump is not the correct approach in the first place, if you do that for SMTP you'd end up with a lot of irrelevant documents too simply because the specification is a few decades old and stuff gets superseded, some things never sees real-world implementations, things no one uses any more, etc. I started making a better list when the article was originally posted, starting from "okay, let's just check what you need for a useful browser normal people can use every day" and ended up with a few dozen things, but I never really posted it as I wasn't quite sure that was fully correct either and because I never really figured out some of the questions above. I think most of the complexity stem not just from the word count, but rather that everything interacts with everything else. Consider the relatively new "position: sticky" in CSS. Okay, great. But it doesn't work well with flexboxes, or RTL, or negative margins, or z-index, etc. etc. [1] Adding what seems like a fairly simple feature is quite complex because it interacts with so many things. It's not hard to imagine a fresh new HTML and CSS which allows all the features the current does but does so in a much simpler and orthogonal way, which would of course break backward compatibility and every website. [1]: In 2020 anyway; I'm not sure on the current state; here are some of the links of my post from 2020 which like most of my posts I never finished: https://bugzilla.mozilla.org/show_bug.cgi?id=1488080 https://bugzilla.mozilla.org/show_bug.cgi?id=1488080 https://bugzilla.mozilla.org/show_bug.cgi?id=1498772 https://bugzilla.mozilla.org/show_bug.cgi?id=1498772 https://bugzilla.mozilla.org/show_bug.cgi?id=1519600 https://bugzilla.mozilla.org/show_bug.cgi?id=1519600 https://bugzilla.mozilla.org/show_bug.cgi?id=1490487 https://bugzilla.mozilla.org/show_bug.cgi?id=1490487 https://bugzilla.mozilla.org/show_bug.cgi?id=1488950 https://bugzilla.mozilla.org/show_bug.cgi?id=1488950 https://bugzilla.mozilla.org/show_bug.cgi?id=1514291 https://bugzilla.mozilla.org/show_bug.cgi?id=1514291 https://bugzilla.mozilla.org/show_bug.cgi?id=1528957 https://bugzilla.mozilla.org/show_bug.cgi?id=1528957 https://bugzilla.mozilla.org/show_bug.cgi?id=1472602 https://bugzilla.mozilla.org/show_bug.cgi?id=1472602 https://bugzilla.mozilla.org/show_bug.cgi?id=1455660 https://bugzilla.mozilla.org/show_bug.cgi?id=1455660 https://bugzilla.mozilla.org/show_bug.cgi?id=1450601 https://bugzilla.mozilla.org/show_bug.cgi?id=1450601 https://bugzilla.mozilla.org/show_bug.cgi?id=1424384 https://bugzilla.mozilla.org/show_bug.cgi?id=1424384 https://bugzilla.mozilla.org/show_bug.cgi?id=1341643 https://bugzilla.mozilla.org/show_bug.cgi?id=1341643 https://bugzilla.mozilla.org/show_bug.cgi?id=1526342 https://bugzilla.mozilla.org/show_bug.cgi?id=1526342 https://bugzilla.mozilla.org/show_bug.cgi?id=1519073 https://bugzilla.mozilla.org/show_bug.cgi?id=1519073 https://bugzilla.mozilla.org/show_bug.cgi?id=1414874 https://bugzilla.mozilla.org/show_bug.cgi?id=1414874

hanselot 4y ago

Given that someone that needs to develop a browser probably needs to hunt and peck through all that trash to find the relevant bits of information, is it not actually more damning that such a vast quantity of irrelevant cruft exists? Is this not the corporate equivalent of creating a walled garden (perhaps not the right phrase here, gastric moat sounds more apt), by exhausting the resources of all that should choose to attempt to scale this mountain of junk? That being said, I can't make any suggestions as to how you could shortcut through that other than just having decades of experience in the field.

arp242 4y ago
Isn't that true for any system that's been around for a few decades? Try implementing XMPP; which XEPs do you pick? It's a long list.[1] Try implementing email: there's probably more RFCs to exclude than include at this point, and what do you need and what is optional? [1]: https://xmpp.org/extensions/ https://xmpp.org/extensions/
- jssfr 4y ago
  The Compliance Suites (also linked on the top of the page you linked) are intended to provide some guidance about what's important. The current edition of those can be found in XEP-0459: https://xmpp.org/extensions/xep-0459.html https://xmpp.org/extensions/xep-0459.html
- im3w1l 4y ago
  This is in fact one of the big issues with xmpp. Everything is sorta-kinda compatible but not really. And email is so getting so complicated that many people are scared of running their own server let alone programming one.
nicoburns 4y ago
One thing the web specs do incredibly well is cross-linking. I've found it quite easy to start with a high-level spec (e.g. flexbox) and drill down into the bits I need because anywhere another spec is referenced it's linked to directly.
deleted 4y ago