5 ms·

> I'd say the W3C wasted a huge amount of time pursuing quests of purity (XHTML) over actually making the web better for users. I see the value in what they wer

> I'd say the W3C wasted a huge amount of time pursuing quests of purity (XHTML) over actually making the web better for users. I see the value in what they were trying to do, but it wasn't letting people do the things they wanted to do on the web. Okay, let’s make a deal: We both write a crawler that can fully reliably parse websites. I implement XHTML1.1, you implement HTML5. We both get 1 month time. What do you think is going to happen? XHTML was a worthy goal – with it, we wouldn’t have a need to run headless Chrome for tests. We could parse the web, and actually use the data. OpenGraph tags would never have been necessary. We wouldn’t need to throw DNNs at rendered output of a browser just to parse data. HTML5 is amazing for existing browser vendors, developers, and in the short-term, users. But everyone else loses. Horribly.

Latty 9y ago
And if we were all using XHTML1.1 now instead, yes, your parser would be easier, except all the richer content would be in flash, and all the web apps would be desktop applications, and you wouldn't be able to parse that at all, even with a full browser. You are acting like everyone would just stop and wait for you to make your dream implementation that's ideal - that's not how the world works. WHATWG was an admission that we can't stop it, so we might as well embrace it. Embracing it has resulted in browsers being far more consistent, and new features being a shared part of the web platform, and not siloed off in plugins and other platforms. That's why W3C is irrelevant now.
- kuschku 9y ago
  You’re assuming XHTML1.1 would never have evolved further, never have gotten more content. And Flash, despite its flaws, would have been a much better starting point for rich content than the ecosystem we have today. Many of the features Flash provided are only available in browsers today through babel.js transpilation. As result, we’re stuck with a language without stdlib and broken syntax. We’re stuck with a document model that’s impossible to work with or parse, and with impossible layout management. If you want web applications, it’d make much more sense to port the Android layout XML format to the web than to attempt to use HTML5 for it, because HTML5 is insanity for building applications. > and all the web apps would be desktop applications I don’t see that as anything bad. The web is for documents, and lightly interactive content. All the rich applications on the web are opaque to any crawler I could write anyway, as I just get "You need JS to view this React app". Desktop applications would be just as parseable, except they would also be less of a resource hog.
  - tptacek 9y ago
    The idea that standards groups exist to push for top-down reconcepting of how the world should work is a common one, and is also a good reason why standards groups fail. Your idea of what the best outcome is won't be the same as every other stakeholders, and no one stakeholder will have exactly the same idea of the best outcome as the market will. Ultimately, the market will win, no matter what your standards group says. The point of standards is interoperability, not rationalization. When standards groups try to rationalize technologies real people work with, they cease to provide value, and instead become obstacles that real engineers end up laboriously working around.
    - kuschku 9y ago
      When I was a child in elementary school, a standards committee decided to change my native language. They replaced the spelling of most words, and many grammatical rules. We were forced to obey these changes, any use of the old rules was counted as mistake in school. Back then, many older books were still using the old rules. By the time I left high school, almost no books with the old rules were left. All had been reprinted. All newspapers had switched. Autocorrect programs had been updated with the new rules as well. In a matter of 8 years, an entire language had changed its orthography and parts of its grammar, top-down, and it worked out fine. I’m sorry, if an entire human language with 120 million speakers can be updated top-down like that, a web spec can as well.
      - tptacek 9y ago
        That sounds horrible.
        kuschku 9y ago
        It was amazing. The new orthography is much simpler, and has far fewer insane rules or exceptions. And most people that have seen the transition, but were born after it, or went to school during it, agree. I know it can work on this scale, I’ve seen it IRL. Many languages do stuff like this, German has the council of German language, and French has their Academy. You can do the same on the web. You just need to have all vendors working together to actually do it.
tptacek 9y ago
The HTML5 person's crawler will parse some significant fraction of real websites, and the XHTML one won't, because people write HTML5 and not XHTML, even if you as a tool vendor would greatly prefer otherwise.
- kuschku 9y ago
  And yet, forcing people to implement opengraph tags, forcing people to drop flash, forcing people to use HTTPS, forcing people to drop Symantec certs, forcing people to drop SHA1 certs – so often the actors behind the WHATWG have managed to get website authors to change what they use. Hell, Google has AMP, which is far more intrusive than XHTML ever was, and yet, they’ve managed to get every major website to implement it. https://www.ampproject.org/docs/troubleshooting/validation_errors https://www.ampproject.org/docs/troubleshooting/validation_e... And yet, somehow, implementing some stricter spec is supposedly impossible?
  - bjelli 9y ago
    There' s a Big difference between "every major website" and "the Web". "every major website" means 100 companies with skilled developer who can and will react to changes in browsers quickly. "The web" consists of millions of websites maintained by individuals and small organisations who have no resources to update the way their web pages are coded every year. It contains HTML generated 10, 20, soon 30 years ago. It contains that one app in your intranet with the table layout that you can't replace and that IOT thing you connected to your home Wi-Fi 7 years ago that has no way of upgrading its web interface. A browser that looses access to "the web" is worst than useless.
    - bonesss 9y ago
      There's also a federal procurement picture: big governments making big purchases aren't fans of incompatibilities and standardised solutions. For a company like MS losing access to "the web" could keep a lot of people from becoming VPs...
pvg 9y ago
HTML5 is amazing for existing browser vendors, developers, and in the short-term, users. But everyone else loses. Horribly. I don't understand who the 'everybody else' is in this case and what and when their horrible losing will be.
- kuschku 9y ago
  People trying to build new tools that parse the web. Try building a crawler without reusing an existing browser engine. Try running unit tests against your own web projects with Selenium without running a headless browser. Phantom.JS gave up because they couldn’t keep up with the complexity, and Chrome headless "just works". Opera gave up on their own browser engine because of the complexity of parsing HTML5 accurately, when the spec is just "whatever Chrome does". We’ve thrown away an entire ecosystem, just for more flashy graphics.
  - pvg 9y ago
    That's a pretty tiny 'everybody else' compared to users, web and browser developers. They seem to be doing mostly ok and I still don't follow your argument that their concerns should somehow reign supreme over those of, you know, the actual everybody else.
    - kuschku 9y ago
      It’s a tiny "everybody else" because it never was given a chance to develop. Maybe you’d also say that the number of people that want to do their online banking with a desktop program that isn’t provided by their bank is a tiny "everybody else". Yet in places where OpenHBCI exists, many people use it – and there is an ecosystem around it, e.g. KMyMoney can integrate with it, and there’s a small widget for KDE to show your current account balance in your tray. If we had a machine parseable web, a similar ecosystem would have developed. When embedding links today, we use OpenGraph tags. But why? If the web was directly machine readable, we could’ve directly embedded that content. Google search shows you as preview excerpts of tables on a page, with an easily machine readable web, similar stuff could’ve happened here. Maybe instead of only embedding YouTube videos, I would have been able to easily embed any part of any web page into any other. Maybe I would have been able to easily embed any part of any web page in a desktop application. Maybe I would have been able to build addons that easily search over content of web pages, in a structured way.
olavk 9y ago
The HTML5 crawler would be able to crawl the web, the XHTML crawler would be able to crawl 0.0001% of the web. XHTML would only improve the web if all existing HTML went away or was changed to XHTML. Since this is never going to happen, XHTML does not simplify anything.
blablabla123 9y ago
IMHO XHTML is pretty painful. Even if you say: okay, there is a server-side auto-generated markup tree and we can formally verify what happens. There is now a solution to that and it's called (server-side) React which is basically a (useful) alternative to XSLT. Except that it outputs HTML5. Even if you argue that the specs are huge. Just compare book sizes about XML, XSLT vs HTML5 and CSS, JS, React. When I actually tried to do some useful work with XSLT (which needs to be mentioned here IMHO), I realized that the - less painful - 2.0 version is hardly implemented by anyone. Regarding the parsing: X(HT)ML lexing is ridiculously easy, for HTML5 it's slightly more difficult but not tough at all. You just need to keep a list of closing vs self-closing tags. Not talking about building in fault-tolerance, that would be tough, yes, even tougher for XML! > XHTML was a worthy goal – with it, we wouldn’t have a need to run headless > Chrome for tests. We could parse the web, and actually use the data. > OpenGraph tags would never have been necessary. We wouldn’t need to throw > DNNs at rendered output of a browser just to parse data. Yes and no. If you use CSS for styling, the answer is no. If you require JS to show the initial show/page, the answer is no as well. But yeah, if you use the whole XML machinery with XSLT and possibly even XPATH, then you would be kind of right. I mean, as long as we properly handle the schemas and dtd's - which almost no parser does AFAIK. So it's true, one can do pretty bad-ass stuff with all the X*. But tooling and library support is not good and has never been. XSLT 1.x is insanely difficult to use and XSLT 2.x hard to fully implement I guess.
fjsolwmv 9y ago
In the long run, we're all dead. We need short term solutions.
lambda 9y ago
> I implement XHTML1.1, you implement HTML5. We both get 1 month time. Do you get to use libraries or not? If you get to use libraries, there are plenty of libraries for both of these tasks, so it will be fairly equivalent. In fact, if you want to really support XHTML well, it will probably be more complex, because you have to take into account namespaces; a tag or attribute is not just a simple string, you have to consider the namespace it's in, so you would have to deal with that. If you have to write it from scratch, I'd actually also bet on it being quicker for HTML5. The exact algorithm is specified in the standard; you just have to translate that from pseudocode into whatever language you're working in. In XHTML, you have to look through several standards (XHTML 1.0, XHTML Modularization, XML 1.0 which includes the DTD, XML Namespaces, XML Schema), and then translate from the specification in a declarative style into a algorithm that can actually be used to parse the document. XML parsing is actually quite complicated. > XHTML was a worthy goal – with it, we wouldn’t have a need to run headless Chrome for tests. We could parse the web, and actually use the data. What? XHTML wouldn't have replaced the use of JavaScript to add features and load content. It wouldn't have replaced people's inappropriate use of tags. It wouldn't have replaced the fact that web pages are written for human consumption, and so don't generally try to include appropriate annotations on data for processing by other tools. > HTML5 is amazing for existing browser vendors, developers, and in the short-term, users. But everyone else loses. Horribly. Imagining that some magical standard is going to make everything better for users is a pipe dream. Providing good data formats and APIs for exporting data is hard, and is a different problem that providing user interfaces and interactivity. There's not some way in which XHTML could have been extended to serve both purposes; they are just too different. This is why, when people care about providing programmatic access to data, they generally provide two endpoints; one serving HTML, CSS, and JavaScript for humans to interact with, and one for providing JSON for machines to parse. There have been endless attempts to try and make one standard that would work for both purposes, and they've failed because that's just not a good approach for solving the problem; instead, it's better to just have an API that both the UI code (whether on the client or server side) can use, and other developers can use if you want to expose it.
ttepasse 9y ago
> OpenGraph tags would never have been necessary. Small correction: XHTML2 had the Metainformation Attributes Module [1]. That then became RDFa in (X)HTML, practically the same syntax and processing model. Facebooks Open Graph stuff is claimed to be RDFa. When I tested it then their Parser did not really do RDFa processing - other CURIE prefixes for the same URI weren't recognized, if I remember correctly. But in effect Open Graph meta information would have looked the same in XHTML2 as in todays WHATTF HTML. For your other argument I agree. WHATWG (and dumb modern style of development) reduced the democratising aspect of the web. But of course the people of WHATWG word for billion dollar companies, which want to have a moat to centralize behind. [1] https://www.w3.org/TR/xhtml2/mod-metaAttributes.html https://www.w3.org/TR/xhtml2/mod-metaAttributes.html

Latty 9y ago

And if we were all using XHTML1.1 now instead, yes, your parser would be easier, except all the richer content would be in flash, and all the web apps would be desktop applications, and you wouldn't be able to parse that at all, even with a full browser. You are acting like everyone would just stop and wait for you to make your dream implementation that's ideal - that's not how the world works. WHATWG was an admission that we can't stop it, so we might as well embrace it. Embracing it has resulted in browsers being far more consistent, and new features being a shared part of the web platform, and not siloed off in plugins and other platforms. That's why W3C is irrelevant now.

kuschku 9y ago
You’re assuming XHTML1.1 would never have evolved further, never have gotten more content. And Flash, despite its flaws, would have been a much better starting point for rich content than the ecosystem we have today. Many of the features Flash provided are only available in browsers today through babel.js transpilation. As result, we’re stuck with a language without stdlib and broken syntax. We’re stuck with a document model that’s impossible to work with or parse, and with impossible layout management. If you want web applications, it’d make much more sense to port the Android layout XML format to the web than to attempt to use HTML5 for it, because HTML5 is insanity for building applications. > and all the web apps would be desktop applications I don’t see that as anything bad. The web is for documents, and lightly interactive content. All the rich applications on the web are opaque to any crawler I could write anyway, as I just get "You need JS to view this React app". Desktop applications would be just as parseable, except they would also be less of a resource hog.
- tptacek 9y ago
  The idea that standards groups exist to push for top-down reconcepting of how the world should work is a common one, and is also a good reason why standards groups fail. Your idea of what the best outcome is won't be the same as every other stakeholders, and no one stakeholder will have exactly the same idea of the best outcome as the market will. Ultimately, the market will win, no matter what your standards group says. The point of standards is interoperability, not rationalization. When standards groups try to rationalize technologies real people work with, they cease to provide value, and instead become obstacles that real engineers end up laboriously working around.
  - kuschku 9y ago
    When I was a child in elementary school, a standards committee decided to change my native language. They replaced the spelling of most words, and many grammatical rules. We were forced to obey these changes, any use of the old rules was counted as mistake in school. Back then, many older books were still using the old rules. By the time I left high school, almost no books with the old rules were left. All had been reprinted. All newspapers had switched. Autocorrect programs had been updated with the new rules as well. In a matter of 8 years, an entire language had changed its orthography and parts of its grammar, top-down, and it worked out fine. I’m sorry, if an entire human language with 120 million speakers can be updated top-down like that, a web spec can as well.
    - tptacek 9y ago
      That sounds horrible.
      - kuschku 9y ago
        It was amazing. The new orthography is much simpler, and has far fewer insane rules or exceptions. And most people that have seen the transition, but were born after it, or went to school during it, agree. I know it can work on this scale, I’ve seen it IRL. Many languages do stuff like this, German has the council of German language, and French has their Academy. You can do the same on the web. You just need to have all vendors working together to actually do it.

tptacek 9y ago

The HTML5 person's crawler will parse some significant fraction of real websites, and the XHTML one won't, because people write HTML5 and not XHTML, even if you as a tool vendor would greatly prefer otherwise.

kuschku 9y ago
And yet, forcing people to implement opengraph tags, forcing people to drop flash, forcing people to use HTTPS, forcing people to drop Symantec certs, forcing people to drop SHA1 certs – so often the actors behind the WHATWG have managed to get website authors to change what they use. Hell, Google has AMP, which is far more intrusive than XHTML ever was, and yet, they’ve managed to get every major website to implement it. https://www.ampproject.org/docs/troubleshooting/validation_errors https://www.ampproject.org/docs/troubleshooting/validation_e... And yet, somehow, implementing some stricter spec is supposedly impossible?
- bjelli 9y ago
  There' s a Big difference between "every major website" and "the Web". "every major website" means 100 companies with skilled developer who can and will react to changes in browsers quickly. "The web" consists of millions of websites maintained by individuals and small organisations who have no resources to update the way their web pages are coded every year. It contains HTML generated 10, 20, soon 30 years ago. It contains that one app in your intranet with the table layout that you can't replace and that IOT thing you connected to your home Wi-Fi 7 years ago that has no way of upgrading its web interface. A browser that looses access to "the web" is worst than useless.
  - bonesss 9y ago
    There's also a federal procurement picture: big governments making big purchases aren't fans of incompatibilities and standardised solutions. For a company like MS losing access to "the web" could keep a lot of people from becoming VPs...

pvg 9y ago

HTML5 is amazing for existing browser vendors, developers, and in the short-term, users. But everyone else loses. Horribly. I don't understand who the 'everybody else' is in this case and what and when their horrible losing will be.

kuschku 9y ago
People trying to build new tools that parse the web. Try building a crawler without reusing an existing browser engine. Try running unit tests against your own web projects with Selenium without running a headless browser. Phantom.JS gave up because they couldn’t keep up with the complexity, and Chrome headless "just works". Opera gave up on their own browser engine because of the complexity of parsing HTML5 accurately, when the spec is just "whatever Chrome does". We’ve thrown away an entire ecosystem, just for more flashy graphics.
- pvg 9y ago
  That's a pretty tiny 'everybody else' compared to users, web and browser developers. They seem to be doing mostly ok and I still don't follow your argument that their concerns should somehow reign supreme over those of, you know, the actual everybody else.
  - kuschku 9y ago
    It’s a tiny "everybody else" because it never was given a chance to develop. Maybe you’d also say that the number of people that want to do their online banking with a desktop program that isn’t provided by their bank is a tiny "everybody else". Yet in places where OpenHBCI exists, many people use it – and there is an ecosystem around it, e.g. KMyMoney can integrate with it, and there’s a small widget for KDE to show your current account balance in your tray. If we had a machine parseable web, a similar ecosystem would have developed. When embedding links today, we use OpenGraph tags. But why? If the web was directly machine readable, we could’ve directly embedded that content. Google search shows you as preview excerpts of tables on a page, with an easily machine readable web, similar stuff could’ve happened here. Maybe instead of only embedding YouTube videos, I would have been able to easily embed any part of any web page into any other. Maybe I would have been able to easily embed any part of any web page in a desktop application. Maybe I would have been able to build addons that easily search over content of web pages, in a structured way.

olavk 9y ago

The HTML5 crawler would be able to crawl the web, the XHTML crawler would be able to crawl 0.0001% of the web. XHTML would only improve the web if all existing HTML went away or was changed to XHTML. Since this is never going to happen, XHTML does not simplify anything.

blablabla123 9y ago

IMHO XHTML is pretty painful. Even if you say: okay, there is a server-side auto-generated markup tree and we can formally verify what happens. There is now a solution to that and it's called (server-side) React which is basically a (useful) alternative to XSLT. Except that it outputs HTML5. Even if you argue that the specs are huge. Just compare book sizes about XML, XSLT vs HTML5 and CSS, JS, React. When I actually tried to do some useful work with XSLT (which needs to be mentioned here IMHO), I realized that the - less painful - 2.0 version is hardly implemented by anyone. Regarding the parsing: X(HT)ML lexing is ridiculously easy, for HTML5 it's slightly more difficult but not tough at all. You just need to keep a list of closing vs self-closing tags. Not talking about building in fault-tolerance, that would be tough, yes, even tougher for XML! > XHTML was a worthy goal – with it, we wouldn’t have a need to run headless > Chrome for tests. We could parse the web, and actually use the data. > OpenGraph tags would never have been necessary. We wouldn’t need to throw > DNNs at rendered output of a browser just to parse data. Yes and no. If you use CSS for styling, the answer is no. If you require JS to show the initial show/page, the answer is no as well. But yeah, if you use the whole XML machinery with XSLT and possibly even XPATH, then you would be kind of right. I mean, as long as we properly handle the schemas and dtd's - which almost no parser does AFAIK. So it's true, one can do pretty bad-ass stuff with all the X*. But tooling and library support is not good and has never been. XSLT 1.x is insanely difficult to use and XSLT 2.x hard to fully implement I guess.

fjsolwmv 9y ago

In the long run, we're all dead. We need short term solutions.

lambda 9y ago

> I implement XHTML1.1, you implement HTML5. We both get 1 month time. Do you get to use libraries or not? If you get to use libraries, there are plenty of libraries for both of these tasks, so it will be fairly equivalent. In fact, if you want to really support XHTML well, it will probably be more complex, because you have to take into account namespaces; a tag or attribute is not just a simple string, you have to consider the namespace it's in, so you would have to deal with that. If you have to write it from scratch, I'd actually also bet on it being quicker for HTML5. The exact algorithm is specified in the standard; you just have to translate that from pseudocode into whatever language you're working in. In XHTML, you have to look through several standards (XHTML 1.0, XHTML Modularization, XML 1.0 which includes the DTD, XML Namespaces, XML Schema), and then translate from the specification in a declarative style into a algorithm that can actually be used to parse the document. XML parsing is actually quite complicated. > XHTML was a worthy goal – with it, we wouldn’t have a need to run headless Chrome for tests. We could parse the web, and actually use the data. What? XHTML wouldn't have replaced the use of JavaScript to add features and load content. It wouldn't have replaced people's inappropriate use of tags. It wouldn't have replaced the fact that web pages are written for human consumption, and so don't generally try to include appropriate annotations on data for processing by other tools. > HTML5 is amazing for existing browser vendors, developers, and in the short-term, users. But everyone else loses. Horribly. Imagining that some magical standard is going to make everything better for users is a pipe dream. Providing good data formats and APIs for exporting data is hard, and is a different problem that providing user interfaces and interactivity. There's not some way in which XHTML could have been extended to serve both purposes; they are just too different. This is why, when people care about providing programmatic access to data, they generally provide two endpoints; one serving HTML, CSS, and JavaScript for humans to interact with, and one for providing JSON for machines to parse. There have been endless attempts to try and make one standard that would work for both purposes, and they've failed because that's just not a good approach for solving the problem; instead, it's better to just have an API that both the UI code (whether on the client or server side) can use, and other developers can use if you want to expose it.

ttepasse 9y ago

> OpenGraph tags would never have been necessary. Small correction: XHTML2 had the Metainformation Attributes Module [1]. That then became RDFa in (X)HTML, practically the same syntax and processing model. Facebooks Open Graph stuff is claimed to be RDFa. When I tested it then their Parser did not really do RDFa processing - other CURIE prefixes for the same URI weren't recognized, if I remember correctly. But in effect Open Graph meta information would have looked the same in XHTML2 as in todays WHATTF HTML. For your other argument I agree. WHATWG (and dumb modern style of development) reduced the democratising aspect of the web. But of course the people of WHATWG word for billion dollar companies, which want to have a moat to centralize behind. [1] https://www.w3.org/TR/xhtml2/mod-metaAttributes.html https://www.w3.org/TR/xhtml2/mod-metaAttributes.html