5 ms·

> Beyond SGML, what's really puzzling to me is that we keep on throwing mud against a wall to see what sticks, and every comp.sci. generation seem to reinvent e

by ekidd 4y ago

> Beyond SGML, what's really puzzling to me is that we keep on throwing mud against a wall to see what sticks, and every comp.sci. generation seem to reinvent everything from scratch without consideration what came before. I'm old enough to remember the height of SGML's popularity. I used to work in the office next to someone who was writing an SGML parser. He was an incredibly skilled developer; he was also the editor of the spec for a major programming language. And he loathed SGML. The SGML standard was a $99 doorstop of a book. Parsing SGML (and DTDs, and capability declarations) was basically impossible using standard parsing theory. Instead, you needed hundreds of hand-built parsing routines. And even when you parsed some SGML, you were left with the frustrating fact that there was no standard "document information set" (if I recall the name correctly) that told you what your parser should return. One of our favorite lunch-time rants was "Someone should define a stripped-down version of SGML that doesn't require a 2.3 MB C++ parsing library (because our users only have 8MB of RAM, and they're using 12 MB of that to run Netscape and an OS)." Then one day, my boss walks in and says, "Hey, our corporate W3 representative just got a copy of this. You'll like it." And he dropped an early draft of XML on my desk. The spec was 30 pages long, and easy to implement. Within a month, pretty much everyone who could abandon SGML had done so. A few legacy vendors continued supplying enterprise SGML users, but every other SGML developer pivoted to the web and XML. It was a chance to trade frustrating niche tools for a hot web market. SGML occupied a weird middle space: It was not only programmer hostile, but its creator bragged about how it was programmer hostile, because users came first. Except SGML was still far too technical for any mainstream users, who normally wanted GUI tools. SGML was not mysteriously forgotten. It was buried at a crossroads with a stake through its heart. And plenty of programmers who worked with SGML volunteered to shovel.

tannhaeuser 4y ago
> SGML was not mysteriously forgotten. It was buried at a crossroads with a stake through its heart. > SGML developer pivoted to the web and XML Appreciate your response, but with respect, that's the kind of hear-say criticism I was almost expecting. First, SGML hasn't gone anywhere. XML is just a proper subset of SGML, written by the same people who developed SGML and updated SGML to allow DTD-less markup aka XML. Next, XML just didn't make it to the web, wich was its entire raison d'etre, as in, chapter one, sentence one of the XML spec. It's true that XML has been used for almost everything and anything except its actual purpose (eg serialization of service payloads when those are consumed by programs rather than pasted into markup, or config file formats). Finally, SGML is the only game in town able to parse HTML based on an international standard. Don't we want a perspective to get us out of the dying web, or at least preserve what's worth preserving of the extant web? Or are you ok with monopolists having captured the web into craptastic browser tech? Now that's something overly complex and user-hostile, and where 12MB doesn't get you very far indeed. SGML complexity is a myth. You can code a parser in about one man-year, with optional features (SHORTREF, LINK, web bindings) taking another half (source: have done it [1]). I will concede that working off the SGML spec alone is ... laborious, and studying it is indeed a non-insignificant part of the work. The spec is only about the markup meta-language as entered by an author, and doesn't say anything about APIs etc (except for ESIS used for test case suites and early Perl-based processors). Closest to a "document information set" would be HyTime "groves" and, later, DOM. Missing from the spec is a formal model for tag inference and for deciding ambiguousness of content models. That void was quickly filled by Brüggemann-Kleins "One-unambiguous grammars" paper ca 1993, plus clarifications by James Clark (author of the SP SGML parser/toolbox [2]). XML Schema appeals to the exact same theory in it's "Unique particle attribution" constraint btw. Now don't get me wrong: there is valid criticism for SGML, and maybe a need for a new subset and/or extension (and I might have ideas to share). Just criticism with a broad brush and false dichotomies doesn't get us anywhere. [1]: http://sgmljs.net http://sgmljs.net [2]: http://www.jclark.com/sp/ http://www.jclark.com/sp/
- treve 4y ago
  > SGML complexity is a myth. You can code a parser in about one man-year This gave me a bit of a chuckle.
- chrismorgan 4y ago
  > SGML complexity is a myth. You can code a parser in about one man-year, with optional features (SHORTREF, LINK, web bindings) taking another half (source: have done it [1]). You’re waaaay off base if you think these figures make SGML complexity a myth. https://bundlephobia.com/package/sgml@0.2.1-beta https://bundlephobia.com/package/sgml@0.2.1-beta shows sgml.js to be about 400KB of minified code, and the bundle also contains sgml-ua.min.js which is 400KB but I think a different 400KB, and a simple line count shows sgml-ua.js and everything-else to each be around 18,000 lines excluding comments. You can implement XML (https://www.w3.org/TR/xml/ https://www.w3.org/TR/xml/) minus doctypes in a couple of hours, no trouble. You can add doctypes (including remote fetching, assuming an HTTP library) and validation in less than the remainder of a man-day. Be generous about doing a bang-up job of it all, and you’ll still easily be done in under a week. (And in a language like JavaScript, it’d probably be a couple of thousand lines at the most, depending of course on how you choose to model it all; possibly under 500.) SGML is very complex compared with XML. SGML has departed like Jehoram of old: to no one’s regret.
  - tannhaeuser 4y ago
    sgmljs.net also bundles a markdown parser, a DTD for W3C HTML 5.2 (a really long string), SGML declaration support, loads of diagnostic messages, an URL parser, SGML LINK with pipelining and templating, implementations for a couple of (HyTime general arch) formal system identifiers such as for date and CSV parsing/formatting, plus a couple other things. It has essentially no external dependencies. The XML subset of SGML is simpler (that's the entire point of it) but implementing DOCTYPEs with the subset of markup declarations supported by XML still will take you much more than "the remainder of a man-day" LOL. Yes unlike with SGML, you can use an off-the-shelf regexp lib to perform actual checking, but you still need to perform model admissibility/ambiguousness checking with halfway usable diagnostics, and require hundreds of test cases. XML might be ok as a markup delivery format (that didn't make it on the web), but it absolutely doesn't begin to meet the bar of an authoring format. If it were, nobody would be using markdown, MediaWiki syntax, or other custom Wiki syntax, and your collection of helper apps and CMSs, template engines, and whatnot. All of which is included in SGML. (Still, I upvoted your comment FWIW)
mananaysiempre 4y ago
XML isn’t hard to parse, sure, but you still have to do parsed entities and default attribute values for the (mandatory) “internal DTD subset”, don’t you? If you omit that, then sure, the rest[1] is basically in the ballpark of JSON. But for a full parser you still have to do a full textual macro system so you have to allocate given untrusted input so ... so now you’re basically doing a good chunk of a simple compiler frontend. (And the macro system it can’t even do namespaces because SGML.) Kind of annoying for a feature that basically nothing except MathML uses. [1] https://dev.yorhel.nl/yxml https://dev.yorhel.nl/yxml

by ekidd 4y ago

tannhaeuser 4y ago

> SGML was not mysteriously forgotten. It was buried at a crossroads with a stake through its heart. > SGML developer pivoted to the web and XML Appreciate your response, but with respect, that's the kind of hear-say criticism I was almost expecting. First, SGML hasn't gone anywhere. XML is just a proper subset of SGML, written by the same people who developed SGML and updated SGML to allow DTD-less markup aka XML. Next, XML just didn't make it to the web, wich was its entire raison d'etre, as in, chapter one, sentence one of the XML spec. It's true that XML has been used for almost everything and anything except its actual purpose (eg serialization of service payloads when those are consumed by programs rather than pasted into markup, or config file formats). Finally, SGML is the only game in town able to parse HTML based on an international standard. Don't we want a perspective to get us out of the dying web, or at least preserve what's worth preserving of the extant web? Or are you ok with monopolists having captured the web into craptastic browser tech? Now that's something overly complex and user-hostile, and where 12MB doesn't get you very far indeed. SGML complexity is a myth. You can code a parser in about one man-year, with optional features (SHORTREF, LINK, web bindings) taking another half (source: have done it [1]). I will concede that working off the SGML spec alone is ... laborious, and studying it is indeed a non-insignificant part of the work. The spec is only about the markup meta-language as entered by an author, and doesn't say anything about APIs etc (except for ESIS used for test case suites and early Perl-based processors). Closest to a "document information set" would be HyTime "groves" and, later, DOM. Missing from the spec is a formal model for tag inference and for deciding ambiguousness of content models. That void was quickly filled by Brüggemann-Kleins "One-unambiguous grammars" paper ca 1993, plus clarifications by James Clark (author of the SP SGML parser/toolbox [2]). XML Schema appeals to the exact same theory in it's "Unique particle attribution" constraint btw. Now don't get me wrong: there is valid criticism for SGML, and maybe a need for a new subset and/or extension (and I might have ideas to share). Just criticism with a broad brush and false dichotomies doesn't get us anywhere. [1]: http://sgmljs.net http://sgmljs.net [2]: http://www.jclark.com/sp/ http://www.jclark.com/sp/

treve 4y ago
> SGML complexity is a myth. You can code a parser in about one man-year This gave me a bit of a chuckle.
chrismorgan 4y ago
> SGML complexity is a myth. You can code a parser in about one man-year, with optional features (SHORTREF, LINK, web bindings) taking another half (source: have done it [1]). You’re waaaay off base if you think these figures make SGML complexity a myth. https://bundlephobia.com/package/sgml@0.2.1-beta https://bundlephobia.com/package/sgml@0.2.1-beta shows sgml.js to be about 400KB of minified code, and the bundle also contains sgml-ua.min.js which is 400KB but I think a different 400KB, and a simple line count shows sgml-ua.js and everything-else to each be around 18,000 lines excluding comments. You can implement XML (https://www.w3.org/TR/xml/ https://www.w3.org/TR/xml/) minus doctypes in a couple of hours, no trouble. You can add doctypes (including remote fetching, assuming an HTTP library) and validation in less than the remainder of a man-day. Be generous about doing a bang-up job of it all, and you’ll still easily be done in under a week. (And in a language like JavaScript, it’d probably be a couple of thousand lines at the most, depending of course on how you choose to model it all; possibly under 500.) SGML is very complex compared with XML. SGML has departed like Jehoram of old: to no one’s regret.
- tannhaeuser 4y ago
  sgmljs.net also bundles a markdown parser, a DTD for W3C HTML 5.2 (a really long string), SGML declaration support, loads of diagnostic messages, an URL parser, SGML LINK with pipelining and templating, implementations for a couple of (HyTime general arch) formal system identifiers such as for date and CSV parsing/formatting, plus a couple other things. It has essentially no external dependencies. The XML subset of SGML is simpler (that's the entire point of it) but implementing DOCTYPEs with the subset of markup declarations supported by XML still will take you much more than "the remainder of a man-day" LOL. Yes unlike with SGML, you can use an off-the-shelf regexp lib to perform actual checking, but you still need to perform model admissibility/ambiguousness checking with halfway usable diagnostics, and require hundreds of test cases. XML might be ok as a markup delivery format (that didn't make it on the web), but it absolutely doesn't begin to meet the bar of an authoring format. If it were, nobody would be using markdown, MediaWiki syntax, or other custom Wiki syntax, and your collection of helper apps and CMSs, template engines, and whatnot. All of which is included in SGML. (Still, I upvoted your comment FWIW)

mananaysiempre 4y ago

XML isn’t hard to parse, sure, but you still have to do parsed entities and default attribute values for the (mandatory) “internal DTD subset”, don’t you? If you omit that, then sure, the rest[1] is basically in the ballpark of JSON. But for a full parser you still have to do a full textual macro system so you have to allocate given untrusted input so ... so now you’re basically doing a good chunk of a simple compiler frontend. (And the macro system it can’t even do namespaces because SGML.) Kind of annoying for a feature that basically nothing except MathML uses. [1] https://dev.yorhel.nl/yxml https://dev.yorhel.nl/yxml