3 ms·

Thanks for the clarifications! I wasn't aware of the Annex K amendment and the custom token facility. Would you say the token facility is actually useful for Ma

• tannhaeuser 7y ago
  > Would you say the token facility is actually useful for Markdown including advanced features such as lists or tables? For capturing complete markdown-to-HTML conversion? No. Lists and tables (tables are btw GitHub-flavored markdown but not supported by John Gruber's original Markdown.PL) aren't so much a problem as reference links where you need unbounded lookahead. Eg markdown allows links such as "[a link][linktarget]", then have a definition for linktarget such as [linktarget]: http://blabla.com "Link title" anywhere before or after the link use in the document, and requires that the link text "Link title" appears as the value of the title attribute of the anchor tag generated from the link use. That doesn't work with SGML short references which are strictly local transformations. However, SGML has the concept of public declaration text to make markdown-to-HTML transformation appear transparently "as if" it were implemented using short references when actually an integrated hardcoded markdown renderer is used. The following DTD captures this concept and tells an SGML parser (that knows about markdown by the given public identifier) to switch on markdown processing: <!DOCTYPE html [ <!ELEMENT html O O (head,body)> <!-- ... --> <!ENTITY % md_shortref_maps PUBLIC "+//IDN sgmljs.net//SHORTREF Markdown//EN"> %md_shortref_maps; ]> ... What this does is that it - virtually - pulls in shortref declarations for markdown into a parameter entity (md_shortref_maps), then references/expands that parameter entity such that it becomes part of the declaration set text. But an SGML parser is free to interpret declarations denoted by the public identifier as purely symbolic, and treat the parameter entity reference as a signal to switch on markdown processing. So this is kindof cheating, but in a way that is conformant with SGML, and retaining its declarative semantics. > Are you saying that an ordinary SGML parser will produce the expected tree for any valid HTML 5 document as specified in the HTML5 spec [...] using your DTD? In [1] (see "TALK" slides), I've reported on results for parsing w3c-tests.org/html tests: > sgmlproc [...] restricted to relevant test cases, succeeds in parsing 942 of 966, or 97.31% of the html52lib tests suite That has to be put into context, though. The test suite normatively referenced by the W3C spec is based on the older html5lib test suite (Python-based parser) and a moving target (the spec doesn't reference a particular commit); frankly, the relation W3C's spec has with html5lib tests isn't sufficiently clear. There's also a (supposedly) more relevant test suite created/maintained as part of W3C's validator.nu effort (a HTML5 parser written in Java which to my knowledge also doesn't align with a particular HTML version, just as WHATWG HTML itself isn't versioned), and which might or might not be maintained. Also, the test suite is written in such a way that tests always produce a result (because that's what HTML5 does - give a consistent meaning to any bytes that could be sent to a browser) using the procedural spec for HTML parsing given in chapter 8, whereas my DTD is based on the normative grammar rules given in chapter 4 of the spec. Finally, I've already reported several flaws in the grammar rules, etc. [1]: http://sgmljs.net/docs/sgml-html-tutorial.html http://sgmljs.net/docs/sgml-html-tutorial.html
  • dukoid 7y ago
    Thanks a lot for all the details!