6 ms·
Elements of a great markup language
- tabtab 4y agoRe: A good markup language describes an abstract hierarchical structure of the document, and lets a separate program to adapt that structure to the desired output. I have to disagree. Often the abstract nature is hard to describe and/or the constructs for it either don't exist, or need cleaning/updating. Often I find myself saying, "I don't know why it's more legible to format this thing such and such way, it just is." Creating a good abstract language or category set for a given domain is not easy. It's good to use abstraction where it's practical, but often you just have to tell them system "just format it like this!" For example, the difference between a button and hyperlink is often blurry. We could abstract it something like: [Action FormatType="Button" Label="Send email to Mom" ActionType="mailto:mom73@moms.sample"/] so that FormatType could be changed to "Hyperlink", but most find this goofy and perhaps long-winded. As far as compact wiki-like shortcuts versus XML, the first often has more escaping issues or confusion. It's not a free lunch, but about best fitting intended use.
- einpoklum 4y agoSeconded. That quote may be a description of a markup language that is easy for programs to work with, not one that's easy for people to work with.
- tabtab 4y agoGood point. What's people-friendly may not be machine-friendly, and vice versa. Further, what's human writing friendly may not be human reader friendly. When writing, verbosity reduction matters most; but when reading, clarity often trumps verbosity concerns.
- abathur 4y agoIn a conversation elsewhere recently I wrote: > I think Markdown is popular because it’s a reasonably intuitive bridge between how we format handwritten documents (which is itself a blend of what makes sense on the page, and an approximation of ~spoken rhetoric) and HTML. > In some cases (like the table example), what’s trivial on the page is painful with a keyboard. The technologies that shaped the old idioms didn’t have editability as a selection pressure. You can make a few edits inline, but at some point you’ll just have to rewrite the document. > Much of the power and pain of code are byproducts of the machines forcing us to be explicit, but we didn’t have the foresight to spend centuries molding ourselves (human languages, pedagogy, rhetoric) around that kind of precision. Markdown meets us very close to where we are. The other path, I think, entails developing toolchains that do expect that precision and requiring the humans to do the changing. I guess the split between the ease of a format for reading/writing is an extension of the difference between shorthand or even cursive and print when it comes to these affordances.
- macintux 4y ago> What's people-friendly may not be machine-friendly, and vice versa. I think we’ve established that to be an axiom, otherwise we wouldn’t have 5 popular configuration formats and 10 popular markup languages.
- tannhaeuser 4y ago> As far as compact wiki-like shortcuts versus XML, the first often has more escaping issues or confusion. It isn't an either-or, and never has been. XML has been created as an SGML subset, and SGML always had short references ie. custom tokens the parser is replacing by something else in a context-dependent way. For example, an asterisk can be replaced by an <em> start-element tag to start emphasized text, and, within emphasized text, asterisks can be replaced by </em> end-element tags.
- einpoklum 4y agoSince when is LaTeX is a "lightweight markup language"? * It's not at all lightweight, I'm pretty sure it's Turing complete up the wazoo. * You can reprogram the f'ing grammar, down to the interpretation of individual input characters... * In fact, I'm not sure you can even call it a markup language. It's a macro-based programmable system on top of a typesetter.
- taeric 4y agoFor the generation of documents and papers, I'm not actually sure that avoidance of Turing complete is really a worthy goal. It can certainly go too far, but I think it is a sympathetic view that the quest for complete separation between content and presentation has given more victims to failed delivery than macros have. I do think that stability is something that needs some discussion here, for a great markup language. Not stability as in "doesn't crash", but in "doesn't change."
- stopdropnhotpot 4y agoCan you get away with ignoring Turing? sure. But it's not going to be a "great markup language" Victims? Why would this requirement cause development difficulty? If anything it would make it easier to deliver, and a lot easier to develop alternative parsers for, and easier to extend as well. We figured out all these things a long time ago but then ignored them because we thought we knew better then those who came before us.
- taeric 4y agoI'm not sure I follow. Or agree, all told. org-mode is a great markup language that lets me put in actual program snippets. I technically can make it self modifying such that it reflects on itself. May be useful for some things, but I don't know why I would go out of my way to do that. Just as I don't know why i would go out of my way to make it not possible. Similarly, I confess that plain TeX is a lot more approachable than I had always assumed it was. Even better, older TeX files still work, today. Granted, early HTML mostly still works, I think. Though, I think xhtml failed miserably. And as nice as it is that HTML still mostly works, I don't know of anyone that actually uses HTML as their authoring language, which is more than a bit of a shame.
- fiddlosopher 4y ago> Markup language which completely falls over this is Markdown. There’s no way to express generic tree structure, conversion to HTML with specific browser tags is hard-coded. This isn't really a fair criticism. True, the original Markdown.pl did not produce a generic tree structure, but that's a fact about the program, not the syntax it parses. Many Markdown and Commonmark implementations do support creation of an abstract syntax tree. Pandoc has done this for the last 17 years. It also provides nestable, generic containers as a syntax extension. > It feels like there’s a smaller, simpler language somewhere Here's my attempt: <https://djot.net https://djot.net>.
- subpixel 4y agoSee also markdoc.dev for instances where some commonality with Markdown is desired or required. Markdoc—> ast —> renderable tree —> HTML or React > we are seriously considering the possibility of drafting a specification for the JSON representation of Markdoc's Abstract Syntax Tree (AST) in order to facilitate interoperability between Markdoc tools
- AB1908 4y agoWait a minute, I recognize that GitHub handle! Are you John McFarlane? What a mind-blowing day to come across the creator of pandoc. You've saved many a student from pain. Thanks for everything.
- matklad 4y ago>It also provides nestable, generic containers as a syntax extension. That’s what I’ve meant: markdown syntax doesn’t allow for this, which requires every extension to be syntax extension. Djot’s span&div do address this problem.
- deleted 4y ago[deleted]
- rrwo 4y agoMarkdown is not markup. It's in the name. The point of markdown is to be readable by humans, using similar annotations that people have been using in ASCII text emails for decades.
- frou_dh 4y agoKudos for fighting the good fight, but honestly this is a losing battle. As time goes on, fewer and fewer of the people in the world using Markdown are interested in this distinction.
- bscphil 4y agoMarkdown is markup. > Markdown is a lightweight markup language for creating formatted text using a plain-text editor. -- Wikipedia > Markdown is a text-to-HTML conversion tool for web writers. Markdown allows you to write using an easy-to-read, easy-to-write plain text format, then convert it to structurally valid XHTML (or HTML). -- Gruber Moreover, Markdown is constrained by the fact that it doesn't have a semantics, strictly speaking. It has a syntax, and that syntax is tied to the HTML canonical form of the document, which any valid Markdown file can be transformed into. The fact that Markdown is written in extremely simple, easy to read (and brilliant IMO) syntax doesn't make it not markup.
- goto11 4y agoWhy doesn't markdown qualify as a markup language?
- xigoi 4y agoPersonally I've found that I prefer a consistent syntax with few special characters — like HTML, but less verbose — which led to the creation of xidoc: https://xidoc.nim.town/ https://xidoc.nim.town/
- emmanueloga_ 4y agoI have a very simple way of evaluating any sort of markup language. How easy is it to edit tables? https://htmlpreview.github.io/?https://github.com/jgm/djot/blob/master/doc/syntax.html#pipe-table https://htmlpreview.github.io/?https://github.com/jgm/djot/b... In that example, xidoc requires a human to manually align ascii chars to make it look pretty... That's a job that computers are good at! I would rather be able to define some sort of data block: [let "myvar;" [csv; col1,col2,col3 1,2,3 4,5,6 ]] [output-table "myvar"]; ... or something. I'm not sure right now how useful is markdown really... It was a nice experiment but I mostly use it for bullet points (*) and headings (#). I would say that's successful enough as a contribution to internet culture! :-) EDIT: seems like I commented on xidoc+djot at the same time, but the criticism is the same for both languages :-) https://djot.net/ https://djot.net/
- xigoi 4y ago> In that example, xidoc requires a human to manually align ascii chars to make it look pretty... That's a job that computers are good at! What do you mean? I'd say that's true of either every or no language, depending on if your editor has an align plugin. How does your proposed CSV-like format solve this?
- woolybully 4y agoJeffrey Kingston’s Lout is pretty good, though it’s only lightweight if LateX is. https://src.fedoraproject.org/lookaside/pkgs/lout/user-guide.pdf/710141fc615b5a113d9269e80ac17bb6/user-guide.pdf https://src.fedoraproject.org/lookaside/pkgs/lout/user-guide...
- abathur 4y agoHmm. I found myself nodding along with what this has to say about syntax design and the responsibilities of well-abstracted markup languages/converters, but I was also a little surprised by some apparent contradictions. > Great markup format unambiguously interprets an input string as an abstract tree model of a document. It doesn’t ascribe semantics to particular tag names or attributes. > Yes! > Markup language which nails this perfectly is HTML. It directly expresses this tree structure. Various viewers for HTML can then render the document in a particular fashion. HTML’s syntax itself doesn’t really care about tag names and semantics: you can imagine authoring HTML documents using an alternative set of tag names. But it's a bit weird to uphold HTML when it seems to really be valuing the underlying syntax and not the big heckin' standard that details the valid elements, attributes, and their semantics. :) > Great markup language defines the semantics of converting text to a document tree This seems to conflict with the first one I quoted. Maybe I'm missing something? Maybe the author is using semantic in different ways, here? Markup languages tend to mix different kinds of elements, and it can make thinking and communicating in this space tricky! I started trying to untangle this knot recently in https://t-ravis.com/post/doc/what_color_is_your_markup/ https://t-ravis.com/post/doc/what_color_is_your_markup/. (I also found banning "semantic" from my writing helped me untangle by making it easier to notice when I was letting it do too much work. https://t-ravis.com/post/doc/semantic_the_8_letter_s-word/ https://t-ravis.com/post/doc/semantic_the_8_letter_s-word/)
- vitiral 4y agoI recently made cxt[1] to try to solve many of the issues you mention. [1]: https://github.com/civboot/cxt https://github.com/civboot/cxt
- woolybully 4y agoWith apologies to Dr. Knuth… The most important thing in a markup language is the name. A language will not succeed without a good name. I have recently invented a very good name, and now I am looking for a suitable language.
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- chrismorgan 4y agoI would like to know the very good name you have come up with.
- bscphil 4y agoIMO this misses the most important thing needed in a great markup language. It should be readable, as a complete document, in its existing text form without any transformation. The ideal syntax would be so good that you would rarely want to transform the document in any way. I think inspiration in this area should come from an unusual place: RFC documents (like https://www.rfc-editor.org/rfc/rfc6877.txt https://www.rfc-editor.org/rfc/rfc6877.txt). These documents are fully readable in their standard text form. The only things missing are conveniences like clickable links, which could be supplied by a viewer application without transforming the text of the document. Markdown comes the closest to being directly readable, but it makes a number of sacrifices to syntax in view of its intended transformation to HTML. It's also limited in scope, patterned after ASCII email syntax. So you miss document-oriented conveniences like standardized headers, proper citations, and pages - all of which RFC documents have! That's in part the reason that a bunch of people have come behind and written extensions to Markdown. IMO someone needs to come back and gather the best of these into a consistent and simple syntax that is oriented around the needs of plain text readers.
- nooyurrsdey 4y ago> It should be readable, as a complete document, in its existing text form without any transformation. Is this _needed_, or is this nice to have? A markup language annotates text and describes _how it should be rendered_. It feels redundant to describe how a document should be rendered (presumable for final consumption) _and_ have the document be readable as-is. Case in point: I'd argue that HTML is a great markup language. I wouldn't call it the most readable in its current form. I agree with the spirit here, but it ultimately feels more "nice to have" than truly required.
- kilociraptor 4y ago>> It should be readable, as a complete document, in its existing text form without any transformation. > Is this _needed_, or is this nice to have? It's kind of the ur usecase of markup languages. Without this property WYSIWYG is significantly better.
- MilStdJunkie 4y agoA subject very near and dear to my heart. Asciidoc, flawed as it is, is still the best game in town, with conditional content, transclusion, and a particularly robust table model. Combined with S1000D architecture, it makes a passable method of writing publications for heavy industry in aerospace and defense. On the double cheap. If your team has some small amount of tech chops. The `include` is a problem, though, and the devs have known this since at least 2017. Two directives under work here. The `ainclude` is an extension that works in the subdoc direction, and `subdoc` is a directive under development that's on the milestones to go into core. Generic blocks is also on the roadmap, and it's available via extension. However. There's deeper problems, I feel, inherent in the DNA of component content systems themselves, as a basic concept. It plagues every CCMS (component content management system), whether it's DITA, S1000D, DocBook (with xincludes), ReST, or otherwise. It's not a markup problem. In S1000D, I call it "the applicability trap". To put this very briefly, you incur risk when you replace any component of natural language with a constructed language. In the case of CCMSs, you have an implied content architecture doing the work of NL in between content "chunks", but not everyone architects content, or, hell, even their product. The writers won't know this until they start, and by then it's too late. Without architecture, you have a bunch of chunks that used to be structured linguistically, and now they're not doing much of anything at all. I can't tell you how many S1000D systems I've seen built around a product that existed as little more than a twinkle in a salesman's eye. You build out an SNS and an applicability model[1], all from a bunch of emails or a spotty CAD diagram, and tell the writers to get cracking. Then, N months and N millions of dollars later, scrap everything because, lo and behold, the actual possible product completely invalidates the content architecture. "How did the doc set cost 19 million dollars?". Well, it's not the markup, it's not the vendor, and whatever miracle editor your salesmen buddies are pushing won't do jack to fix it, either. What the writers need is a quantifiable test for architecture before they start, or else, sooner or later, the Applicability Trap will get them. Either that, or write individual BDMs[2] for every single product variant that spews out of the pieholes of business development. I feel a little dumb getting worked up here. All of this crap is going to be glitter in the river come 2030 anyway, because we'll be training our pocket AIs for most of this. [1](sort of a structure of conditions to filter content) [2] Big Dumb Manuals, like mamma used to make
- 4y ago
- gizmo 4y agoThe main problem continues to be, in my view, that we don't have the equivalent of plain text for formatted documents or trees. Plain text can be copy-pasted, read by any program, edited anywhere, and that's great. Binary formats are inaccessible and hard to work with by comparison, but this is only because of mundane technical reasons. We could easily make new binary formats that would be terrific for lightly marked up text, and the editors that go along with them. Today bold, italic, tables don't fit in plain text, but carriage returns and tab characters do, as well as Halloween pumpkin emojis. There is no good reason for this, this is just where we kind of ended up. You can't put a floating point number in text, or a price, or a phone number, or date. And when you don't have standards that support this you can't have any meaningful kind of data exchange. So forget about copying a table from a spreadsheet anywhere. Everybody ends up shoehorning this necessary functionality in text editors and it won't ever work. How can computing still be at the stage where if you want a table you have to conjure up a bunch of pipe and dash characters that hopefully compiles into a table that looks presentable? And then you have to switch between "code view" and "presentation view" to check if you did it right. It would be funny if this wasn't so tragic.
- eslaught 4y agoI'm not convinced. What you're talking about sounds an awful lot like MS Word. It has a document model. It has a renderer for said document model. The document model even has first-class abstractions (Word templates). You can directly edit the rendered representation. And it has a mode to show (at least certain kinds of) markup directly in the rendered document. But the thing that all binary formats share is a lack of transparency. When I have a text file with, say, Markdown formatting, I know I'm seeing 100% of the formatting. Because the irony of not having a "code view" / "presentation view" split is that I'm never 100% sure what I'm actually looking at. Even in "regular" documents, I find that this can matter (especially when you're using Word templates and you want to do some sort of a global format change—it really matters what formatting you've applied to the document and not just what the current rendered output looks like). In documents like HTML that need to be processed in one of a variety of ways, this matters even more. I frankly cannot imagine a solution in which you do the sorts of sophisticated transformations people routinely do with HTML without caring very deeply about what "code" gets generated. If I'm not mistaken, predecessors to MS Word actually had a code mode, with a corresponding input language to control it. All of those solutions died off, at least in terms of being popular in the long term.
- djedr 4y ago> More or less, what I want from markup is to convert a text string into a document tree: enum Element { Text(String), Node { tag: String, attributes: Map<String, String> children: Vec<Element>, } } fn parse_markup(input: &str) -> Element { ... } > Markup language which nails this perfectly is HTML. The reason HTML nails it perfectly is because this is modeled after HTML. If I were to make up a markup language, I wouldn't follow that model. In particular I would get rid of attributes which to me are a restricted kind of children with a specialized syntax. This is both unnecessary and undesirable in many cases. The major problem with attributes is the <String, String> mapping. Once something is defined to be an attribute, it cannot be sensibly extended without creating an unnecessary problem. For example the `class` attribute in HTML looks like it was originally designed to hold a single class name. Then people realized that it would be desirable to have multiple classes per element. So, to keep things backwards-compatible, the value of the attribute was extended to hold a space-separated list of classes instead. Essentially creating a little DSL inside of the attribute's value. If `class` was instead a kind of child, initially limited to a single instance per element, extending to multiple instances in a backward-compatible manner would not require introducing the DSL. It would be natural. Just allow many `class` children. A valid argument in HTML in favor of attributes is conciseness. But that's an artifact of the syntax. We could make up a language with syntax for nodes as concise as HTML attributes, eliminating that argument. > It feels like there’s a smaller, simpler language somewhere Certainly. I have experimented with many different designs for such a markup language on top of Jevko[0]. One interesting design that trivially maps to HTML looks like this: h1 [Title] p [ paragraph ] p [ [paragraph with a ] a [ href=[...] [link] ] ] Another one looks like this: [h1][Title] [p][paragraph] [p][ paragraph with a [href[...] a][link] ] Both are extremely simple and minimal, but also extensible and lend themselves to writing by hand. [0] https://news.ycombinator.com/item?id=33287620 https://news.ycombinator.com/item?id=33287620
- Ygg2 4y agoAs much as I like matklad's work here I disagree. You can't have docs is AST and stuff like list element start with minuses. First requirement want to nest and organize everything, second one relies on some "natural" (highly contextual) way to do it. What looks natural for list of strings will not be natural for lists of tables of lists of maping between location snippit and some text.
- robocat 4y ago> Markup language which completely falls over this is Markdown. There’s no way to express generic tree structure Isn’t using a basic list structure for documents a feature, not a bug? For example, the article itself does not need or use any hierarchy. Surely keeping documents simple is part of the raison d'être for markdown? Hierarchical structures for documents usually need more software support, which does provide some nice features such as nested formatting, table cells containing complex elements (including other tables), and auto-numbering of headings/outlines. However a hierarchy is more fragile when text documents are edited as plain text or using text based tools. Surely a prime purpose of markdown and asciidoc is to be simple plain text. A hierarchy also seems to lead to either (a) hard formatting requirements like strict XHTML, or (b) loose requirements that cause many corner case bugs between implementations (like HTML used to be when XHTML was created).
- matklad 4y ago> For example, the article itself does not need or use any hierarchy. This two-column layout makes use of hierarchical capabilities of asciidoctor: https://matklad.github.io/2022/10/28/elements-of-a-great-markup-language.html#lists https://matklad.github.io/2022/10/28/elements-of-a-great-mar...
- robocat 4y agoHa - thanks for the correction! That example with the table on the right is sure to be relatively fragile markdown? I am considering implementing a database representation of simple documents. Hierarchical nesting (even with NoSQL) seems like a complexity worth avoiding if possible, so I do really appreciate your example. You don’t seem to use tables in your previous ten articles. The benefit versus cost of the two examples within your article are good to think about, thanks.
- hn_throwaway_99 4y agoCompletely agree. I just don't understand why the author cares about some of his "axioms": > A good markup language describes an abstract hierarchical structure of the document, and lets a separate program to adapt that structure to the desired output. Who cares? It's trivial to generate all these other formats from HTML. The thing that's great about Markdown is it is readable in plain-text source, can be styled in any number of nice HTML presentations, and then is easy to convert to things like nice, document structured PDFs. I just don't understand the problem. The author also doesn't really make any argument as to why a hierarchical document structure is so important to him in the first place. He just kinda states it as being of critical importance, but I can't see why, at all.
- remram 4y agoI think this is missing a little justification. They go from this, which I agree with: > A good markup language describes an abstract hierarchical structure of the document, and lets a separate program to adapt that structure to the desired output. > Great markup format unambiguously interprets an input string as an abstract tree model of a document. To this: > [In Markdown] there's no way to express generic tree structure, conversion to HTML with specific browser tags is hard-coded. I understand why you'd want the markup to be turned into a tree, but that's different from saying you should be able to build any tree with the markup. Isn't the point to encode text documents? The fact that you can't build the entire layout of your HTML result in the markup language doesn't seem like a sensible requirement. Why do you want or need infinitely-nested structure in documents?
- User23 4y agoSomething like parinfer is what’s needed here. Thread the needle between syntactical structures and direct manipulation thereof. We need to move past believing that a 1.5 dimensional editing interface is a good way to work with n-dimensional structures.
- intrasight 4y agoXML has all the elements. Plus attributes ;) And schemas and schema validation on top!
- encryptluks2 4y agoI like the idea of markdown but there is just too much missing with most implementations and I think the primary use case is to move away from XML due to verbosity and readability. I'd really like to see a new API to allow creators to create their own markup languages directly in the browser with native support to map the markup to the underlying renderer.
- kukkeliskuu 4y agoArticle misses one element of great markup language: to reduce complexity, there should be only one mechanism for expressing things. HTML-style attributes are unnecessary. Anything you can express with an element, you can also express with attributes. Attributes are there probably because the overall model of XML/SGML is so complex.
- goto11 4y agoAttributes exist because HTML is a markup language where text is the primary content and markup adds structure and metadata. Eliminating attributes would mean metadata should be expressed as text content, which in turn means it is not really a markup language anymore, just a generic data structure syntax like JSON or s-expressions.
- qwerty456127 4y ago> A good markup language describes an abstract hierarchical structure of the document, and lets a separate program to adapt that structure to the desired output. This is why I like FB2 and dislike ePUB.