5 ms·

After considering two or three dozens of different "lightweight" markup languages (i.e. not flavors) and designing a few (plus at least three failed attempts to

After considering two or three dozens of different "lightweight" markup languages (i.e. not flavors) and designing a few (plus at least three failed attempts to improve Markdown), I'm now almost convinced that Scribble [1] or Perl POD [2] have one of the best possible designs. Not that they are optimal, but they are definitely heading at the correct direction. They are extremely generic: every Scribble syntax is prefixed `@`, every POD construct is either `letter<...>` or `=word`. And yet they are surprisingly unobstructive and readable enough. Compare with BBCode, which is also pretty generic but too annoying. They are readily extensible and need no special one-off syntaxes for most things. Yes, using asterisks or underscores for emphasis is handy, but how about strike-throughs? Underlines? Non-emphasis italics (e.g. species name)? Language tags? Abbreviations? Do they nest? Do they work in any context? Why not adding some harmless options to the existing constructs? Syntax extensibility should be one of the foremost concerns of markup language design and they got it right. Edit: I should stress that these concerns are only valid for writings. For casual comments, less is better (ideally no-follow links and verbatim texts should suffice). [1] https://docs.racket-lang.org/scribble/ https://docs.racket-lang.org/scribble/ [2] https://perldoc.perl.org/perlpod.html https://perldoc.perl.org/perlpod.html

perlgeek 8y ago
I'm a big fan of Perl myself, but when writing POD, I occasionally stumble about the C<...> for inline code. The problem is that you have to take care that there is no > in the inline code, and if there is, you have to switch to C<< ... >>. With Markdown's `...`, this happened far less often to me.
- lifthrasiir 8y ago
 You are right that it is suboptimal. (I do think that Scribble's choice of braces is not too bad in this regard.) On the other hand, I can't readily type backquotes in my mobile phone or recent macOS [1] so there is a trade-off :-) [1] Annoyingly enough, macOS Sierra and later maps a backquote to a Won sign in Korean keyboard layouts.
 - elcritch 8y ago
 Never read scribble before but I kinda like it. It reminds me of the good parts of latex. As in @action vs \action. Though I don’t like the table syntax as much from a WYSIWYG level (E.g. it doesn’t look table-ly) but for complex tables it don’t matter much. Can you readily extend scribble with Racket and how’s the install/usage story?
 - lifthrasiir 8y ago
 I think it is in the default Racket install. And yes, that's a problem; it is actually a Racket code behind the scene so it is fundamentally tied to Racket (that can be good or bad depending on your use case, and some strength points can be attributed to that).
 - Eugeleo 8y ago
 I know we're talking about docs right now, which is where Scribble really shines. And when it comes to blogging/prose, I'd recommend Pollen[1], which is a writing system based on scribble. It is fairly lightweight and extensible with Racket (which is by the way actually a pretty awesome language). I'm writing my college notes in it and couldn't be happier. The dev is very responsive on mailing lists and would gladly help you out if you have some trouble initially. There is a pretty thorough getting started guide on the linked page. There's a few blogs written in Pollen and even a few books (Ever heard about Flatland? Or Practical Typography?). [1] http://pollenpub.com http://pollenpub.com
astrobe_ 8y ago
Reading the comments here, it feels like people miss the whole point of markdown. Markdown's raison d'être is, "if you follow these conventions when doing ASCII art formatting, the my program can transform this to HTML or whatever other format". The conventions being mostly the ones used in the pre-HTML email era, like underlining headings with a bunch of ======. Orgmode is completely different to that. Orgmode is an "outliner/GTD" at heart, but this aspect has been buried under a ton of features. Then if you want "prawfishunal qvalitat" advanced formatting, typesetting and layout control what you really want is something like Latex. Yet another whole different thing.
- lifthrasiir 8y ago
  But Markdown was advertised as "a text-to-HTML conversion tool for web writers" [1]. The raison d'être as you stated is not an end goal but a possible benefit. Gruber may have thought that the extensibility of Markdown is done by raw HTML; now we know that's not enough. [1] https://daringfireball.net/projects/markdown/ https://daringfireball.net/projects/markdown/
- hyperion2010 8y ago
  The thing about org mode is that it has everything you need to do the full fancy latex typesetting without loosing the ability to export to other formats the way you do if you write raw latex. Of course there are times when you have no choice but to duplicate a block if you are doing something really fancy, but that is a fairly rare occurrence.
- omaranto 8y ago
  Of course Org Mode is completely different, because it can do tons of other stuff. But among it's many features, it also has lightweight markup syntax about which you could claim "if you follow these conventions when doing ASCII art formatting, the my program can transform this to HTML or whatever other format". And the conventions in Org Mode are also email-like in the sense that they don't look out of place in something meant to be plain text. (Maybe finding some large email corpus could answer which syntax is more email-like between Org Mode and Markdown, and maybe Markdown wins on that front, but my point is both syntaxes are lightweight for the most commonly needed markup.)
Crinus 8y ago
I do not have anything online at the moment to show, but a small document processor i wrote some time ago used a @-based syntax which was essentially a tree of named nodes with attributes - essentially the same thing as XML, but with a simpler syntax that was something like: @Foo; is a standalone empty node named Foo @Foo[aa] has a "aa" attribute without a value @Foo[bb=cc] has a "bb" attribute with "cc" value @Foo[dd="hello, world",ee] has "dd" attribute with "hello, world" value and "ee" attribute without value @Foo{blah} has a "blah" text child @Foo{blah @Bar;} has "blah" and Bar children @Foo Bar shortcut for @Foo{Bar} @Foo{!{...}!} has ... as unprocessed content So you can type something like @P{This is a @I nice @Link[target=foo]{example} of @TT{code} @Smile;.} which would be sort of equivalent to XML This is a nice <Link target="foo">example</Link> of <TT>code</TT> <Smile/> (but with less repetition, especially for longer documents). In my processor the tree is passed to a script that does the actual handling by going through the nodes, children, etc and creating the output text (HTML or whatever) - the processor itself has no knowledge of specific node names, any meaning lies in the scripts. My biggest issue with the above though is that i do not like editing markup by hand, so at some point i want to replace all the above with a GUI tool that does the same job but in a more graphical way without the need for (visible to the user) markup syntax (so the above example would be edited as just text but with the "nice", "example" and "code" bits having representational styling).
- enriquto 8y ago
 having to close paragraphs is useless. It moves to the user of the markup a concern that is only of interest to the implementor. That should we exactly the other way round. I prefer html5, where you can leave tags open as if there's no tomorrow.
 - Crinus 8y ago
 This is just the way i have the scripts setup since it makes more sense for me for paragraphs to be full nodes that contain their children content, but it is trivial to make empty P nodes act as paragraph separators.
 - 8y ago
fouc 8y ago
Those are not lightweight markup languages at all? The main point of markdown is that it is supposed to look like a normal & readable text document in its own right. The transformation to html isn't absolutely necessary for readability.
- neilv 8y ago
  I'm leaning towards agreement with you, especially for embedded API docs in code. I've been using Scribble for embedded API docs for years,[1] and the formatted result looks great, and is very powerful and extensible (Scribble is actually a better LaTeX), but it's not nearly as readable in code as it could be. I think I actually preferred my earlier three-comment-character-Texinfo format I had before, and I even wrote a converter to Scribble for it (so that it would look idiomatic), except Texinfo format just couldn't handle Racket types. Now that I've put Racket on hold while I Rust a bit, in the interests of job-hunting, I'm happy to see Rust is using Markdown (coincidentally, also with the three-comment-character prefix). [1] Here's one formatted example, everything coming from the code files and package metadata, no separate documentation files: https://www.neilvandyke.org/racket/roomba/ https://www.neilvandyke.org/racket/roomba/
Ultimatt 8y ago
You might want to check out the Perl 6 extensions to POD https://docs.perl6.org/language/pod https://docs.perl6.org/language/pod
Ovid 8y ago
As a huge fan of Perl and very comfortable writing POD, I despise POD. I would much prefer Markdown for documentation. It's easier to write and easier to read. Just try creating lists in each.
fxfan 8y ago
I haven't tried dozens but I really really wanted to like asciidoc and rst. But they both require me to type n number of '=' for title string of length n. I confess I fail to see the utility in this kind of design. I'm still looking (Scribble seems anything but lightweight!). Perlpod probably looks perfect but I have to see more. Is there a markup that can do numbered lists?
- earenndil 8y ago
  Why do you need markup for that? 1. first item 2. second 3. and so on Markdown does parse that, but it seems superfluous to me tbh.
  - deleted 8y ago
    [deleted]
- ben509 8y ago
  I feel the same way about RST; really wanted to like it, fought with it, gave up because Sphinx consistently seems to make simple things hard. Regarding section titles, I think the theory is it makes it easier to stick new header levels into a document. (For those who aren't familiar with RST: If it sees underlines like ====, ----, ^^^^ or ~~~~~, it decides what the header levels are based on the order in which it sees them.) In practice, it means a typo makes it completely ignore your header, you have to learn a set of acceptable characters for headers, and there's no way to start a document with anything less than shouty H1. Nice idea, though. > Is there a markup that can do numbered lists? Correctly and usably? NOPE.

perlgeek 8y ago

I'm a big fan of Perl myself, but when writing POD, I occasionally stumble about the C<...> for inline code. The problem is that you have to take care that there is no > in the inline code, and if there is, you have to switch to C<< ... >>. With Markdown's `...`, this happened far less often to me.

lifthrasiir 8y ago
You are right that it is suboptimal. (I do think that Scribble's choice of braces is not too bad in this regard.) On the other hand, I can't readily type backquotes in my mobile phone or recent macOS [1] so there is a trade-off :-) [1] Annoyingly enough, macOS Sierra and later maps a backquote to a Won sign in Korean keyboard layouts.
- elcritch 8y ago
  Never read scribble before but I kinda like it. It reminds me of the good parts of latex. As in @action vs \action. Though I don’t like the table syntax as much from a WYSIWYG level (E.g. it doesn’t look table-ly) but for complex tables it don’t matter much. Can you readily extend scribble with Racket and how’s the install/usage story?
  - lifthrasiir 8y ago
    I think it is in the default Racket install. And yes, that's a problem; it is actually a Racket code behind the scene so it is fundamentally tied to Racket (that can be good or bad depending on your use case, and some strength points can be attributed to that).
  - Eugeleo 8y ago
    I know we're talking about docs right now, which is where Scribble really shines. And when it comes to blogging/prose, I'd recommend Pollen[1], which is a writing system based on scribble. It is fairly lightweight and extensible with Racket (which is by the way actually a pretty awesome language). I'm writing my college notes in it and couldn't be happier. The dev is very responsive on mailing lists and would gladly help you out if you have some trouble initially. There is a pretty thorough getting started guide on the linked page. There's a few blogs written in Pollen and even a few books (Ever heard about Flatland? Or Practical Typography?). [1] http://pollenpub.com http://pollenpub.com

astrobe_ 8y ago

Reading the comments here, it feels like people miss the whole point of markdown. Markdown's raison d'être is, "if you follow these conventions when doing ASCII art formatting, the my program can transform this to HTML or whatever other format". The conventions being mostly the ones used in the pre-HTML email era, like underlining headings with a bunch of ======. Orgmode is completely different to that. Orgmode is an "outliner/GTD" at heart, but this aspect has been buried under a ton of features. Then if you want "prawfishunal qvalitat" advanced formatting, typesetting and layout control what you really want is something like Latex. Yet another whole different thing.

lifthrasiir 8y ago
But Markdown was advertised as "a text-to-HTML conversion tool for web writers" [1]. The raison d'être as you stated is not an end goal but a possible benefit. Gruber may have thought that the extensibility of Markdown is done by raw HTML; now we know that's not enough. [1] https://daringfireball.net/projects/markdown/ https://daringfireball.net/projects/markdown/
hyperion2010 8y ago
The thing about org mode is that it has everything you need to do the full fancy latex typesetting without loosing the ability to export to other formats the way you do if you write raw latex. Of course there are times when you have no choice but to duplicate a block if you are doing something really fancy, but that is a fairly rare occurrence.
omaranto 8y ago
Of course Org Mode is completely different, because it can do tons of other stuff. But among it's many features, it also has lightweight markup syntax about which you could claim "if you follow these conventions when doing ASCII art formatting, the my program can transform this to HTML or whatever other format". And the conventions in Org Mode are also email-like in the sense that they don't look out of place in something meant to be plain text. (Maybe finding some large email corpus could answer which syntax is more email-like between Org Mode and Markdown, and maybe Markdown wins on that front, but my point is both syntaxes are lightweight for the most commonly needed markup.)

Crinus 8y ago

I do not have anything online at the moment to show, but a small document processor i wrote some time ago used a @-based syntax which was essentially a tree of named nodes with attributes - essentially the same thing as XML, but with a simpler syntax that was something like: @Foo; is a standalone empty node named Foo @Foo[aa] has a "aa" attribute without a value @Foo[bb=cc] has a "bb" attribute with "cc" value @Foo[dd="hello, world",ee] has "dd" attribute with "hello, world" value and "ee" attribute without value @Foo{blah} has a "blah" text child @Foo{blah @Bar;} has "blah" and Bar children @Foo Bar shortcut for @Foo{Bar} @Foo{!{...}!} has ... as unprocessed content So you can type something like @P{This is a @I nice @Link[target=foo]{example} of @TT{code} @Smile;.} which would be sort of equivalent to XML This is a nice <Link target="foo">example</Link> of <TT>code</TT> <Smile/> (but with less repetition, especially for longer documents). In my processor the tree is passed to a script that does the actual handling by going through the nodes, children, etc and creating the output text (HTML or whatever) - the processor itself has no knowledge of specific node names, any meaning lies in the scripts. My biggest issue with the above though is that i do not like editing markup by hand, so at some point i want to replace all the above with a GUI tool that does the same job but in a more graphical way without the need for (visible to the user) markup syntax (so the above example would be edited as just text but with the "nice", "example" and "code" bits having representational styling).

enriquto 8y ago
having to close paragraphs is useless. It moves to the user of the markup a concern that is only of interest to the implementor. That should we exactly the other way round. I prefer html5, where you can leave tags open as if there's no tomorrow.
- Crinus 8y ago
  This is just the way i have the scripts setup since it makes more sense for me for paragraphs to be full nodes that contain their children content, but it is trivial to make empty P nodes act as paragraph separators.
  - 8y ago

fouc 8y ago

Those are not lightweight markup languages at all? The main point of markdown is that it is supposed to look like a normal & readable text document in its own right. The transformation to html isn't absolutely necessary for readability.

neilv 8y ago
I'm leaning towards agreement with you, especially for embedded API docs in code. I've been using Scribble for embedded API docs for years,[1] and the formatted result looks great, and is very powerful and extensible (Scribble is actually a better LaTeX), but it's not nearly as readable in code as it could be. I think I actually preferred my earlier three-comment-character-Texinfo format I had before, and I even wrote a converter to Scribble for it (so that it would look idiomatic), except Texinfo format just couldn't handle Racket types. Now that I've put Racket on hold while I Rust a bit, in the interests of job-hunting, I'm happy to see Rust is using Markdown (coincidentally, also with the three-comment-character prefix). [1] Here's one formatted example, everything coming from the code files and package metadata, no separate documentation files: https://www.neilvandyke.org/racket/roomba/ https://www.neilvandyke.org/racket/roomba/

Ultimatt 8y ago

You might want to check out the Perl 6 extensions to POD https://docs.perl6.org/language/pod https://docs.perl6.org/language/pod

Ovid 8y ago

As a huge fan of Perl and very comfortable writing POD, I despise POD. I would much prefer Markdown for documentation. It's easier to write and easier to read. Just try creating lists in each.

fxfan 8y ago

I haven't tried dozens but I really really wanted to like asciidoc and rst. But they both require me to type n number of '=' for title string of length n. I confess I fail to see the utility in this kind of design. I'm still looking (Scribble seems anything but lightweight!). Perlpod probably looks perfect but I have to see more. Is there a markup that can do numbered lists?

earenndil 8y ago
Why do you need markup for that? 1. first item 2. second 3. and so on Markdown does parse that, but it seems superfluous to me tbh.
- deleted 8y ago
  [deleted]
ben509 8y ago
I feel the same way about RST; really wanted to like it, fought with it, gave up because Sphinx consistently seems to make simple things hard. Regarding section titles, I think the theory is it makes it easier to stick new header levels into a document. (For those who aren't familiar with RST: If it sees underlines like ====, ----, ^^^^ or ~~~~~, it decides what the header levels are based on the order in which it sees them.) In practice, it means a typo makes it completely ignore your header, you have to learn a set of acceptable characters for headers, and there's no way to start a document with anything less than shouty H1. Nice idea, though. > Is there a markup that can do numbered lists? Correctly and usably? NOPE.