10 ms·

Djot: A light markup language by the creator of Pandoc and CommonMark

276 points by eevilspock 4y ago

tosh 4y ago
Related: "Beyond Markdown" https://johnmacfarlane.net/beyond-markdown.html https://johnmacfarlane.net/beyond-markdown.html
mikl 4y ago
Given the ubiquity of Markdown, and how painful it is to build a completely compliant parser, I really hope Djot (or something like it) would take off. Shame that the creator of Markdown blocks any efforts to to fix or standardise the format.
- jdelman 4y ago
  CommonMark is the "standardized" version, isn't it?
  - anderskaseorg 4y ago
    The creator of the original Markdown requested that the standardized version be renamed to avoid using the word “Markdown”. https://blog.codinghorror.com/standard-markdown-is-now-common-markdown/ https://blog.codinghorror.com/standard-markdown-is-now-commo...
    - mikl 4y ago
      …and threatened legal action against anyone using the word “Markdown” in a way he did not approve of. Jeff Atwood goes out of his way to be courteous to Gruber in this post, but frankly, I think Gruber was being a jerk here, using his claim to the name to tyrannise an open source community that he has otherwise not been involved with in the slightest the last ~17 years.
      - tonto 4y ago
        TIL Aaron Schwartz had at least some role in creating markdown also https://en.wikipedia.org/wiki/John_Gruber#History https://en.wikipedia.org/wiki/John_Gruber#History
    - jdelman 4y ago
      Yes, that's exactly my point. There already is a standardized version, created by the author of the post itself.
adlpz 4y ago
I understand the rationale and how CommonMark parsing is not trivial and could be simplified, but the resulting language misses, for me, the best part of Markdown: that it happens to be pretty much just what I'd write in plain text anyways. The odd newline requirements on lists and blocks, the special syntax for raw HTML and so on makes Djot feel more artificial to me.
- layer8 4y ago
  What odd newline requirements? I skimmed the syntax reference and couldn’t find any.
  - gernb 4y ago
    In markdown My favorite number is probably the number 1. It's the smallest natural number that is > 0. With pencils, though, I prefer a # 2. is a paragraph, a list item, a block quote, and a heading (4 things). In djot it's just a single paragraph. If you want it to be 4 things you have to add a newline between each one. Personally I think djot gets this right.
    - layer8 4y ago
      Ah, thanks. I agree that it makes sense to require an empty line before block elements that typically also require an empty line after.
    - bccdee 4y ago
      Agreed with the exception that djot also applies this to nested lists, which I think is a huge misstep. User ergonomics are more important than convenient parsing, and while many markdown parsers interpret your example in an un-ergonomic way, djot rejects this: - foo - bar - baz - foobar and demands this: - foo - bar - baz - foobar which is an unergonomic choice.
      - joshjob42 4y ago
        This is so horrifying I did not believe you and had to go test to make sure. Jesus that's awful. Force a newline on me before a list? Ugh but fine. Force it for every sub bullet? No freaking way, that looks catastrophically bad. This forcing new lines thing almost totally eliminates any desire I have to experiment with it, when I was very excited about it when I first saw it come up. Hopefully they'll change their mind. They want source code to be readable then make a decision that makes our eyes bleed.
civopsec 4y ago
> In djot, we just get rid of indented code blocks. Most people prefer fenced code blocks anyway, and we don't need two different ways of writing code blocks (goal 11). Sensible. Mostly since it makes other things easier (goal 5), second because one thing is only represented in one way, and thirdly (least important) since indented code blocks are kind of a pain to format compared to fenced code blocks.
- fabiospampinato 4y ago
  There are two types of fenced code blocks, and the one most people write, the one with backticks, basically can't be written in some languages because their keyboards like a backtick key.
  - civopsec 4y ago
    Turning off dead keys can help.
gernb 4y ago
The odds of replacing markdown and all it's issues seem nearly impossible given its ubiquity and I've run into many of those problems but, this seems just as arbitrary in many ways, For example: > Block-level elements can't interrupt paragraphs (or headings), because of goal 7 It then goes on to show they do interrupt paragraphs - this then - this other thing vs - this then - this other thing The 2nd is 2 list items but it's just the first with being interrupted by a block-level element.
- chriswarbo 4y ago
  The second example is indented; line-wrapping doesn't introduce indentation, so an indented line cannot be 'interrupting a paragraph'
  - gernb 4y ago
    I think you're missing the point (or I'm missing yours). I want both examples above to be one line item, not 2. djot says a blank line is required to start a new block but then sabotages itself by making an exception for list items because the author wants more compact lists. I'd prefer no compact lists rather than strange exceptions to the rules.
dschuessler 4y ago
I recently found out that the author is John MacFarlane, a philosophy professor I have read papers from in totally unrelated contexts. I was more than surprised to see that he is the original author of pandoc. It boggles my mind how someone with an academic career in a somewhat unrelated field can have a GitHub profile like him. It's really impressive. On topic, though, preceding sublists with empty lines is a complete non-starter for me. However, since I don't hard-wrap lines (goal 7), but use soft-wrap only, I am not in the target audience anyways.
- abathur 4y ago
  Good sign of a persistent yak-shaver :)
- IshKebab 4y ago
  Yeah that stuck out to me as the most objectionable thing at first glance too. Otherwise it looks reasonably sane. I currently use AsciiDoc and it's ok but this looks slightly better I would say. Both are clearly better than RST.
- eevilspock 4y ago
  He is also the author of the CommonMark spec: https://spec.commonmark.org/0.30/ https://spec.commonmark.org/0.30/, https://commonmark.org https://commonmark.org RE goal 7: here is my questioning of that goal and JGM's response, my counter, etc: https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/2?u=vas https://talk.commonmark.org/t/djot-a-light-markup-language-b....
  - aidenn0 4y ago
    I get the thrust of "not wanting to pay for hard-wrapping" but I'm not sure I understand what you think a better design would be; does a single newline always introduce a new block element?
    - eevilspock 4y ago
      No. Within a paragraph it would introduce a hard line break, e.g. like you'd use for poetry. You'd use two line breaks, i.e. a blank line, to start a new paragraph. In other words, the plain text analog to WYSIWYG.
solarkraft 4y ago
HTML is quite good at what it was invented for. Why is it so out of fashion?
- leephillips 4y ago
 17^th^ is more pleasant to read and write than 17th, for example. For me, I use Markdown because I can transform it with Pandoc and my own filters to any other kind of document: https://lee-phillips.org/panflute-gnuplot/ https://lee-phillips.org/panflute-gnuplot/
- abathur 4y ago
 Verbosity is the obvious answer, but this past year I stumbled into a conclusion that wasn't obvious to me before: "semantic" HTML isn't serving authors' needs--it's serving the UA's needs. I picked at what I mean a little in a post this summer: https://t-ravis.com/post/doc/the_gizmos_role_in_markup/ https://t-ravis.com/post/doc/the_gizmos_role_in_markup/ (I've also made the first post in an unfinished series that will continue to explore these ideas about markup, but a bit less directly: https://t-ravis.com/post/doc/what_color_is_your_markup/ https://t-ravis.com/post/doc/what_color_is_your_markup/)
- Cyberdog 4y ago
 Aside from the sibling answers, Markdown, though originally intended solely for HTML output, is useful for writing other types of documents; I've written an e-book which was eventually destined for PDF format as a series of Markdown files. I could have used HTML instead, but aside from being more difficult to write, Markdown's document orientation solves some problems with that (should the HTML-to-PDF translator handle CSS and evaluate JavaScript? How does it deal with <audio> or <video> tags?) by not making them a possibility in the first place. Similarly, for things like comments on message boards or blogs, a user can just dump a bunch of text into the text box without knowing the first thing about Markdown and expect it to look more or less like how it was entered, with paragraph breaks and such. If you force these people to use HTML instead, you're forcing them to at least learn and use - which is probably simple for those of us reading HN, but I don't consider it a reasonable request for the normies on Reddit. So, sure, HTML is quite good at what it was invented for, but not everything that involves text input on the internet or elsewhere should be HTML.
Cyberdog 4y ago
As someone who's been using Markdown since before it was cool, I love it! I think writing the implementation in Lua is an interesting take since Lua out-of-the-box does not support standard regular expressions; it instead has its own pattern-matching thing which is a bit more limited. But it looks like they've embraced that limitation to force themselves to write something that doesn't need a full regex library to be sanely parsed. My biggest complaint is that asterisks map to and underscores map to (in HTML terms). This is not backwards-compatible with Markdown where (asterisk)foo(asterisk) gets you foo, and it feels objectively backwards, if that makes sense. I wonder if there's any chance they could reverse that.
- bjeanes 4y ago
 > My biggest complaint is that asterisks map to and underscores map to (in HTML terms). This is not backwards-compatible with Markdown where (asterisk)foo(asterisk) gets you foo, and it feels objectively backwards, if that makes sense. I wonder if there's any chance they could reverse that. Interesting. I suspect something like this will always be subjective, but I find the opposite to be true. *bold* and _italics_ make the most sense to me and is always what I wished Markdown did. Probably, this is because I was familiar with Textile[1] before I used Markdown, and this is what it does. Today, Slack also uses this convention instead of the Markdown convention (though I believe it _used_ to use the latter). [1]: https://textile-lang.com/ https://textile-lang.com/
 - Cyberdog 4y ago
 > Interesting. I suspect something like this will always be subjective, but I find the opposite to be true. bold and _italics_ make the most sense to me and is always what I wished Markdown did. Yeah… After posting that message, I remembered that _foo_ in Markdown also results in foo - so the underscores are backwards-compatible. But I've just always used asterisks so I completely forgot about it. So I guess they were bound to make some people upset no matter which one they made and which one they made , and I'm on the losing side. :P Whatever. If this ends up taking over the world as Markdown did (and I hope it does), I'll just get used to it, I suppose. Incidentally, how did you "deactivate" HN's parsing of the asterisks in your reply?
joemaller1 4y ago
How do you say it?
- dcre 4y ago
  Jot.
- jez 4y ago
  The homepage lists the IPA https://djot.net/ https://djot.net/ /dʒɑt/ /dʒ/ is like both the `j` and `dg` in english judge /a/ is like the a in father in most American english dialects /dʒɑt/ is also the complete IPA for the word "jot" (as in "to jot down") in American English https://en.wiktionary.org/wiki/jot https://en.wiktionary.org/wiki/jot
  - Cyberdog 4y ago
    So then why is it called "Djot" instead of "Jot?"
    - macintux 4y ago
      I would wager the answer is simply findability, although reducing the likelihood of trademark collisions is a nice benefit too.
    - euroderf 4y ago
      maybe "žot" ? or "ʤot" ?
0xbadcafebee 4y ago
I never realized the problem with markup until that phrase "light markup". The problem is that it's designed for a human to edit it by hand with a text editor. It's a programmer designing for a programmer, rather than for a user. It's a plumber designing a sink. A mechanic designing a radio. A busboy designing tableware. What we should have instead of markup, is a WYSIWYG with keyboard shortcuts. Confluence, for example, will convert Markdown into rich text in real time, and has keyboard shortcuts for its other layout/style options. But the point is to edit it in a GUI, see your changes live, and not need to learn a language in order to edit a document. There are so many problems you avoid by giving the user tools to make their life easier. Markup may be one tiny part of that, but it shouldn't be considered the complete solution.
- civopsec 4y ago
  Interestingly though in this case it’s an amateur programmer (since being a professor of philosophy is his day job).
  - nerpderp82 4y ago
    The amateur programmer in this case switched from Haskell to Lua. Interesting.
    - simonmic 4y ago
      I doubt it's that. Rather, you can write easy-to-install pandoc plugins in Lua, so prototyping with it makes a lot of sense. https://pandoc.org/custom-readers.html https://pandoc.org/custom-readers.html https://pandoc.org/custom-writers.html https://pandoc.org/custom-writers.html
- nequo 4y ago
  > say, by clicking on the display itself, clicking "Menu -> Layout -> Options" selecting the layout we want, and seeing it displayed immediately Maybe I am misunderstanding what you are saying. Isn’t this the same as LibreOffice Writer and Microsoft Word? What is the existing problem that such an interface would solve?
eslaught 4y ago
So this is what happened to the energy that had been going into CommonMark. I'd been wondering, because the spec releases for CommonMark have been slowing down for quite a while [1], and it felt like certain issues just never saw attention [2]. I'm glad that energy didn't just dissipate and actually went into something useful. On a practical level, nearly everything this does is better served by Pandoc Markdown today. But maybe in some not-so-distant future I'll be able to make use of this. [1]: https://spec.commonmark.org/ https://spec.commonmark.org/ [2]: https://talk.commonmark.org/t/issues-we-must-resolve-before-1-0-release-6-remaining/1287 https://talk.commonmark.org/t/issues-we-must-resolve-before-...
gravypod 4y ago
(Opinions are my own) I like the ideas behind this but I do not think this will take off as much as CommonMark has. My main reasons for thinking this: 1. Users don't really care about edge cases. CommonMark (cmark-gfm really) is "good enough". If something looks odd people go "oh, strange" and make a 3 second edit and forget about it. 2. There's already multiple parsers for every language for markdown. Here there is a single Lua implementation. 3. GitHub is the king. If it cannot render in GitHub it's not real. If it's harder for me to write in the 99% case then it's not worth it. If it's harder for me to implement within my application then I'm too lazy to switch. Nice things in here: 1. Less ambiguous parsing. For the parser infrastructure I support at $JOB this makes supporting questions from users a HUGE pain in the ass. 2. AST-first design. Seems like everything appears in the AST (even references!) which is a huge win over cmark-gfm. There's also source positions included which is a MASSIVE win for tooling. 3. Custom Attributes implementation. This is really nice for extensibility. It is less readable but having it as an option is nice.
- detaro 4y ago
  Plenty of use cases for markup languages that never touch Github, luckily.
- thomascgalvin 4y ago
  I'm a fan of Asciidoc, and I think djot is a nice "next" markdown, but: > If it cannot render in GitHub it's not real. This is 100% accurate. If I limit myself to CommonMark, with maybe a few extras tossed in gently, I can be pretty sure that my meaning will be correctly rendered. Anything else, no matter how nice the syntax is, or how powerful the expressions are, won't render for the people I want it to renfer for.
- 0cf8612b2e1e 4y ago
  2) The design is still incomplete. Eventually more parsers could be made. Bonus, they will evidently be simple to write 3) Github has renderers for multiple formats if this language ever took off, not unreasonable for them to add one to the pile.
  - 4y ago
howenterprisey 4y ago
Love it, will see how many places I can get this introduced. The things that pissed the author of this off are precisely the things that piss me off. Unconcerned with the newlines in list syntax; that's how I'd write them myself, anyway. I like many things about this, including the various uses of curly braces.
tonto 4y ago
look forward to having standardized e.g. "figure captions", looks like it's being worked out here https://github.com/jgm/djot/issues/28 https://github.com/jgm/djot/issues/28 pandoc is interesting because with markdown it currently converts the "alt text" for figures ![alttexthere](img.jpg) in markdown->pdf conversion into the figure captions, which is cool but makes me think: it is not alt anymore, anyone should have it
lynx23 4y ago
This looks pretty intersting. Besides, I find it fascinating how jgm apparently is happy on both sides of the "typed language" spectrum. pandoc in Haskell, and djot in Lua. True, pandoc has acquired a lot of Lua in the recent years, but still. I wonder how it feels switching from the cozy safety of a language like Haskell, back to the "good old" interpreter days. I personally have dropped writing anything larger then 1k lines in a dynamic language. Have been bitten by my own inconsistencies too much in the past. I wonder what someone like jgm has to say about the difference of writing Haskell and Lua.
- jinwoo68 4y ago
  I'm also happy with both Common Lisp and static functional languages like OCaml and Haskell. Each has its own advantages (and disadvantages).
- nequo 4y ago
  > pandoc in Haskell, and djot in Lua. I wonder if jgm’s djot parser being in Lua is also influenced by the fact that he is a Vim user: https://github.com/jgm/dotvim https://github.com/jgm/dotvim (Assuming Neovim because this repo has some Lua files.)
- KolenCh 4y ago
  My guess about the use of lua would be simply because he don't want to commit it to pandoc yet. If you push it to pandoc, it's next to impossible to remove it, or change its behavior in a backward incompatible way. So naturally, since the embedded lua is expanding its functionality over the years, including the ability to write custom writer and reader, this is the way to be able to tap into pandoc without committing. Tapping into pandoc is important here, as the community who likely will provide feedbacks and insights are there. I'll bet eventually when it becomes mature, a native Haskell implementation will comes.
w10-1 4y ago
Most excellent! If Github supported it alongside markdown, this would drive the markdown parsers to support this as a flavor, and then the community could decide; it will be fascinating to analyze the open-source population. I'm finding dozens of applications for a lingua franca of mildly-structured, human-readable language that transforms as easily into data as documents and UI's. I've built out 10+ markdown applications using 3-4 parsers, and always had to reduce the effective semantics to a small subset of features, and then had to use metadata or convention to do what attributes will do much more cleanly. As for Lua: yes it's the best for integrating into pandoc performance-wise, but ay! the table structures and metadata tables to get anything like typing are mind bending, and I am still miffed at being blocked at a critical time by a bug in pandoc's lua initializer for tables. Aside language/stdlib developers, jgm is my greatest benefactor, and he deserves a cadre of loyal and competent implementors.
__tmk__ 4y ago
Wow, this is all I ever wanted. The only thing that's maybe missing is a natural way to do captions for images and tables, and a syntax for spoilers. But these can probably be built on top of the div and span syntax.
deafpolygon 4y ago
After doing Markdown for a while... and discovering Org. I'll keep Org. Thanks.

276 points by eevilspock 4y ago

tosh 4y ago

Related: "Beyond Markdown" https://johnmacfarlane.net/beyond-markdown.html https://johnmacfarlane.net/beyond-markdown.html

mikl 4y ago

Given the ubiquity of Markdown, and how painful it is to build a completely compliant parser, I really hope Djot (or something like it) would take off. Shame that the creator of Markdown blocks any efforts to to fix or standardise the format.

jdelman 4y ago
CommonMark is the "standardized" version, isn't it?
- anderskaseorg 4y ago
  The creator of the original Markdown requested that the standardized version be renamed to avoid using the word “Markdown”. https://blog.codinghorror.com/standard-markdown-is-now-common-markdown/ https://blog.codinghorror.com/standard-markdown-is-now-commo...
  - mikl 4y ago
    …and threatened legal action against anyone using the word “Markdown” in a way he did not approve of. Jeff Atwood goes out of his way to be courteous to Gruber in this post, but frankly, I think Gruber was being a jerk here, using his claim to the name to tyrannise an open source community that he has otherwise not been involved with in the slightest the last ~17 years.
    - tonto 4y ago
      TIL Aaron Schwartz had at least some role in creating markdown also https://en.wikipedia.org/wiki/John_Gruber#History https://en.wikipedia.org/wiki/John_Gruber#History
  - jdelman 4y ago
    Yes, that's exactly my point. There already is a standardized version, created by the author of the post itself.

adlpz 4y ago

I understand the rationale and how CommonMark parsing is not trivial and could be simplified, but the resulting language misses, for me, the best part of Markdown: that it happens to be pretty much just what I'd write in plain text anyways. The odd newline requirements on lists and blocks, the special syntax for raw HTML and so on makes Djot feel more artificial to me.

layer8 4y ago
What odd newline requirements? I skimmed the syntax reference and couldn’t find any.
- gernb 4y ago
  In markdown My favorite number is probably the number 1. It's the smallest natural number that is > 0. With pencils, though, I prefer a # 2. is a paragraph, a list item, a block quote, and a heading (4 things). In djot it's just a single paragraph. If you want it to be 4 things you have to add a newline between each one. Personally I think djot gets this right.
  - layer8 4y ago
    Ah, thanks. I agree that it makes sense to require an empty line before block elements that typically also require an empty line after.
  - bccdee 4y ago
    Agreed with the exception that djot also applies this to nested lists, which I think is a huge misstep. User ergonomics are more important than convenient parsing, and while many markdown parsers interpret your example in an un-ergonomic way, djot rejects this: - foo - bar - baz - foobar and demands this: - foo - bar - baz - foobar which is an unergonomic choice.
    - joshjob42 4y ago
      This is so horrifying I did not believe you and had to go test to make sure. Jesus that's awful. Force a newline on me before a list? Ugh but fine. Force it for every sub bullet? No freaking way, that looks catastrophically bad. This forcing new lines thing almost totally eliminates any desire I have to experiment with it, when I was very excited about it when I first saw it come up. Hopefully they'll change their mind. They want source code to be readable then make a decision that makes our eyes bleed.

civopsec 4y ago

> In djot, we just get rid of indented code blocks. Most people prefer fenced code blocks anyway, and we don't need two different ways of writing code blocks (goal 11). Sensible. Mostly since it makes other things easier (goal 5), second because one thing is only represented in one way, and thirdly (least important) since indented code blocks are kind of a pain to format compared to fenced code blocks.

fabiospampinato 4y ago
There are two types of fenced code blocks, and the one most people write, the one with backticks, basically can't be written in some languages because their keyboards like a backtick key.
- civopsec 4y ago
  Turning off dead keys can help.

gernb 4y ago

The odds of replacing markdown and all it's issues seem nearly impossible given its ubiquity and I've run into many of those problems but, this seems just as arbitrary in many ways, For example: > Block-level elements can't interrupt paragraphs (or headings), because of goal 7 It then goes on to show they do interrupt paragraphs - this then - this other thing vs - this then - this other thing The 2nd is 2 list items but it's just the first with being interrupted by a block-level element.

chriswarbo 4y ago
The second example is indented; line-wrapping doesn't introduce indentation, so an indented line cannot be 'interrupting a paragraph'
- gernb 4y ago
  I think you're missing the point (or I'm missing yours). I want both examples above to be one line item, not 2. djot says a blank line is required to start a new block but then sabotages itself by making an exception for list items because the author wants more compact lists. I'd prefer no compact lists rather than strange exceptions to the rules.

dschuessler 4y ago

I recently found out that the author is John MacFarlane, a philosophy professor I have read papers from in totally unrelated contexts. I was more than surprised to see that he is the original author of pandoc. It boggles my mind how someone with an academic career in a somewhat unrelated field can have a GitHub profile like him. It's really impressive. On topic, though, preceding sublists with empty lines is a complete non-starter for me. However, since I don't hard-wrap lines (goal 7), but use soft-wrap only, I am not in the target audience anyways.

abathur 4y ago
Good sign of a persistent yak-shaver :)
IshKebab 4y ago
Yeah that stuck out to me as the most objectionable thing at first glance too. Otherwise it looks reasonably sane. I currently use AsciiDoc and it's ok but this looks slightly better I would say. Both are clearly better than RST.
eevilspock 4y ago
He is also the author of the CommonMark spec: https://spec.commonmark.org/0.30/ https://spec.commonmark.org/0.30/, https://commonmark.org https://commonmark.org RE goal 7: here is my questioning of that goal and JGM's response, my counter, etc: https://talk.commonmark.org/t/djot-a-light-markup-language-by-jgm/4229/2?u=vas https://talk.commonmark.org/t/djot-a-light-markup-language-b....
- aidenn0 4y ago
  I get the thrust of "not wanting to pay for hard-wrapping" but I'm not sure I understand what you think a better design would be; does a single newline always introduce a new block element?
  - eevilspock 4y ago
    No. Within a paragraph it would introduce a hard line break, e.g. like you'd use for poetry. You'd use two line breaks, i.e. a blank line, to start a new paragraph. In other words, the plain text analog to WYSIWYG.

solarkraft 4y ago

HTML is quite good at what it was invented for. Why is it so out of fashion?

leephillips 4y ago
17^th^ is more pleasant to read and write than 17th, for example. For me, I use Markdown because I can transform it with Pandoc and my own filters to any other kind of document: https://lee-phillips.org/panflute-gnuplot/ https://lee-phillips.org/panflute-gnuplot/
abathur 4y ago
Verbosity is the obvious answer, but this past year I stumbled into a conclusion that wasn't obvious to me before: "semantic" HTML isn't serving authors' needs--it's serving the UA's needs. I picked at what I mean a little in a post this summer: https://t-ravis.com/post/doc/the_gizmos_role_in_markup/ https://t-ravis.com/post/doc/the_gizmos_role_in_markup/ (I've also made the first post in an unfinished series that will continue to explore these ideas about markup, but a bit less directly: https://t-ravis.com/post/doc/what_color_is_your_markup/ https://t-ravis.com/post/doc/what_color_is_your_markup/)
Cyberdog 4y ago
Aside from the sibling answers, Markdown, though originally intended solely for HTML output, is useful for writing other types of documents; I've written an e-book which was eventually destined for PDF format as a series of Markdown files. I could have used HTML instead, but aside from being more difficult to write, Markdown's document orientation solves some problems with that (should the HTML-to-PDF translator handle CSS and evaluate JavaScript? How does it deal with <audio> or <video> tags?) by not making them a possibility in the first place. Similarly, for things like comments on message boards or blogs, a user can just dump a bunch of text into the text box without knowing the first thing about Markdown and expect it to look more or less like how it was entered, with paragraph breaks and such. If you force these people to use HTML instead, you're forcing them to at least learn and use - which is probably simple for those of us reading HN, but I don't consider it a reasonable request for the normies on Reddit. So, sure, HTML is quite good at what it was invented for, but not everything that involves text input on the internet or elsewhere should be HTML.

Cyberdog 4y ago

As someone who's been using Markdown since before it was cool, I love it! I think writing the implementation in Lua is an interesting take since Lua out-of-the-box does not support standard regular expressions; it instead has its own pattern-matching thing which is a bit more limited. But it looks like they've embraced that limitation to force themselves to write something that doesn't need a full regex library to be sanely parsed. My biggest complaint is that asterisks map to and underscores map to (in HTML terms). This is not backwards-compatible with Markdown where (asterisk)foo(asterisk) gets you foo, and it feels objectively backwards, if that makes sense. I wonder if there's any chance they could reverse that.

bjeanes 4y ago
> My biggest complaint is that asterisks map to and underscores map to (in HTML terms). This is not backwards-compatible with Markdown where (asterisk)foo(asterisk) gets you foo, and it feels objectively backwards, if that makes sense. I wonder if there's any chance they could reverse that. Interesting. I suspect something like this will always be subjective, but I find the opposite to be true. *bold* and _italics_ make the most sense to me and is always what I wished Markdown did. Probably, this is because I was familiar with Textile[1] before I used Markdown, and this is what it does. Today, Slack also uses this convention instead of the Markdown convention (though I believe it _used_ to use the latter). [1]: https://textile-lang.com/ https://textile-lang.com/
- Cyberdog 4y ago
 > Interesting. I suspect something like this will always be subjective, but I find the opposite to be true. bold and _italics_ make the most sense to me and is always what I wished Markdown did. Yeah… After posting that message, I remembered that _foo_ in Markdown also results in foo - so the underscores are backwards-compatible. But I've just always used asterisks so I completely forgot about it. So I guess they were bound to make some people upset no matter which one they made and which one they made , and I'm on the losing side. :P Whatever. If this ends up taking over the world as Markdown did (and I hope it does), I'll just get used to it, I suppose. Incidentally, how did you "deactivate" HN's parsing of the asterisks in your reply?

joemaller1 4y ago

How do you say it?

dcre 4y ago
Jot.
jez 4y ago
The homepage lists the IPA https://djot.net/ https://djot.net/ /dʒɑt/ /dʒ/ is like both the `j` and `dg` in english judge /a/ is like the a in father in most American english dialects /dʒɑt/ is also the complete IPA for the word "jot" (as in "to jot down") in American English https://en.wiktionary.org/wiki/jot https://en.wiktionary.org/wiki/jot
- Cyberdog 4y ago
  So then why is it called "Djot" instead of "Jot?"
  - macintux 4y ago
    I would wager the answer is simply findability, although reducing the likelihood of trademark collisions is a nice benefit too.
  - euroderf 4y ago
    maybe "žot" ? or "ʤot" ?

0xbadcafebee 4y ago

I never realized the problem with markup until that phrase "light markup". The problem is that it's designed for a human to edit it by hand with a text editor. It's a programmer designing for a programmer, rather than for a user. It's a plumber designing a sink. A mechanic designing a radio. A busboy designing tableware. What we should have instead of markup, is a WYSIWYG with keyboard shortcuts. Confluence, for example, will convert Markdown into rich text in real time, and has keyboard shortcuts for its other layout/style options. But the point is to edit it in a GUI, see your changes live, and not need to learn a language in order to edit a document. There are so many problems you avoid by giving the user tools to make their life easier. Markup may be one tiny part of that, but it shouldn't be considered the complete solution.

civopsec 4y ago
Interestingly though in this case it’s an amateur programmer (since being a professor of philosophy is his day job).
- nerpderp82 4y ago
  The amateur programmer in this case switched from Haskell to Lua. Interesting.
  - simonmic 4y ago
    I doubt it's that. Rather, you can write easy-to-install pandoc plugins in Lua, so prototyping with it makes a lot of sense. https://pandoc.org/custom-readers.html https://pandoc.org/custom-readers.html https://pandoc.org/custom-writers.html https://pandoc.org/custom-writers.html
nequo 4y ago
> say, by clicking on the display itself, clicking "Menu -> Layout -> Options" selecting the layout we want, and seeing it displayed immediately Maybe I am misunderstanding what you are saying. Isn’t this the same as LibreOffice Writer and Microsoft Word? What is the existing problem that such an interface would solve?

eslaught 4y ago

So this is what happened to the energy that had been going into CommonMark. I'd been wondering, because the spec releases for CommonMark have been slowing down for quite a while [1], and it felt like certain issues just never saw attention [2]. I'm glad that energy didn't just dissipate and actually went into something useful. On a practical level, nearly everything this does is better served by Pandoc Markdown today. But maybe in some not-so-distant future I'll be able to make use of this. [1]: https://spec.commonmark.org/ https://spec.commonmark.org/ [2]: https://talk.commonmark.org/t/issues-we-must-resolve-before-1-0-release-6-remaining/1287 https://talk.commonmark.org/t/issues-we-must-resolve-before-...

gravypod 4y ago

(Opinions are my own) I like the ideas behind this but I do not think this will take off as much as CommonMark has. My main reasons for thinking this: 1. Users don't really care about edge cases. CommonMark (cmark-gfm really) is "good enough". If something looks odd people go "oh, strange" and make a 3 second edit and forget about it. 2. There's already multiple parsers for every language for markdown. Here there is a single Lua implementation. 3. GitHub is the king. If it cannot render in GitHub it's not real. If it's harder for me to write in the 99% case then it's not worth it. If it's harder for me to implement within my application then I'm too lazy to switch. Nice things in here: 1. Less ambiguous parsing. For the parser infrastructure I support at $JOB this makes supporting questions from users a HUGE pain in the ass. 2. AST-first design. Seems like everything appears in the AST (even references!) which is a huge win over cmark-gfm. There's also source positions included which is a MASSIVE win for tooling. 3. Custom Attributes implementation. This is really nice for extensibility. It is less readable but having it as an option is nice.

detaro 4y ago
Plenty of use cases for markup languages that never touch Github, luckily.
thomascgalvin 4y ago
I'm a fan of Asciidoc, and I think djot is a nice "next" markdown, but: > If it cannot render in GitHub it's not real. This is 100% accurate. If I limit myself to CommonMark, with maybe a few extras tossed in gently, I can be pretty sure that my meaning will be correctly rendered. Anything else, no matter how nice the syntax is, or how powerful the expressions are, won't render for the people I want it to renfer for.
0cf8612b2e1e 4y ago
2) The design is still incomplete. Eventually more parsers could be made. Bonus, they will evidently be simple to write 3) Github has renderers for multiple formats if this language ever took off, not unreasonable for them to add one to the pile.
- 4y ago

howenterprisey 4y ago

Love it, will see how many places I can get this introduced. The things that pissed the author of this off are precisely the things that piss me off. Unconcerned with the newlines in list syntax; that's how I'd write them myself, anyway. I like many things about this, including the various uses of curly braces.

tonto 4y ago

look forward to having standardized e.g. "figure captions", looks like it's being worked out here https://github.com/jgm/djot/issues/28 https://github.com/jgm/djot/issues/28 pandoc is interesting because with markdown it currently converts the "alt text" for figures ![alttexthere](img.jpg) in markdown->pdf conversion into the figure captions, which is cool but makes me think: it is not alt anymore, anyone should have it

lynx23 4y ago

This looks pretty intersting. Besides, I find it fascinating how jgm apparently is happy on both sides of the "typed language" spectrum. pandoc in Haskell, and djot in Lua. True, pandoc has acquired a lot of Lua in the recent years, but still. I wonder how it feels switching from the cozy safety of a language like Haskell, back to the "good old" interpreter days. I personally have dropped writing anything larger then 1k lines in a dynamic language. Have been bitten by my own inconsistencies too much in the past. I wonder what someone like jgm has to say about the difference of writing Haskell and Lua.

jinwoo68 4y ago
I'm also happy with both Common Lisp and static functional languages like OCaml and Haskell. Each has its own advantages (and disadvantages).
nequo 4y ago
> pandoc in Haskell, and djot in Lua. I wonder if jgm’s djot parser being in Lua is also influenced by the fact that he is a Vim user: https://github.com/jgm/dotvim https://github.com/jgm/dotvim (Assuming Neovim because this repo has some Lua files.)
KolenCh 4y ago
My guess about the use of lua would be simply because he don't want to commit it to pandoc yet. If you push it to pandoc, it's next to impossible to remove it, or change its behavior in a backward incompatible way. So naturally, since the embedded lua is expanding its functionality over the years, including the ability to write custom writer and reader, this is the way to be able to tap into pandoc without committing. Tapping into pandoc is important here, as the community who likely will provide feedbacks and insights are there. I'll bet eventually when it becomes mature, a native Haskell implementation will comes.

w10-1 4y ago

Most excellent! If Github supported it alongside markdown, this would drive the markdown parsers to support this as a flavor, and then the community could decide; it will be fascinating to analyze the open-source population. I'm finding dozens of applications for a lingua franca of mildly-structured, human-readable language that transforms as easily into data as documents and UI's. I've built out 10+ markdown applications using 3-4 parsers, and always had to reduce the effective semantics to a small subset of features, and then had to use metadata or convention to do what attributes will do much more cleanly. As for Lua: yes it's the best for integrating into pandoc performance-wise, but ay! the table structures and metadata tables to get anything like typing are mind bending, and I am still miffed at being blocked at a critical time by a bug in pandoc's lua initializer for tables. Aside language/stdlib developers, jgm is my greatest benefactor, and he deserves a cadre of loyal and competent implementors.

__tmk__ 4y ago

Wow, this is all I ever wanted. The only thing that's maybe missing is a natural way to do captions for images and tables, and a syntax for spoilers. But these can probably be built on top of the div and span syntax.

deafpolygon 4y ago

After doing Markdown for a while... and discovering Org. I'll keep Org. Thanks.