3 ms·

Last year we changed the source format for MDN from some extremely messy, WYSIWYG-authored HTML to something that would be easier for authors to use. We conside

Last year we changed the source format for MDN from some extremely messy, WYSIWYG-authored HTML to something that would be easier for authors to use. We considered Asciidoc and reST, and despite its limitations, we chose Markdown (GFM specifically) for two reasons: 1. We get a _lot_ of casual contributors to MDN: about 180-200 unique contributors/month, most of whom we never see again. Almost all of them can contribute much more easily with Markdown than with anything else. Many of these people are unlikely to put even an half an hour into learning a new syntax. 2. Markdown has great tooling support. For example, if we want to run Prettier over our embedded code samples, it's really easy if we are in Markdown. If we are in Markdown we will get nice formatting just about everywhere, including GitHub of course and most people's editors. One thing that made the choice easier for us is that MDN's a very mature doc site, so we had a very good idea of which Markdown limitations were likely to be a problem for us. If you're interested, we blogged about this project: https://openwebdocs.org/content/posts/markdown-conversion/ https://openwebdocs.org/content/posts/markdown-conversion/.

BiteCode_dev 4y ago
You made the right choice, rst is not user friendly, even for tech savvy people. I'll take the opportunity to do a shameless plug for the amazing "Markedly Structured Text", or MyST, a markdown flavor that is both easy to write like markdown, and expressive like rst. Basically, if you know markdown, you can write decent MyST already. In fact, any markdown is valid MyST and .md is a valid file extension for MyST. Once you want more, the syntax offers richer construct, but still easy enough to use. Check it out: https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#syntax-core https://myst-parser.readthedocs.io/en/latest/syntax/syntax.h... I wanted to love asciidoc because the format rocks, but it never took off and the tooling is still lacking. MyST piggy back on markdown ecosystem, this is less of a problem. Also, the devs behind it also have a very good track record at creating a good FOSS ecosystem.
- mdaniel 4y ago
  https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#show-backticks-inside-raw-markdown-blocks https://myst-parser.readthedocs.io/en/latest/syntax/syntax.h... is excellent, and one of my major pain points. I usually give up and just use ` when I need to talk about backticking in markdown but that idea is much, much better but I think https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#comments https://myst-parser.readthedocs.io/en/latest/syntax/syntax.h... is misguided; AFAIK  is legal in markdown and far less likely to surprise someone, both with an abnormal comment character as well as the forced block break
  - uranusjr 4y ago
    The “one more backtick” syntax is a very early Markdown feature explicitly mentioned by John Gruber (originally only for inline code, of course). It might even be in the original “spec”, I don’t remember, but it dates way back and is implemented by virtually all half-decent Markdown parsers. It’s one of the least utilised feature, however, so huge kudos for MyST to clearly explain it and raise awareness.
- josephg 4y ago
  Oh that’s lovely. Does it also support note blocks (like are used to render that page)? And are there implementations in other languages?
  - BiteCode_dev 4y ago
    Yes, all admonitions (note, warning, danger) are supported, and worst case scenario, you can fallback on inlined rst if you really miss something. But so far I never needed to. > And are there implementations in other languages? That's the probably the biggest limition for myst: for now it's very new and is only implemented in Python. I supposed it's a good opportunity to learn rust by creating a parser :)
vedranm 4y ago
I work in an academic setting and I can second the sentiment. For a while, we used reStructuredText for writing the teaching materials. Every so often I would have the students that would get inspired to contribute something to the teaching materials, but would subsequently get demotivated by having to learn the rST syntax and tooling. After a few years, I gave up and switched from rST and Sphinx to Markdown and MkDocs [2]. We addressed the limitations of Markdown with PyMdown Extensions [3]. Still haven't looked back; for our specific use case of writing (computer science) teaching materials, Markdown is a better choice than rST. [1] https://gaseri.org/en/blog/2017-07-29-why-we-use-restructuredtext-and-sphinx-static-site-generator-for-maintaining-teaching-materials/ https://gaseri.org/en/blog/2017-07-29-why-we-use-restructure... [2] https://gaseri.org/en/blog/2021-08-16-markdown-vs-restructuredtext-for-teaching-materials/ https://gaseri.org/en/blog/2021-08-16-markdown-vs-restructur... [3] https://facelessuser.github.io/pymdown-extensions/ https://facelessuser.github.io/pymdown-extensions/
- wbamberg 4y ago
  Thanks @vedranm! I especially like your side-by-side comparison of the process of contributing using Markdown versus reST. It really encapsulates the difference that reasonably seamless tool support makes. I need to look more into MkDocs...
  - vedranm 4y ago
    You are welcome. I have no connection to its development aside from some minor translation updates and bug reports, but I can only recommend it. What I like is how easy it is to host MkDocs on GitHub Pages via built-in support, but you can even make it behave like Jekyll with GitHub Actions. Shameless self-plug of another blog post: https://gaseri.org/en/blog/2022-11-01-publishing-material-for-mkdocs-website-to-github-pages-using-custom-actions-workflow/ https://gaseri.org/en/blog/2022-11-01-publishing-material-fo...
russellbeattie 4y ago
I still cannot believe MDN decided to abandon web standards for the garbage that is Markdown. Worse yet, they seem proud of themselves for doing it. Rather than actually solve the problem of filtering WYSIWYG input/output, providing a solution for creating easy to create and maintain semantic HTML, Mozilla shunted it aside and now use a stack of custom tooling and libraries and yet another Markdown-alike variant text format with their own scripting embeds and then patted themselves on the backs for all the effort they "saved". Mozilla is supposed to be the standard bearer for web standards. It's sad to see that bunch of myopic techies decided to go with a trendy solution that just trashed a core principle of the organization (#6) for one of its most important products. It set an example others are now following, and as a result, MDN put the web back probably two decades.
- nine_k 4y ago
  Rather than embarking on something that proved to be hard by decades of trying, or inventing something complex and unique, Mozilla took a less powerful existing tool, very simple and well known to every contributor. They acted as if they tried to make it simple and easy for the community to keep MDN up and up to date. Outrageous!
  - hakre 4y ago
    Well, by the book and its origins, there is no improvement just because in Markdown you can use any HTML. Markdown itself is part of the try when you follow the path of its history and when marketed as a subset, it means (and that approach may be valid), let us go back and reduce to the early set of HTML tags.
    - nine_k 4y ago
      I wrote enough basic HTML tags back in 1995 to say that writing _this way_ is way more ergonomic than <i>this way</i>. The thing is that HTML (and SGML) was invented, but Markdown was discovered as a set of best practices through decades of text-only mailing lists. This is something that many people found natural enough.

BiteCode_dev 4y ago

You made the right choice, rst is not user friendly, even for tech savvy people. I'll take the opportunity to do a shameless plug for the amazing "Markedly Structured Text", or MyST, a markdown flavor that is both easy to write like markdown, and expressive like rst. Basically, if you know markdown, you can write decent MyST already. In fact, any markdown is valid MyST and .md is a valid file extension for MyST. Once you want more, the syntax offers richer construct, but still easy enough to use. Check it out: https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#syntax-core https://myst-parser.readthedocs.io/en/latest/syntax/syntax.h... I wanted to love asciidoc because the format rocks, but it never took off and the tooling is still lacking. MyST piggy back on markdown ecosystem, this is less of a problem. Also, the devs behind it also have a very good track record at creating a good FOSS ecosystem.

mdaniel 4y ago
https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#show-backticks-inside-raw-markdown-blocks https://myst-parser.readthedocs.io/en/latest/syntax/syntax.h... is excellent, and one of my major pain points. I usually give up and just use ` when I need to talk about backticking in markdown but that idea is much, much better but I think https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#comments https://myst-parser.readthedocs.io/en/latest/syntax/syntax.h... is misguided; AFAIK  is legal in markdown and far less likely to surprise someone, both with an abnormal comment character as well as the forced block break
- uranusjr 4y ago
  The “one more backtick” syntax is a very early Markdown feature explicitly mentioned by John Gruber (originally only for inline code, of course). It might even be in the original “spec”, I don’t remember, but it dates way back and is implemented by virtually all half-decent Markdown parsers. It’s one of the least utilised feature, however, so huge kudos for MyST to clearly explain it and raise awareness.
josephg 4y ago
Oh that’s lovely. Does it also support note blocks (like are used to render that page)? And are there implementations in other languages?
- BiteCode_dev 4y ago
  Yes, all admonitions (note, warning, danger) are supported, and worst case scenario, you can fallback on inlined rst if you really miss something. But so far I never needed to. > And are there implementations in other languages? That's the probably the biggest limition for myst: for now it's very new and is only implemented in Python. I supposed it's a good opportunity to learn rust by creating a parser :)

vedranm 4y ago

I work in an academic setting and I can second the sentiment. For a while, we used reStructuredText for writing the teaching materials. Every so often I would have the students that would get inspired to contribute something to the teaching materials, but would subsequently get demotivated by having to learn the rST syntax and tooling. After a few years, I gave up and switched from rST and Sphinx to Markdown and MkDocs [2]. We addressed the limitations of Markdown with PyMdown Extensions [3]. Still haven't looked back; for our specific use case of writing (computer science) teaching materials, Markdown is a better choice than rST. [1] https://gaseri.org/en/blog/2017-07-29-why-we-use-restructuredtext-and-sphinx-static-site-generator-for-maintaining-teaching-materials/ https://gaseri.org/en/blog/2017-07-29-why-we-use-restructure... [2] https://gaseri.org/en/blog/2021-08-16-markdown-vs-restructuredtext-for-teaching-materials/ https://gaseri.org/en/blog/2021-08-16-markdown-vs-restructur... [3] https://facelessuser.github.io/pymdown-extensions/ https://facelessuser.github.io/pymdown-extensions/

wbamberg 4y ago
Thanks @vedranm! I especially like your side-by-side comparison of the process of contributing using Markdown versus reST. It really encapsulates the difference that reasonably seamless tool support makes. I need to look more into MkDocs...
- vedranm 4y ago
  You are welcome. I have no connection to its development aside from some minor translation updates and bug reports, but I can only recommend it. What I like is how easy it is to host MkDocs on GitHub Pages via built-in support, but you can even make it behave like Jekyll with GitHub Actions. Shameless self-plug of another blog post: https://gaseri.org/en/blog/2022-11-01-publishing-material-for-mkdocs-website-to-github-pages-using-custom-actions-workflow/ https://gaseri.org/en/blog/2022-11-01-publishing-material-fo...

russellbeattie 4y ago

I still cannot believe MDN decided to abandon web standards for the garbage that is Markdown. Worse yet, they seem proud of themselves for doing it. Rather than actually solve the problem of filtering WYSIWYG input/output, providing a solution for creating easy to create and maintain semantic HTML, Mozilla shunted it aside and now use a stack of custom tooling and libraries and yet another Markdown-alike variant text format with their own scripting embeds and then patted themselves on the backs for all the effort they "saved". Mozilla is supposed to be the standard bearer for web standards. It's sad to see that bunch of myopic techies decided to go with a trendy solution that just trashed a core principle of the organization (#6) for one of its most important products. It set an example others are now following, and as a result, MDN put the web back probably two decades.

nine_k 4y ago
Rather than embarking on something that proved to be hard by decades of trying, or inventing something complex and unique, Mozilla took a less powerful existing tool, very simple and well known to every contributor. They acted as if they tried to make it simple and easy for the community to keep MDN up and up to date. Outrageous!
- hakre 4y ago
  Well, by the book and its origins, there is no improvement just because in Markdown you can use any HTML. Markdown itself is part of the try when you follow the path of its history and when marketed as a subset, it means (and that approach may be valid), let us go back and reduce to the early set of HTML tags.
  - nine_k 4y ago
    I wrote enough basic HTML tags back in 1995 to say that writing _this way_ is way more ergonomic than <i>this way</i>. The thing is that HTML (and SGML) was invented, but Markdown was discovered as a set of best practices through decades of text-only mailing lists. This is something that many people found natural enough.