3 ms·

nbconvert/pandoc is very limited in how it can be extended because it only relies on jinja templating rather than a proper AST. You couldn't achieve a lot of th

nbconvert/pandoc is very limited in how it can be extended because it only relies on jinja templating rather than a proper AST. You couldn't achieve a lot of the things that sphinx/jupyter book can now do with that. Trust me I know because, before starting work on https://github.com/executablebooks/MyST-Parser https://github.com/executablebooks/MyST-Parser I used the nbconvert/pandoc approach to create https://github.com/chrisjsewell/ipypublish https://github.com/chrisjsewell/ipypublish ;-)

zdw 6y ago
This looks great, as extended commonmark support knocks down the last major pain point to switching to Sphinx for docs in the projects I've worked on. The only remaining tooling gap is a markdown linter implemented in python.
- dmix 6y ago
  Why a python linter? Text editors are usually language agnostic and separate from other tooling.
  - zdw 6y ago
    If the rest of the documentation workflow is in python, it would remove the need to install another entirely separate scripting language runtime to do those checks on every developer laptop, CI system, etc.
heisenzombie 6y ago
This is interesting, so do you think the JupyterBook solution could/should replace nbconvert? I was always a bit frustrated by the way nbconvert pulled in pandoc as a dependency, but then didn't make use of any of its power (pandoc surely has a powerful AST). Instead they re-implemented filters and templates themselves. It struck me that it would have been much cleaner to write a proper 'ipynb > AST' Pandoc reader, and then expose pandoc AST filters and templates. I was a bit suspicious of moving away from pandoc to sphinx, because sphinx seems markedly less powerful and general than pandoc. However, you seem to think it's the opposite! Does Sphinx have an AST? I can't see any reference?
- chrisjsewell 6y ago
  pandocs AST is actually a little limited, in terms of extensability, because it has to resove to JSON. The scorreia.com/software/panflute/code.html is a good python API for it. By contrast sphinx uses docutils AST: https://github.com/live-clones/docutils/blob/master/docutils/docutils/nodes.py https://github.com/live-clones/docutils/blob/master/docutils..., the documentation is not great (working on that) but it is pretty powerful.
- fiddlosopher 6y ago
  In fact, pandoc now does have an ipynb reader and writer. You can convert between ipynb and any of the formats pandoc supports, and you can manipulate the AST using filters.
  - chrisjsewell 6y ago
    Now use that to replicate the example pages here: https://myst-nb.readthedocs.io/en/latest/examples/basic.html#markdown https://myst-nb.readthedocs.io/en/latest/examples/basic.html... and here https://jupyterbook.org/interactive/launchbuttons.html https://jupyterbook.org/interactive/launchbuttons.html, and let me know how you get on ;-)
  - heisenzombie 6y ago
    Oh! Thanks! I'll definitely take a look at that!
  - tikej 6y ago
    Oh, this is huge for me! My main concern with jupyter is that I prefer orgmode (and by a lot), but I know that it's niche and most people don't even heard about it ipnbs are preferred form of notebook exchange. I will check how good translation to/from orgmode is then.

zdw 6y ago

This looks great, as extended commonmark support knocks down the last major pain point to switching to Sphinx for docs in the projects I've worked on. The only remaining tooling gap is a markdown linter implemented in python.

dmix 6y ago
Why a python linter? Text editors are usually language agnostic and separate from other tooling.
- zdw 6y ago
  If the rest of the documentation workflow is in python, it would remove the need to install another entirely separate scripting language runtime to do those checks on every developer laptop, CI system, etc.

heisenzombie 6y ago

This is interesting, so do you think the JupyterBook solution could/should replace nbconvert? I was always a bit frustrated by the way nbconvert pulled in pandoc as a dependency, but then didn't make use of any of its power (pandoc surely has a powerful AST). Instead they re-implemented filters and templates themselves. It struck me that it would have been much cleaner to write a proper 'ipynb > AST' Pandoc reader, and then expose pandoc AST filters and templates. I was a bit suspicious of moving away from pandoc to sphinx, because sphinx seems markedly less powerful and general than pandoc. However, you seem to think it's the opposite! Does Sphinx have an AST? I can't see any reference?

chrisjsewell 6y ago
pandocs AST is actually a little limited, in terms of extensability, because it has to resove to JSON. The scorreia.com/software/panflute/code.html is a good python API for it. By contrast sphinx uses docutils AST: https://github.com/live-clones/docutils/blob/master/docutils/docutils/nodes.py https://github.com/live-clones/docutils/blob/master/docutils..., the documentation is not great (working on that) but it is pretty powerful.
fiddlosopher 6y ago
In fact, pandoc now does have an ipynb reader and writer. You can convert between ipynb and any of the formats pandoc supports, and you can manipulate the AST using filters.
- chrisjsewell 6y ago
  Now use that to replicate the example pages here: https://myst-nb.readthedocs.io/en/latest/examples/basic.html#markdown https://myst-nb.readthedocs.io/en/latest/examples/basic.html... and here https://jupyterbook.org/interactive/launchbuttons.html https://jupyterbook.org/interactive/launchbuttons.html, and let me know how you get on ;-)
- heisenzombie 6y ago
  Oh! Thanks! I'll definitely take a look at that!
- tikej 6y ago
  Oh, this is huge for me! My main concern with jupyter is that I prefer orgmode (and by a lot), but I know that it's niche and most people don't even heard about it ipnbs are preferred form of notebook exchange. I will check how good translation to/from orgmode is then.