4 ms·
I did run into some problems with nbconvert from time to time. It's worth noting that nbdev doesn't use nbconvert at all - it uses Quarto instead (which AFAICT
by jph00 4y ago
I did run into some problems with nbconvert from time to time. It's worth noting that nbdev doesn't use nbconvert at all - it uses Quarto instead (which AFAICT does everything nbconvert does plus a lot more, and is faster, more extensible, and more "batteries included").
Having said that, it's possible that, given your experiences with needing to re-run some slow cells and having trouble making that work well, you might prefer to use "pure Quarto" instead of nbdev. With Quarto you can write your report as a .qmd file directly: https://quarto.org/ https://quarto.org/ .
Personally, I quite like the notebook environment for situations like this where there are some really slow cells -- I mainly do deep learning, and some of my cells take many hours to run -- since that state is cached and I can easily manipulate it and visualise it afterwards. I generally will then add some kind of serialization or caching once it's working so I don't have to re-run the slow bits every time. I'll often also use nbdev to export a .py script from the notebook so it's easy to re-run the whole thing from scratch.
(BTW we also released something today that's particularly helpful for this workflow: https://fastai.github.io/execnb/ https://fastai.github.io/execnb/ . Basically, it's a parameterised notebook runner. It doesn't rely on Jupyter or nbclient or nbconvert. It's in the same general category as Papermill, but it's much more lightweight and requires learning far fewer new concepts.)