3 ms·
It looks like the final solution will have to run notebooks twice - once to check for errors and once more to render to documentation. The concept of running c
by wodenokoto 2y ago
It looks like the final solution will have to run notebooks twice - once to check for errors and once more to render to documentation.
The concept of running code examples inside documentation as a part of tests is well known, and extending it to end-to-end tests / user guides is a good idea.
Next step might be to add hidden code cells with asserts, to check that the code not only runs, but creates the expected output.
- freetonik 2y agoThe documentation is rendered without running the notebook in my case, because we store notebooks with all the outputs, so the doc builder (Sphinx) just converts them to HTML as is.
- wodenokoto 2y agoTo me, the reasonable path would be to have the CI built the notebook, check it for errors and then render it to the docs. As per the article, they have to run the notebook locally, commit all the outputs, then the CI checks if the notebook can run, and then renders the _committed_ output to the docs. This means that the verified code and output can be out of sync, partly defeating the purpose of the CI.
- foundart 2y agoHaving the user guides be notebooks is a very interesting idea and the step from there to e2e test as the author describes seems like a very logical one.
- foundart 2y agoThe short article is definitely worth a read, as is the Netflix post it links to