2 ms·
> Crafting workflows out of notebooks is a really bad idea; an anti-pattern. If you want to go down the road of "workflows for data scientists" https://westur
by westurner 4y ago
> Crafting workflows out of notebooks is a really bad idea; an anti-pattern. If you want to go down the road of "workflows for data scientists"
https://westurner.github.io/hnlog/ https://westurner.github.io/hnlog/ Ctrl-F "DVC" ( https://dvc.org/ https://dvc.org/ ) , https://westurner.github.io/hnlog/#comment-24261118 https://westurner.github.io/hnlog/#comment-24261118 "Ten Simple Rules for Reproducible Computational Research", “Ten Simple Rules for Creating a Good Data Management Plan”, PROV
pygwalker https://github.com/Kanaries/pygwalker https://github.com/Kanaries/pygwalker :
> PyGWalker: Turn your pandas dataframe into a Tableau-style User Interface for visual analysis
"Generate code from GUI interactions; State restoration & Undo" https://github.com/Kanaries/pygwalker/issues/90 https://github.com/Kanaries/pygwalker/issues/90
- westurner 4y agoThe Scientific Method is testing, so testing (tests, assertions, fixtures) should be core to any scientific workflow system. - [ ] (It's not possible to run `!pytest` in a Jupyter notebook without installing an extension with JupyterLite in WASM onnly where there's not yet a terminal or even yet a slow-but-usable [cheerpx] webvm bridged to jupyter kernel WASM ~process-space.) awesome-jupyter#testing: https://github.com/markusschanta/awesome-jupyter#testing https://github.com/markusschanta/awesome-jupyter#testing ml-tooling/best-of-jupyter lists papermill/papermill under "Interactive Widgets/Visualization" https://github.com/ml-tooling/best-of-jupyter#interactive-widgets--visualization https://github.com/ml-tooling/best-of-jupyter#interactive-wi... "Markdown based notebooks" would store files next to the .ipynb.md, which implies a need for an MHTML/ZIP-like archive (for report notebook artifacts produced by scientific workflow systems with provenance metadata); but W3C Web Bundles avoid modifying linked resources with new specs: https://github.com/jupyter/enhancement-proposals/pull/103#issuecomment-1510418219 https://github.com/jupyter/enhancement-proposals/pull/103#is...