3 ms·
Your experience aligns with what I've seen. All the notebooks are in one place. Some are for important production jobs, other are for data exploration. It's
by MrPowers 6y ago
Your experience aligns with what I've seen.
All the notebooks are in one place. Some are for important production jobs, other are for data exploration.
It's easy to make a little edit in a notebook and accidentally break production jobs.
Comparatively harder to make an edit in a git repo and do a deploy that'll break production jobs (e.g. if the JAR doesn't compile or the CI errors out cause the tests don't pass).
Notebook based production jobs get even more dangerous when NotebookA depends on NotebookB and so on.
- alexott 6y agoTreat notebooks like other code - separate onto staging and production, with defined promotions between them - it’s possible. You can run tests in CI/CD pipelines, etc. You can set permissions so nobody can update production notebooks manually, ...