2 ms·
> Yeah but jupyter notebooks suck at providing reproducible data science. Why? I have no problem with reproducibility when I use a little bit of discipline. Y
by kortex 3y ago
> Yeah but jupyter notebooks suck at providing reproducible data science.
Why? I have no problem with reproducibility when I use a little bit of discipline.
Your workflow does indeed sound nice but also sounds like it involves way more tooling and institutional knowledge. Anywhere I can learn more about it or see the scripts you use?
- steve_gh 3y agoI completely agree that a bit of discipline does wonders. But it is not just your discipline, it is the team's discipline. What I need to ensure is that anyone picking up a piece of analysis 3 months later can reproduce exactly what was done. I've been burnt in the past by having to go back to the original analyst and be told "oh you run this bit of this notebook, then paste the results in over here, then run that". By insisting that everything is scripted and that there are no manual steps, we get a reproducible analytics pipeline. The starting point for our methodology is the book "Guerilla Analytics" by Enda Ridge. It's worth reading.