4 ms·
One of my most upvoted comments says something to the effect of "notebooks bad", so you're preaching to the choir here -- however: - I work with several people
by qsort 4y ago
One of my most upvoted comments says something to the effect of "notebooks bad", so you're preaching to the choir here -- however:
- I work with several people who are purely data scientists, and I lean on "culture" rather than "Jupyter". In some circles, probably influenced by academia, programming is considered to be low status work. You are not going to solve the problem by switching to .py files, even though for most tasks literally anything is better than Jupyter.
That they don't use git, or that git wasn't originally even a concern, is a consequence of that low-status perception. If you pitched something to the developer community and told them "oh, by the way, you can't use git", they'd synthesize tomatoes out of thin air to throw at you.
- I'd carve an exception for stuff that satisfies ALL of the following: (a) is self-contained in a single notebook, (b) has no dependencies on anything non-standard, (c) is demonstrative in nature, or a personal exercise rather than production software. For example, I wrote my solutions to the Advent of Code problems in a notebook and I liked the experience, especially how you could mix math and code.
- HuwFulcher 4y agoI would also lean more towards culture and the environment that Jupyter provides only perpetuates it. I think what made me leave Data Science in the end was that I wasn’t driven by work outside of the notebooks (i.e. coming up with a mathematically superior solution) but driven by building ML driven systems as a whole. I think notebooks are a great way of presenting findings and showing your workings at the same time. If that was their main use then I wouldn’t have any issues
- gradschoolfail 4y agoI would say that in academia, explorability and immediacy is way prioritized over reproducibility and maintainability.. Two kinds of human beings?
- qsort 4y agoThe incentives in academia are different. The objective is to publish, code is important only insofar as it allows you to achieve that goal. This is not to say that academics can't code, but even if you are a professor who cares passionately about making high-quality software, you're fighting uphill, because that's not what you're being evaluated on. If you want to make the argument about "two kinds of people", I think it's more about A-type/B-type data scientists in the industry. I'm really mostly a developer and not a data scientist, but when I assist in DS tasks I wear a distinct B-type hat, and that informs my perspective. A-type people have different priorities and that's fine; my gripe is when you try to import A-type practices in a B-type scenario.