5 ms·

This is a good point. On the other hand, developer productivity in notebook environments is so high that iterating your way to correct code may be faster than

This is a good point. On the other hand, developer productivity in notebook environments is so high that iterating your way to correct code may be faster than trying to plan it all out in the beginning. There is also the aspect of exploration. The hardest part of most data programmer projects is understanding the data and optimal wrangling.

smohare 5y ago
This is something I often wonder about. In my own experience the perceived productivity gains in prototyping often come with externalized technical debt that lead to an overall loss of productivity. Notebook-driven development tends to produce models that are too tightly coupled with an underlying data pipeline, as the workflow very much facilitates this. There’s a measured loss of productivity when a model and pipeline need to be decoupled, e.g., to split the pipeline or repurpose the model. Obviously coupling components is not unique to notebooks, but there just seems to be a culture in notebook-driven development to disproportionately push poorly designed code artifacts. The MLEs and data scientist I’ve worked with that write simple functional modules tend to introduce less technical debt, at the expense sometimes of taking longer to push out a demo prototype. All that said, I think the aforementioned issues are not truly intrinsic to a notebook-driven development, but rather an emerging culture that has rather aptly been called Potemkin data science: https://mcorrell.medium.com/potemkin-data-science-fba2b5ba5cc6 https://mcorrell.medium.com/potemkin-data-science-fba2b5ba5c...
dash2 5y ago
>On the other hand, developer productivity in notebook environments is so high that iterating your way to correct code may be faster than trying to plan it all out in the beginning. In science, what kills you isn't the mistake you see and correct. It's the mistake you don't see, that then gets published.
- da39a3ee 5y ago
  Interesting, but not true. You clearly don't work in science: the main thing is getting publications. Mostly people don't pay that much attention to them, so if there are errors, (a) probably no one will notice and (b) you can always explain it away with some obfuscation.
  - dash2 5y ago
    I do work in science, and I care about getting things right.
da39a3ee 5y ago
> On the other hand, developer productivity in notebook environments is so high that iterating your way to correct code may be faster than trying to plan it all out in the beginning. That's not a fair comparison though is it? Those of us who use git, the vast majority of the time we iterate our way to correct code; we don't plan it out at the beginning. It's just that we iterate over the course of a sequence of commits, pausing on each commit to study application behavior. Notebooks users also iterate over a sequence of code evolutions, except mostly without systematic checkpointing and without clearing in-memory state between code changes.

smohare 5y ago

This is something I often wonder about. In my own experience the perceived productivity gains in prototyping often come with externalized technical debt that lead to an overall loss of productivity. Notebook-driven development tends to produce models that are too tightly coupled with an underlying data pipeline, as the workflow very much facilitates this. There’s a measured loss of productivity when a model and pipeline need to be decoupled, e.g., to split the pipeline or repurpose the model. Obviously coupling components is not unique to notebooks, but there just seems to be a culture in notebook-driven development to disproportionately push poorly designed code artifacts. The MLEs and data scientist I’ve worked with that write simple functional modules tend to introduce less technical debt, at the expense sometimes of taking longer to push out a demo prototype. All that said, I think the aforementioned issues are not truly intrinsic to a notebook-driven development, but rather an emerging culture that has rather aptly been called Potemkin data science: https://mcorrell.medium.com/potemkin-data-science-fba2b5ba5cc6 https://mcorrell.medium.com/potemkin-data-science-fba2b5ba5c...

dash2 5y ago

da39a3ee 5y ago
Interesting, but not true. You clearly don't work in science: the main thing is getting publications. Mostly people don't pay that much attention to them, so if there are errors, (a) probably no one will notice and (b) you can always explain it away with some obfuscation.
- dash2 5y ago
  I do work in science, and I care about getting things right.

da39a3ee 5y ago

> On the other hand, developer productivity in notebook environments is so high that iterating your way to correct code may be faster than trying to plan it all out in the beginning. That's not a fair comparison though is it? Those of us who use git, the vast majority of the time we iterate our way to correct code; we don't plan it out at the beginning. It's just that we iterate over the course of a sequence of commits, pausing on each commit to study application behavior. Notebooks users also iterate over a sequence of code evolutions, except mostly without systematic checkpointing and without clearing in-memory state between code changes.