4 ms·

(A frequent Jupyter Notebook user here. For data exploration, and teaching deep learning - then Colab is indispensable.) The main question is: what are the alt

by stared 7y ago

(A frequent Jupyter Notebook user here. For data exploration, and teaching deep learning - then Colab is indispensable.) The main question is: what are the alternatives, for data exploration (and sharing its results). Similarly, for data science tool demos, Notebooks shine. IMHO the problem is not in the notebooks, but in how they are being used (i.e. the workflow). By writing scripts in py files, and using notebooks only to show their results (processed data, charts, etc) we get the best of both worlds. The only build-in problem with Jupyter Notebooks is JSON, mixing input and output (and making it pain to work with version control). But here RMarkdown (and a few other alternatives) work well.

fhennig 7y ago
Yes, the article mentions users copy pasting snippets from their personal "library". Well, that could just be made into an actual library of functions to call. I'm currently at uni enrolled in an AI/ML degree, and there are a lot of people with no previous exposure to programming. It's just that most people don't know that these things are possible, don't want to learn another tool (IDE) and are not interested in longevity of the code, just in the results. This shouldn't sound like me complaining, I totally understand. I think a lot of the stuff could be solved with just better tooling, but a familiarity with software development is definitely helpful. Also a while back streamlit (https://www.streamlit.io/ https://www.streamlit.io/) was here on HN and since then I've been meaning to try it. I think this could be a good approach to bring together the best of both worlds.
- sillysaurusx 7y ago
  most people don't know that these things are possible, don't want to learn another tool (IDE) and are not interested in longevity of the code, just in the results. This approach is arguably more effective than wasting time trying to refactor everything into a library of functions. Programming-as-crafting needs to be more of a thing. Not everything is written to be long-lasting. Even HN ushered the ugly code into hook functions that weren't shipped with the main codebase.
karlicoss 7y ago
I recommend nbstripout https://github.com/kynan/nbstripout https://github.com/kynan/nbstripout It eases most of the pain regarding version control. You can use it as a 'git filter', so only inputs would be shown in diffs and committed (and also works with interactive adding!), while keeping outputs in your working tree.
- potatohead00 7y ago
  Came here just to make sure this got mentioned!
amirathi 7y ago
> The only build-in problem with Jupyter Notebooks is JSON, mixing input and output (and making it pain to work with version control) Absolutely. JSON notebook format makes it very hard to do code reviews, merge in remote changes etc. After being frustrated with lack of solutions, I built ReviewNB[1] specifically to do notebook code reviews on GitHub. Alternatively, Nbdime[2] is also a nice open source library to see diffs locally & merge in changes. [1] https://www.reviewnb.com/ https://www.reviewnb.com/ [2] https://github.com/jupyter/nbdime https://github.com/jupyter/nbdime

by stared 7y ago

fhennig 7y ago

Yes, the article mentions users copy pasting snippets from their personal "library". Well, that could just be made into an actual library of functions to call. I'm currently at uni enrolled in an AI/ML degree, and there are a lot of people with no previous exposure to programming. It's just that most people don't know that these things are possible, don't want to learn another tool (IDE) and are not interested in longevity of the code, just in the results. This shouldn't sound like me complaining, I totally understand. I think a lot of the stuff could be solved with just better tooling, but a familiarity with software development is definitely helpful. Also a while back streamlit (https://www.streamlit.io/ https://www.streamlit.io/) was here on HN and since then I've been meaning to try it. I think this could be a good approach to bring together the best of both worlds.

sillysaurusx 7y ago
most people don't know that these things are possible, don't want to learn another tool (IDE) and are not interested in longevity of the code, just in the results. This approach is arguably more effective than wasting time trying to refactor everything into a library of functions. Programming-as-crafting needs to be more of a thing. Not everything is written to be long-lasting. Even HN ushered the ugly code into hook functions that weren't shipped with the main codebase.

karlicoss 7y ago

I recommend nbstripout https://github.com/kynan/nbstripout https://github.com/kynan/nbstripout It eases most of the pain regarding version control. You can use it as a 'git filter', so only inputs would be shown in diffs and committed (and also works with interactive adding!), while keeping outputs in your working tree.

potatohead00 7y ago
Came here just to make sure this got mentioned!

amirathi 7y ago

> The only build-in problem with Jupyter Notebooks is JSON, mixing input and output (and making it pain to work with version control) Absolutely. JSON notebook format makes it very hard to do code reviews, merge in remote changes etc. After being frustrated with lack of solutions, I built ReviewNB[1] specifically to do notebook code reviews on GitHub. Alternatively, Nbdime[2] is also a nice open source library to see diffs locally & merge in changes. [1] https://www.reviewnb.com/ https://www.reviewnb.com/ [2] https://github.com/jupyter/nbdime https://github.com/jupyter/nbdime