5 ms·

It's probably an unrelated post (apologies in advance) but I wanted to shoutout to the Marimo (https://marimo.io https://marimo.io), it's the only Jupyter alter

by _l7dh 2y ago

It's probably an unrelated post (apologies in advance) but I wanted to shoutout to the Marimo (https://marimo.io https://marimo.io), it's the only Jupyter alternative that really got me excited, it's like Streamlit and Jupyter had a kid (and the kid took the best genes from both).

CraigJPerry 2y ago
>> marimo notebooks are pure Python and stored as .py files That sounds like a solid improvement. I’m going to give this a test drive. I feel like modularity is one of the hardest aspects of Jupyter notebooks in a team environment. I’d be interested to hear if anyone has cracked a workflow with notebooks for larger teams. Notebooks are easy for solo or very small teams, and the literate programming style benefits still apply in larger teams but there’s a lot of friction: “hey just %run this shared notebook with a bunch of useful utilities in it - oops yeah it tries to write some files because of some stuff unrelated to your use case in there (that’s essential to my use case)” My current best that I know of is to keep “calculation” (pure) code in a .py and just the “action“ (side-effectful) code in the notebook. Then as far as physically possible, keep the data outside of notebook (usually a database or csv’s). That helps avoid the main time sink pitfalls (resolving git conflicts, versioning, testing etc) but it doesn’t solve for example tooling you might want to run - maybe mypy against that action code - sure you can use nbqa but… interested to learn better approaches.
- szvsw 2y ago
  I actually think the problem you are describing is actually sometimes helpful from a design perspective, if you can be conscientious enough to periodically review your notebooks and figure out what is the actual useful code which should be properly integrated into the codebase vs what is the “one-off” / non-modular code. Like you mentioned, calculation vs side-effects is one way to help you decide but not the only. There’s definitely no single answer. The key is to just periodically figure out what ought to be refactored into library code, which notebooks should just be straight up deleted (hopefully as many as possible - you can always get them back in your git history if needed!), and so on.
- qsort 2y ago
  I'm mostly in the camp that notebooks aren't that great for software development, they thrive as an "excel for coders" of sorts, but take a look at nbdev from fast.ai. The literate programming aspect is very nice and I wish it was explored more.
ThouYS 2y ago
marimo is really cool, albeit "pure python" is only true insofar as the diff is concerned. other than that, it's an unconnected group of functions that need the marimo runtime to stitch together. would be cool if marimo could "unroll" the compute graph into a standalone python script that doesn't need the marimo library
- mscolnick 2y ago
  it’s already possible to do this: marimo export script nb.py Pure-python also helps to work with existing tools out of the box: formatting, linting, pytest, importing notebooks as modules, composition, PEP 723 inline metadata
  - 3eb7988a1663 2y ago
    If you are going to do that, you could stick with Jupyter + nbconvert. I rarely use notebooks directly anymore unless I require the output to be stored. Do most everything in VSCode with interactive .py files. Gets you the same notebook-y experience + all of the Python tooling.
    - akshayka 2y ago
      marimo notebooks are actually DAGs as cells, reusable out-of-the-box in three diffferent ways: as interactive computing notebooks in a reactive environment with no hidden state, as Python scripts, as data apps. So Jupyter + nbconvert (or perhaps you meant jupytext) is not a replacement for marimo.
  - deleted 2y ago
    [deleted]
- deleted 2y ago
  [deleted]

by _l7dh 2y ago

CraigJPerry 2y ago

>> marimo notebooks are pure Python and stored as .py files That sounds like a solid improvement. I’m going to give this a test drive. I feel like modularity is one of the hardest aspects of Jupyter notebooks in a team environment. I’d be interested to hear if anyone has cracked a workflow with notebooks for larger teams. Notebooks are easy for solo or very small teams, and the literate programming style benefits still apply in larger teams but there’s a lot of friction: “hey just %run this shared notebook with a bunch of useful utilities in it - oops yeah it tries to write some files because of some stuff unrelated to your use case in there (that’s essential to my use case)” My current best that I know of is to keep “calculation” (pure) code in a .py and just the “action“ (side-effectful) code in the notebook. Then as far as physically possible, keep the data outside of notebook (usually a database or csv’s). That helps avoid the main time sink pitfalls (resolving git conflicts, versioning, testing etc) but it doesn’t solve for example tooling you might want to run - maybe mypy against that action code - sure you can use nbqa but… interested to learn better approaches.

szvsw 2y ago
I actually think the problem you are describing is actually sometimes helpful from a design perspective, if you can be conscientious enough to periodically review your notebooks and figure out what is the actual useful code which should be properly integrated into the codebase vs what is the “one-off” / non-modular code. Like you mentioned, calculation vs side-effects is one way to help you decide but not the only. There’s definitely no single answer. The key is to just periodically figure out what ought to be refactored into library code, which notebooks should just be straight up deleted (hopefully as many as possible - you can always get them back in your git history if needed!), and so on.
qsort 2y ago
I'm mostly in the camp that notebooks aren't that great for software development, they thrive as an "excel for coders" of sorts, but take a look at nbdev from fast.ai. The literate programming aspect is very nice and I wish it was explored more.

ThouYS 2y ago

marimo is really cool, albeit "pure python" is only true insofar as the diff is concerned. other than that, it's an unconnected group of functions that need the marimo runtime to stitch together. would be cool if marimo could "unroll" the compute graph into a standalone python script that doesn't need the marimo library

mscolnick 2y ago
it’s already possible to do this: marimo export script nb.py Pure-python also helps to work with existing tools out of the box: formatting, linting, pytest, importing notebooks as modules, composition, PEP 723 inline metadata
- 3eb7988a1663 2y ago
  If you are going to do that, you could stick with Jupyter + nbconvert. I rarely use notebooks directly anymore unless I require the output to be stored. Do most everything in VSCode with interactive .py files. Gets you the same notebook-y experience + all of the Python tooling.
  - akshayka 2y ago
    marimo notebooks are actually DAGs as cells, reusable out-of-the-box in three diffferent ways: as interactive computing notebooks in a reactive environment with no hidden state, as Python scripts, as data apps. So Jupyter + nbconvert (or perhaps you meant jupytext) is not a replacement for marimo.
- deleted 2y ago
  [deleted]
deleted 2y ago
[deleted]