9 ms·

Show HN: Marimo – an open-source reactive notebook for Python

448 points by akshayka 3y ago

Hi HN! We’re excited to share marimo, an open-source reactive notebook for Python [1]. marimo aims to solve well-known problems with traditional notebooks [2]: marimo notebooks are reproducible (no hidden state), git-friendly (stored as Python files), executable as Python scripts, and deployable as web apps. GitHub repo: https://github.com/marimo-team/marimo https://github.com/marimo-team/marimo In marimo, a notebook’s code, outputs, and program state are always consistent. Run a cell and marimo reacts by automatically running the cells that reference its declared variables. Delete a cell and marimo scrubs its variables from program memory, eliminating hidden state. Our reactive runtime is based on static analysis, so it’s performant. If you’re worried about accidentally triggering expensive computations, you can disable specific cells from auto-running. marimo comes with UI elements like sliders, a dataframe transformer, and interactive plots that are automatically synchronized with Python [3]. Interact with an element and the cells that use it are automatically re-run with its latest value. Reactivity makes these UI elements more useful and ergonomic than Jupyter’s ipywidgets. Every marimo notebook can be run as a script from the command line, with cells executed in a topologically sorted order, or served as an interactive web app, using the marimo CLI. We’re a team of just two developers. We chose to develop marimo because we believe that the Python community deserves a better programming environment to do research and communicate it; experiment with code and share it; and learn computational science and teach it. We’ve seen lots of research start in Jupyter notebooks (much of my own has), only to fail to reproduce; lots of promising prototypes built that were never made real; and lots of tutorials written that failed to engage students. marimo has been developed with the close input of scientists and engineers, and with inspiration from many tools, including Pluto.jl and streamlit. We open-sourced it recently because we feel it’s ready for broader use. Please try it out (pip install marimo && marimo tutorial intro). We’d appreciate your feedback! [1] https://github.com/marimo-team/marimo https://github.com/marimo-team/marimo [2] https://docs.marimo.io/faq.html#faq-problems https://docs.marimo.io/faq.html#faq-problems [3] https://docs.marimo.io/api/inputs/index.html https://docs.marimo.io/api/inputs/index.html

warthog 3y ago
Did not work a lot with Jupyter nbs but I think it would be good for you to put more emphasis into Jupyter vs Marimo into your website
- alsodumb 3y ago
  [flagged]
- pvg 3y ago
  It's right in the linked FAQ https://docs.marimo.io/faq.html#faq-jupyter https://docs.marimo.io/faq.html#faq-jupyter
  - noahlt 3y ago
    It's there, but warthog is right, it should be a toplevel section like "A reactive programming environment" — yes ideally people would read the description and understand the differences themselves, or consult the FAQ, but the fact is that most people will understand Marimo in relation to Jupyter and so you might as well optimize that path.
- deleted 3y ago
  [deleted]
bluish29 3y ago
That's one interesting project. As someone who relies heavily on collaboration with people using Jupyter Notebook. The most annoying points about reproducing their work are the environment and the hidden state of Jupyter Notebooks. This does to address directly the second problem. It does however by sacrificing flexibility. I might need to change a cell just to test a new thing (without affecting the other cells) but thats a trade off if you focus on reproducibility. I know that requirements.txt is the standard solution to the other problem. But generating and using it is annoying. The command pio freeze will list all the packages in bloated way (there is better ways) but I always hoped to find a notebook system that will integrate this information natively and have a way to embed that into a notebook in a form that I can share with other people. Unfortunately I can't see support for something in any of the available solutions (at least up to my knowledge).
- akshayka 3y ago
  Yes, the second half of reproducibility is for sure packages. A solution for reproducible environments is on our roadmap (https://marimo-team.notion.site/The-marimo-roadmap-e5460b9f203e4adeae633523fce87a35 https://marimo-team.notion.site/The-marimo-roadmap-e5460b9f2...), but we haven't quite figured it out yet. It's a bit challenging because Python has so many different solutions for package management. If you have any ideas we'd love to hear them.
  - bluish29 3y ago
    The link redirect does not specify which point in the list you are referring to but I guess it is "Install missing packages from...". If so, then I really wonder if you mean supporting something like '!pip install numpy' like Jupyter or something else? I don't think this is really a solution, not to mention that this raise the question. Does it support running shell commands using '!' like Jupyter Notebook?
    - akshayka 3y ago
      Oh, sorry for not being more clear. That's not the one. It's "Package management: make notebooks reproducible down to the packages they use": https://marimo-team.notion.site/840c475fd7ca4e3a8c6f20c86fce51a0?v=0a96d9f02aa2459388dbadf36a9b6167&p=a67d9dd024f94defb995cd2bc6f6c4b6&pm=s https://marimo-team.notion.site/840c475fd7ca4e3a8c6f20c86fce... Does that align with what you're talking about? That page has some scrawled brainstormed notes. But we haven't spent time designing a solution yet.
SushiHippie 3y ago
Looks cool! Have you looked into WASM? Something like a jupyterlite [0] alternative for marimo? And are there plans to integrate linting and formatting with ruff? [1] [0] https://jupyterlite.readthedocs.io/en/stable/ https://jupyterlite.readthedocs.io/en/stable/ [1] https://github.com/astral-sh/ruff https://github.com/astral-sh/ruff (ruff format is almost 100% compatible with black formatting)
- akshayka 3y ago
  We started looking into WASM this week, and did some light exploratory coding toward it. It's on our roadmap: https://marimo-team.notion.site/The-marimo-roadmap-e5460b9f203e4adeae633523fce87a35?pvs=4 https://marimo-team.notion.site/The-marimo-roadmap-e5460b9f2... A ruff integration is a great idea. I'll add it to the roadmap.
  - SushiHippie 3y ago
    Perfect, thank you!
  - SushiHippie 3y ago
    <2 cents> I see some package management stuff on the roadmap. Maybe you could take a look at the cargo cli, like pixi did [0]. IMO it's a nice user experience. [0] https://prefix.dev/ https://prefix.dev/ </2 cents>
    - akshayka 3y ago
      Thanks for the suggestion. We'll definitely take a look.
  - prabir 3y ago
    Looking forward to the WASM integration. Being able to use plain filesystem such as nextcloud and able to run it there would be great. I have been trying to get juypterlite wasm in my next cloud alternative that I have been working so would love to try this.
hedgehog 3y ago
This looks quite nice and it might compose well with a cache library like the one posted on HN recently (XetCache, https://news.ycombinator.com/item?id=38696631 https://news.ycombinator.com/item?id=38696631).
- noahlt 3y ago
  Yeah, having worked on alternative notebooks before, one of the big implicit features of Jupyter notebooks is that long-running cells (downloading data, training models) don't get spuriously re-run. Having an excellent cache might reduce spurious re-running of cells, but I wonder if it would be sufficient.
  - akshayka 3y ago
    We've thought briefly about cell-level caching; or at least it's a topic that's come up a couple times now with our users. Perhaps we could add it as a configuration option, at the granularity of individual cells. Our users have found that `functools.cache` goes a long way. We also let users disable cells (and their descendants), which can be useful if you're iterating on a cell that's close to the root of your notebook DAG: https://docs.marimo.io/guides/reactivity.html#disabling-cells https://docs.marimo.io/guides/reactivity.html#disabling-cell...
    - smacke 3y ago
      ipyflow has a %%memoize magic which looks quite similar to %%xetmemo (just without specifying the inputs / outputs explicitly): https://github.com/ipyflow/ipyflow/?tab=readme-ov-file#memoization https://github.com/ipyflow/ipyflow/?tab=readme-ov-file#memoi... Would be cool if we could come up with a standard that works across notebooks / libraries!
      - hedgehog 3y ago
        Function-level caching is the best match for how I'd use it. Often the reason for bothering to cache is that the underlying process is slow, so some kind of future-with-progress wrapper could also be interesting. An example of how that could be used would be wrapping a file transfer so the cell can show progress and then when the result is ready unwrap the value for use in other cells. Or another example would be training in PyTorch, yield progress or stats during the run and then the final run data when complete.
dimatura 3y ago
I already use jupytext to store notebooks as code but the improved state management and notebook-as-app features are pretty compelling and I'm trying it out. Unfortunately, I'm quite used to very specific vim keybindings in Jupyter (https://github.com/lambdalisue/jupyter-vim-binding https://github.com/lambdalisue/jupyter-vim-binding) that make it pretty hard to use anything else :/
- aldanor 3y ago
  If you're a vimmer and a jupyter user, do yourself a favour and switch from browser to vscode: vim emulation is much better overall and you get proper python lsp experience, with jumping to definitions, type inference, copilot, and all that. (Neovim user myself, as much as I dislike vscode for everything else, as of now it's hard to replace it when using jupyter)
peter_l_downs 3y ago
Marimo are wonderful little pets, I used to have some and really liked it. I should get some more. Never failed to start a conversation when guests came over. https://soltech.com/blogs/blog/how-to-care-for-your-marimo-moss-ball https://soltech.com/blogs/blog/how-to-care-for-your-marimo-m...
mondrian 3y ago
Looks cool. This is kind of like streamlit, which (I think) tried to escape the limitations of notebooks by giving you an API to quickly make a shareable app with sliders/charts etc. (Yet it retains some notebook concepts like 'cells'). Marimo kind of takes the reactive widgets of streamlit and brings them back into a notebook-like UI, and provides a way to export the notebooks into shareable apps.
- akshayka 3y ago
  Thanks! One way we differ from streamlit is that ML/data/experimentation work can start in marimo — i.e., you can use marimo for traditional notebooking work, without ever making an app. But you can also use marimo to make shareable apps as you've articulated.
bsdz 3y ago
This is a great idea. I'd been planning to create something similar where cells are topologically ordered based on their dependency structure; although I was thinking perhaps to integrate with Jupyter more, eg use their existing kernel web sockets infrastructure. In my mind, one would be able to zoom out and see a graph view where hovering over a node would show its corresponding cell with content / output. Each node might be coloured according to execution status. That said, I'm not a UI expert and I never got around to it. So thanks for your efforts, I'll definitely give it a spin.
- akshayka 3y ago
  That sounds really cool! marimo has a dependency graph viewer built-in, but we could definitely improve it. Coloring nodes by execution status, and annotating cells with their variable defs/refs, would be great quality-of-life improvements.
simonw 3y ago
This is amazing. I'm a big user of both Jupyter notebooks and Observable notebooks (https://observablehq.com/ https://observablehq.com/) and the thing I miss most from Observable when I'm using Jupyter is the lack of cell reactivity. You've solved that incredibly well! I also really like that the Marimo file format is just Python. Here's an example saved file from playing around with the intro: https://gist.github.com/simonw/e6e6e4b45d1bed9fc1482412743b8205 https://gist.github.com/simonw/e6e6e4b45d1bed9fc1482412743b8... Nice that it's Apache 2 licensed too. Wow, I just found the GitHub Copilot feature too!
- mscolnick 3y ago
  Myles here (other core contributor) - We are thrilled to see you have such a strong positive reaction. It means a lot coming from you - I initially learned web development using Django and landed my first contracting gig with Django. I drifted away from writing Python and towards Typescript - but marimo has brought me back to writing Python.
  - arthurwu 3y ago
    let's go!! so excited to see this get deserved attention
  - mind-blight 3y ago
    Congrats Myles! Super excited that you all have finally open sourced! I'm gonna start moving my Jupyter notebooks over to this asap. I love that it's all just .py files. Have you had anyone use Marimo to write production web app code? I've been doing a lot of AI experiments for the new venture, and it's been a pain to have to switch back and forth between .ipynb files and regular py files
    - mscolnick 3y ago
      People have used marimo for production web apps. They won't get you as far as writing HTML/JS. But great for internal tools or external showcases, tutorials, interactive blogs, etc. Our friends at SLAC use marimo for their internal exploration experiments and publishing interactive apps. He is an example: https://marimo.io/@public/signal-decomposition https://marimo.io/@public/signal-decomposition
zengid 3y ago
Very cool! This is something Jack Rusher cries for in his talk "Stop Writing Dead Programs" https://www.youtube.com/watch?v=8Ab3ArE8W3s https://www.youtube.com/watch?v=8Ab3ArE8W3s
- JayCeeM 3y ago
  Also from Joel Grus as well "I don't like notebooks" https://www.youtube.com/watch?v=7jiPeIFXb6U https://www.youtube.com/watch?v=7jiPeIFXb6U
elijahbenizzy 3y ago
You've built observable but for python. Love it!
esafak 3y ago
Could this be used with MDX or something to embed interactive examples in documentation? That is an underserved use case.
- mscolnick 3y ago
  It is not possible at the moment (we use iframes in our documentation), but once we support WASM, it should be possible.
rurban 3y ago
I'll definitely try it out tomorrow! Could fix a lot of problems with my current project.
Beefin 3y ago
we use the jupyter-server kernel gateway api at https://nux.ai https://nux.ai would love to explore using marimo's API for code execution
smacke 3y ago
I'm a big fan of Marimo (and of Akshay and Myles in particular); it's great to finally see a viable competitor to Jupyter as it can only mean good things for the ecosystem of scientific tooling as a whole.
j0e1 3y ago
This is a welcome alternative to Jupyter Notebooks/lab- great work! One thing that would be nice is an ability to see previews on GitHub of the Marimo notebook (like Jupyter Notebook). I am not sure if this is possible given you would have to run the code to see the output.
carterschonwald 3y ago
Awesome! I’ve been wanting this sort of thing for a long time. But I’ve only been aware of the Julia tool pluto
krawczstef 3y ago
how do you read the resulting python files? That's what I'm struggling with -- but I guess the point is you don't read them, you use marimo for that?
- akshayka 3y ago
  Thanks for the question. Each cell is represented as a function that maps its referenced variables to the variables it defines. Cells are sorted in the order they appear on the notebook page. If you run `marimo tutorial fileformat`, that'll open a tutorial notebook that explains the fileformat in some detail.
aqader 3y ago
this is really cool, can’t wait to try it out for some ML pipeline development. kudos myles and akshay!
Micoloth 3y ago
Wow.. Really great work, finally someone is doing it! Since I've thought about this for a long time (I've actually even made a very simplified version last year [1]), I want to contribute a few thoughts: - cool that you have a Vscode extension, but I was a little disappointed that it opens a full browser view instead of using the existing, good Notebook interface of Vscode. (I get you want to show the whole Frontend- But I'd love to be able to run the Reactive Kernel within the full Vscode ecosystem.. Included Github Copilot is cool, but that's not all) - As other comments said, if you want to go for reproducibility, the part about Package Management is very important. And it's also mostly solved, with Poetry etc... - If you want to go for easy deployment of the NB code to Production, another very cool feature would be to extract (as a script) all the code needed to produce a given cell of output! This should be very easy since you already have the DAG.. It actually even existed at some point in VSCode Python extension, then they removed it Again, great job [1] https://github.com/micoloth/vscode-reactive-jupyter https://github.com/micoloth/vscode-reactive-jupyter
- smacke 3y ago
  You're probably referring to nbgather (https://github.com/microsoft/gather https://github.com/microsoft/gather), which shipped with VSCode for a while. nbgather used static slicing to get all the code necessary to reconstruct some cell. I actually worked with Andrew Head (original nbgather author) and Shreya Shankar to implement something similar in ipyflow (but with dynamic slicing and a not-as-nice interface): https://github.com/ipyflow/ipyflow?tab=readme-ov-file#state-api https://github.com/ipyflow/ipyflow?tab=readme-ov-file#state-... I have no doubt something like this will make its way into marimo's roadmap at some point :)
yowlingcat 3y ago
This is very cool. I think I need to play around with this a bit more to wrap my head around the reactivity element, but the basic shift of ipynb to standard Python would be such a huge workflow improvement for my team. We use jupyter notebooks when prototyping and trying to code review unwieldy python-in-JSON is miserable. Great to see an alternative that's worked its way around that.
exe34 3y ago
That's amazing! Can I edit it in another editor, save the file and have it updated live in the browser notebook? Or does it have to recompute everything?
- akshayka 3y ago
  Not yet, but that's something we do want to support.
paddy_m 3y ago
Very exciting! I took a quick look and I have a couple of questions. 1. Can you describe your interactive widget story? I see that you integrated altair, and there is some custom written react code around it [0] [1]. I'd be interested in porting my table widget to your platform at some point. 2. How much, if any does this depend on the jupyter ecosystem? 3. How does this interact with the jupyter ecosystem? [0] https://github.com/marimo-team/marimo/blob/b52faf3caf9aa73f4902791306d02a20968fc4be/frontend/src/plugins/impl/vega/VegaPlugin.tsx#L46 https://github.com/marimo-team/marimo/blob/b52faf3caf9aa73f4... [1] https://github.com/marimo-team/marimo/blob/b52faf3caf9aa73f4902791306d02a20968fc4be/marimo/_plugins/ui/_impl/altair_chart.py#L175 https://github.com/marimo-team/marimo/blob/b52faf3caf9aa73f4...
- akshayka 3y ago
  1. We don't have a public plugin API yet, but we will in the future. Our (internal) plugins are represented as custom elements: Python writes the HTML (e.g., `<marimo-vega ...>` and the frontend instantiates it. In the meantime, maybe we can help you port your table widget and make it a marimo plugin. You can reach us in our Discord (https://discord.gg/JE7nhX6mD8 https://discord.gg/JE7nhX6mD8) or at Github. 2. marimo was built from scratch, it doesn't depend on Jupyter or IPython at all. 3. marimo doesn't interact with the Jupyter ecosystem. We have brainstormed the possibility of a compatibility layer that allows Jupyter widgets to be used as marimo plugins, but right now that's just an idea.
aredox 3y ago
Awesome! What would be the best way to use it locally in a minimal, self-contained install?
- derHackerman 3y ago
  Try using pipx!
bravura 3y ago
I am most intrigued by the annotation demo you showed, since annotation is painful to set up for small projects. Can you talk about it in more detail? Can I tell who the user is so I can have multiple annotators? Can I use gold data to determine which annotators aren't paying attention? Where do I learn more about how to build this kind of tool? Overall, kudos, I signed up for the waitlist.
ametrau 3y ago
Thank you. Jupyter has me taking my hair out a lot of the time. Some completely bizarre design decisions
bitsrat 3y ago
I read in a comment that Marimo is an alternate to Jupyter. Does it not depend on Jupyter Server or ipykernel ? Is it a replacement for Jupyter lab ? I am thinking of Jupyter as all the components in this diagram - https://docs.jupyter.org/en/latest/projects/architecture/content-architecture.html https://docs.jupyter.org/en/latest/projects/architecture/con... Sorry did not get to look into the codebase yet
- mscolnick 3y ago
  Correct, it does not depend on Jupyter. It’s built from the ground up with different principles in mind
chris_nielsen 3y ago
I love this, but Im using DataSpell from JetBrains at the moment because it has 2 killer features: 1. Variable viewer so I can see the current value of all variables in scope. 2. Interactive debugger Maybe the variable viewer is only important because Jupyter notebooks don’t track and rerun dependencies? So I wouldn’t need it with Marimo. But the interactive debugger is priceless. Any plan to add debugging?
- mscolnick 3y ago
  1. We do have a variable viewer. We have a few helper panels in the bottom left. 2. PDB support is planned and was scoped out yesterday. Appreciate the feedback!
  - chris_nielsen 3y ago
    That's awesome, ok I'm going to go check it out. Great work!
jwilber 3y ago
This is amazing!
ingenieroariel 3y ago
The list of dependencies seems very short, apart from tornado it does not seem like the other ones pull in a lot of other deps. Congrats, this looks very useful and awesome. dependencies = [ # cli "click>=8.0,<9", # python 3.8 compatibility "importlib_resources>=5.10.2; python_version < \"3.9\"", # code completion "jedi>=0.18.0", # compile markdown to html "markdown>=3.4,<4", # add features to markdown "pymdown-extensions>=9.0,<11", # syntax highlighting of code in markdown "pygments>=2.13,<3", # for reading, writing configs "tomlkit>= 0.12.0", # web server "tornado>=6.1,<7", # python <=3.9 compatibility "typing_extensions>=4.4.0; python_version < \"3.10\"", # for cell formatting; if user version is not compatible, no-op "black", ]
rossjudson 3y ago
Arrggghh. Now I have to learn Python, which I've been actively resisting and making jokes about for years.
nnx 3y ago
Very interesting project, a breeze of fresh air and welcome competition to Jupyter. I guess it's still very early but the onboarding for Mario VSCode is not great at the moment, no idea how to actually start writing a Marimo notebook (no "Create: New Marimo notebook" option like Jupyter's). Then I then tried clone the cookbook repo, and get "module not found" errors that are even less friendly than when it happens on Jupyter: have to figure out which cell the error actually comes from to even know which module is missing.
petters 3y ago
Looks really impressive! But state is not tracked perfectly. Sometimes you have to manually re-run the cell. For example if one cell defines a dataclass d and another cell changes d.x = "new value". Then other cells using d.x will not know that it has changed.
wisty 3y ago
Cool. On a side note, I think the old Jupytext extension is hugely underrated. It lets Jupyter run a .py file (with markdown notes as comment in the file, displayed as notes in the web page). Both of these solve the most important part of this problems in iPython - horrible git interaction, horrible programming practice to discouraging writing library files, though Jupyter fixes most of the weird non-deterministic behaviour by forcing you to rerun the script every time you load it (rather than reactive techniques). State is OK for power users but it's known to be a massive pain for people who are just learning programming, and an issue in large projects or with interaction. With this new project having reactive updates I think it's definitely going to be great for beginners, or in gnarly projects. I wonder if it runs on pyodide (a cPython compiled to run in the browser, with matplotlib and scipy bundled).
mvelbaum 3y ago
Does this allow to run a long running task in the background so that a user can close & reopen the tab and continue seeing all the output that has been produced thus far? This is currently being worked on in Jupyter: https://github.com/jupyterlab/jupyterlab/pull/15448 https://github.com/jupyterlab/jupyterlab/pull/15448
Onawa 3y ago
Aren't many of the issues with Jupyter being mentioned in this thread solved by Quarto? I have been advocating for it's use more at work, and NIH has even started offering classes on it through the NIH library.
- TtjM 3y ago
  Exactly my thoughts too, especially regarding reproducibility issues Quarto has been great in the past for projects at my workplace. I have yet to try Marimo but synchronised code cells are what seems to set it apart. Quarto + jupyter-cache [1] was the closest I have managed to get to that experience but that approach has its constraints. [1]: https://github.com/executablebooks/jupyter-cache https://github.com/executablebooks/jupyter-cache
petters 3y ago
Defining the same variable more than once is an error. The reason for this is obvious. But if the variable is never used in a cell that does not first write to it, reusing variable name should be possible. Allowing that would be good, because many notebook cells start with "fig, ax = plt.subplots(2, 2)" and this is currently not allowed more than once.
- jkl5xx 3y ago
  Does the local underscore variables feature solve this? Or the approach outlined in the plots tutorial? IMO, not allowing redeclaration is more valuable than supporting this use case. A slight paradigm shift away from your example gives you the significant benefits of a reactive environment with fewer edge cases/quirks. I'd much rather have a notebook error out instead of silently overwriting a value. You save so much time debugging.
  - bluish29 3y ago
    > Does the local underscore variables feature solve this I tried this yesterday trying to convert a Jupyter notebook with a log of fig, axs, and it was very annoying converting all of them. I tried local _ with fig_ and ax1_ …etc. but it is considered a variable that cannot be reused too. Furthermore, I expected local vs global variables to be cell based somehow, but that was naive on my part. It does static analysis, not dynamic, so defining something like _suffix and add it to all reused variables and assign different values for each cell will need a dynamical analysis to work.
  - petters 3y ago
    Yes, but what I proposed seems like no risk of silently overwriting? If there are dependencies between cells there will still be an error
stuaxo 3y ago
This is good, I've been waiting for something like this to solve the issue of determinism in notebooks.
robsh 3y ago
It would be amazing if it could be deployed with pyodide/wasm as an alternative to a Python web server. Truly a standalone interactive notebook, hosted with plain html.
mbrezu 3y ago
Marimo looks and feels great! Have you considered adding support for mermaid.js in the markdown? I tried including some mermaid.js in a `mo.md` invocation, but it didn't render the diagram :-) https://mermaid.js.org/ https://mermaid.js.org/
- mscolnick 3y ago
  We’ve been thinking about it (but had no requests for it yet). I will look into adding it this week. If you would want to make the contribution, feel free to jump/chat in the discord.
rrr_oh_man 3y ago
Do you guys have anything resembling RStudio-style doc-aware code completion? [1] I swear it's the bane of my existence whenever doing anything inside Jupyter. Coming from RStudio it always feels like operating in a vacuum. [1] https://rstudioblog.files.wordpress.com/2015/02/s3.png https://rstudioblog.files.wordpress.com/2015/02/s3.png
- mscolnick 3y ago
  Yes, we do!
oakhaven 3y ago
At first I thought that in effect, this project only removes a couple of Ctrl+Enter keystrokes in Jupyter-notebook workflow. But after trying out the intro I think it looks good, I really like the simple convert to a webapp. I wonder if the state/data in the generated app are stored server-side of sent to browser. I went through the slider example in the intro and I noticed that when I change the icon, the slider position goes back to 1. I tried to fix it so that the slider-selected value is preserved over icon changes, but didn't manage, it doesn't seem straightforward.
zhenyakovalyov 3y ago
this is very impressive. the only one bit that is in my muscle memory from using Jupyter is 'A' (add above), 'B' (add below) and 'D-D' (delete) shortcuts. kudos for adding polars support!
garfieldnate 3y ago
The readme says that I can convert Jupyter notebooks, but to what extent does this actually work? What if I've imported custom JS to render mathjax or added custom CSS? What if I've added inline graphics or videos?

448 points by akshayka 3y ago

warthog 3y ago

Did not work a lot with Jupyter nbs but I think it would be good for you to put more emphasis into Jupyter vs Marimo into your website

alsodumb 3y ago
[flagged]
pvg 3y ago
It's right in the linked FAQ https://docs.marimo.io/faq.html#faq-jupyter https://docs.marimo.io/faq.html#faq-jupyter
- noahlt 3y ago
  It's there, but warthog is right, it should be a toplevel section like "A reactive programming environment" — yes ideally people would read the description and understand the differences themselves, or consult the FAQ, but the fact is that most people will understand Marimo in relation to Jupyter and so you might as well optimize that path.
deleted 3y ago
[deleted]

bluish29 3y ago

That's one interesting project. As someone who relies heavily on collaboration with people using Jupyter Notebook. The most annoying points about reproducing their work are the environment and the hidden state of Jupyter Notebooks. This does to address directly the second problem. It does however by sacrificing flexibility. I might need to change a cell just to test a new thing (without affecting the other cells) but thats a trade off if you focus on reproducibility. I know that requirements.txt is the standard solution to the other problem. But generating and using it is annoying. The command pio freeze will list all the packages in bloated way (there is better ways) but I always hoped to find a notebook system that will integrate this information natively and have a way to embed that into a notebook in a form that I can share with other people. Unfortunately I can't see support for something in any of the available solutions (at least up to my knowledge).

akshayka 3y ago
Yes, the second half of reproducibility is for sure packages. A solution for reproducible environments is on our roadmap (https://marimo-team.notion.site/The-marimo-roadmap-e5460b9f203e4adeae633523fce87a35 https://marimo-team.notion.site/The-marimo-roadmap-e5460b9f2...), but we haven't quite figured it out yet. It's a bit challenging because Python has so many different solutions for package management. If you have any ideas we'd love to hear them.
- bluish29 3y ago
  The link redirect does not specify which point in the list you are referring to but I guess it is "Install missing packages from...". If so, then I really wonder if you mean supporting something like '!pip install numpy' like Jupyter or something else? I don't think this is really a solution, not to mention that this raise the question. Does it support running shell commands using '!' like Jupyter Notebook?
  - akshayka 3y ago
    Oh, sorry for not being more clear. That's not the one. It's "Package management: make notebooks reproducible down to the packages they use": https://marimo-team.notion.site/840c475fd7ca4e3a8c6f20c86fce51a0?v=0a96d9f02aa2459388dbadf36a9b6167&p=a67d9dd024f94defb995cd2bc6f6c4b6&pm=s https://marimo-team.notion.site/840c475fd7ca4e3a8c6f20c86fce... Does that align with what you're talking about? That page has some scrawled brainstormed notes. But we haven't spent time designing a solution yet.

SushiHippie 3y ago

Looks cool! Have you looked into WASM? Something like a jupyterlite [0] alternative for marimo? And are there plans to integrate linting and formatting with ruff? [1] [0] https://jupyterlite.readthedocs.io/en/stable/ https://jupyterlite.readthedocs.io/en/stable/ [1] https://github.com/astral-sh/ruff https://github.com/astral-sh/ruff (ruff format is almost 100% compatible with black formatting)

akshayka 3y ago
We started looking into WASM this week, and did some light exploratory coding toward it. It's on our roadmap: https://marimo-team.notion.site/The-marimo-roadmap-e5460b9f203e4adeae633523fce87a35?pvs=4 https://marimo-team.notion.site/The-marimo-roadmap-e5460b9f2... A ruff integration is a great idea. I'll add it to the roadmap.
- SushiHippie 3y ago
  Perfect, thank you!
- SushiHippie 3y ago
  <2 cents> I see some package management stuff on the roadmap. Maybe you could take a look at the cargo cli, like pixi did [0]. IMO it's a nice user experience. [0] https://prefix.dev/ https://prefix.dev/ </2 cents>
  - akshayka 3y ago
    Thanks for the suggestion. We'll definitely take a look.
- prabir 3y ago
  Looking forward to the WASM integration. Being able to use plain filesystem such as nextcloud and able to run it there would be great. I have been trying to get juypterlite wasm in my next cloud alternative that I have been working so would love to try this.

hedgehog 3y ago

This looks quite nice and it might compose well with a cache library like the one posted on HN recently (XetCache, https://news.ycombinator.com/item?id=38696631 https://news.ycombinator.com/item?id=38696631).

noahlt 3y ago
Yeah, having worked on alternative notebooks before, one of the big implicit features of Jupyter notebooks is that long-running cells (downloading data, training models) don't get spuriously re-run. Having an excellent cache might reduce spurious re-running of cells, but I wonder if it would be sufficient.
- akshayka 3y ago
  We've thought briefly about cell-level caching; or at least it's a topic that's come up a couple times now with our users. Perhaps we could add it as a configuration option, at the granularity of individual cells. Our users have found that `functools.cache` goes a long way. We also let users disable cells (and their descendants), which can be useful if you're iterating on a cell that's close to the root of your notebook DAG: https://docs.marimo.io/guides/reactivity.html#disabling-cells https://docs.marimo.io/guides/reactivity.html#disabling-cell...
  - smacke 3y ago
    ipyflow has a %%memoize magic which looks quite similar to %%xetmemo (just without specifying the inputs / outputs explicitly): https://github.com/ipyflow/ipyflow/?tab=readme-ov-file#memoization https://github.com/ipyflow/ipyflow/?tab=readme-ov-file#memoi... Would be cool if we could come up with a standard that works across notebooks / libraries!
    - hedgehog 3y ago
      Function-level caching is the best match for how I'd use it. Often the reason for bothering to cache is that the underlying process is slow, so some kind of future-with-progress wrapper could also be interesting. An example of how that could be used would be wrapping a file transfer so the cell can show progress and then when the result is ready unwrap the value for use in other cells. Or another example would be training in PyTorch, yield progress or stats during the run and then the final run data when complete.

dimatura 3y ago

I already use jupytext to store notebooks as code but the improved state management and notebook-as-app features are pretty compelling and I'm trying it out. Unfortunately, I'm quite used to very specific vim keybindings in Jupyter (https://github.com/lambdalisue/jupyter-vim-binding https://github.com/lambdalisue/jupyter-vim-binding) that make it pretty hard to use anything else :/

aldanor 3y ago
If you're a vimmer and a jupyter user, do yourself a favour and switch from browser to vscode: vim emulation is much better overall and you get proper python lsp experience, with jumping to definitions, type inference, copilot, and all that. (Neovim user myself, as much as I dislike vscode for everything else, as of now it's hard to replace it when using jupyter)

peter_l_downs 3y ago

Marimo are wonderful little pets, I used to have some and really liked it. I should get some more. Never failed to start a conversation when guests came over. https://soltech.com/blogs/blog/how-to-care-for-your-marimo-moss-ball https://soltech.com/blogs/blog/how-to-care-for-your-marimo-m...

mondrian 3y ago

Looks cool. This is kind of like streamlit, which (I think) tried to escape the limitations of notebooks by giving you an API to quickly make a shareable app with sliders/charts etc. (Yet it retains some notebook concepts like 'cells'). Marimo kind of takes the reactive widgets of streamlit and brings them back into a notebook-like UI, and provides a way to export the notebooks into shareable apps.

akshayka 3y ago
Thanks! One way we differ from streamlit is that ML/data/experimentation work can start in marimo — i.e., you can use marimo for traditional notebooking work, without ever making an app. But you can also use marimo to make shareable apps as you've articulated.

bsdz 3y ago

This is a great idea. I'd been planning to create something similar where cells are topologically ordered based on their dependency structure; although I was thinking perhaps to integrate with Jupyter more, eg use their existing kernel web sockets infrastructure. In my mind, one would be able to zoom out and see a graph view where hovering over a node would show its corresponding cell with content / output. Each node might be coloured according to execution status. That said, I'm not a UI expert and I never got around to it. So thanks for your efforts, I'll definitely give it a spin.

akshayka 3y ago
That sounds really cool! marimo has a dependency graph viewer built-in, but we could definitely improve it. Coloring nodes by execution status, and annotating cells with their variable defs/refs, would be great quality-of-life improvements.

simonw 3y ago

This is amazing. I'm a big user of both Jupyter notebooks and Observable notebooks (https://observablehq.com/ https://observablehq.com/) and the thing I miss most from Observable when I'm using Jupyter is the lack of cell reactivity. You've solved that incredibly well! I also really like that the Marimo file format is just Python. Here's an example saved file from playing around with the intro: https://gist.github.com/simonw/e6e6e4b45d1bed9fc1482412743b8205 https://gist.github.com/simonw/e6e6e4b45d1bed9fc1482412743b8... Nice that it's Apache 2 licensed too. Wow, I just found the GitHub Copilot feature too!

mscolnick 3y ago
Myles here (other core contributor) - We are thrilled to see you have such a strong positive reaction. It means a lot coming from you - I initially learned web development using Django and landed my first contracting gig with Django. I drifted away from writing Python and towards Typescript - but marimo has brought me back to writing Python.
- arthurwu 3y ago
  let's go!! so excited to see this get deserved attention
- mind-blight 3y ago
  Congrats Myles! Super excited that you all have finally open sourced! I'm gonna start moving my Jupyter notebooks over to this asap. I love that it's all just .py files. Have you had anyone use Marimo to write production web app code? I've been doing a lot of AI experiments for the new venture, and it's been a pain to have to switch back and forth between .ipynb files and regular py files
  - mscolnick 3y ago
    People have used marimo for production web apps. They won't get you as far as writing HTML/JS. But great for internal tools or external showcases, tutorials, interactive blogs, etc. Our friends at SLAC use marimo for their internal exploration experiments and publishing interactive apps. He is an example: https://marimo.io/@public/signal-decomposition https://marimo.io/@public/signal-decomposition

zengid 3y ago

Very cool! This is something Jack Rusher cries for in his talk "Stop Writing Dead Programs" https://www.youtube.com/watch?v=8Ab3ArE8W3s https://www.youtube.com/watch?v=8Ab3ArE8W3s

JayCeeM 3y ago
Also from Joel Grus as well "I don't like notebooks" https://www.youtube.com/watch?v=7jiPeIFXb6U https://www.youtube.com/watch?v=7jiPeIFXb6U

elijahbenizzy 3y ago

You've built observable but for python. Love it!

esafak 3y ago

Could this be used with MDX or something to embed interactive examples in documentation? That is an underserved use case.

mscolnick 3y ago
It is not possible at the moment (we use iframes in our documentation), but once we support WASM, it should be possible.

rurban 3y ago

I'll definitely try it out tomorrow! Could fix a lot of problems with my current project.

Beefin 3y ago

we use the jupyter-server kernel gateway api at https://nux.ai https://nux.ai would love to explore using marimo's API for code execution

smacke 3y ago

I'm a big fan of Marimo (and of Akshay and Myles in particular); it's great to finally see a viable competitor to Jupyter as it can only mean good things for the ecosystem of scientific tooling as a whole.

j0e1 3y ago

This is a welcome alternative to Jupyter Notebooks/lab- great work! One thing that would be nice is an ability to see previews on GitHub of the Marimo notebook (like Jupyter Notebook). I am not sure if this is possible given you would have to run the code to see the output.

carterschonwald 3y ago

Awesome! I’ve been wanting this sort of thing for a long time. But I’ve only been aware of the Julia tool pluto

krawczstef 3y ago

how do you read the resulting python files? That's what I'm struggling with -- but I guess the point is you don't read them, you use marimo for that?

akshayka 3y ago
Thanks for the question. Each cell is represented as a function that maps its referenced variables to the variables it defines. Cells are sorted in the order they appear on the notebook page. If you run `marimo tutorial fileformat`, that'll open a tutorial notebook that explains the fileformat in some detail.

aqader 3y ago

this is really cool, can’t wait to try it out for some ML pipeline development. kudos myles and akshay!

Micoloth 3y ago

Wow.. Really great work, finally someone is doing it! Since I've thought about this for a long time (I've actually even made a very simplified version last year [1]), I want to contribute a few thoughts: - cool that you have a Vscode extension, but I was a little disappointed that it opens a full browser view instead of using the existing, good Notebook interface of Vscode. (I get you want to show the whole Frontend- But I'd love to be able to run the Reactive Kernel within the full Vscode ecosystem.. Included Github Copilot is cool, but that's not all) - As other comments said, if you want to go for reproducibility, the part about Package Management is very important. And it's also mostly solved, with Poetry etc... - If you want to go for easy deployment of the NB code to Production, another very cool feature would be to extract (as a script) all the code needed to produce a given cell of output! This should be very easy since you already have the DAG.. It actually even existed at some point in VSCode Python extension, then they removed it Again, great job [1] https://github.com/micoloth/vscode-reactive-jupyter https://github.com/micoloth/vscode-reactive-jupyter

smacke 3y ago
You're probably referring to nbgather (https://github.com/microsoft/gather https://github.com/microsoft/gather), which shipped with VSCode for a while. nbgather used static slicing to get all the code necessary to reconstruct some cell. I actually worked with Andrew Head (original nbgather author) and Shreya Shankar to implement something similar in ipyflow (but with dynamic slicing and a not-as-nice interface): https://github.com/ipyflow/ipyflow?tab=readme-ov-file#state-api https://github.com/ipyflow/ipyflow?tab=readme-ov-file#state-... I have no doubt something like this will make its way into marimo's roadmap at some point :)

yowlingcat 3y ago

This is very cool. I think I need to play around with this a bit more to wrap my head around the reactivity element, but the basic shift of ipynb to standard Python would be such a huge workflow improvement for my team. We use jupyter notebooks when prototyping and trying to code review unwieldy python-in-JSON is miserable. Great to see an alternative that's worked its way around that.

exe34 3y ago

That's amazing! Can I edit it in another editor, save the file and have it updated live in the browser notebook? Or does it have to recompute everything?

akshayka 3y ago
Not yet, but that's something we do want to support.

paddy_m 3y ago

Very exciting! I took a quick look and I have a couple of questions. 1. Can you describe your interactive widget story? I see that you integrated altair, and there is some custom written react code around it [0] [1]. I'd be interested in porting my table widget to your platform at some point. 2. How much, if any does this depend on the jupyter ecosystem? 3. How does this interact with the jupyter ecosystem? [0] https://github.com/marimo-team/marimo/blob/b52faf3caf9aa73f4902791306d02a20968fc4be/frontend/src/plugins/impl/vega/VegaPlugin.tsx#L46 https://github.com/marimo-team/marimo/blob/b52faf3caf9aa73f4... [1] https://github.com/marimo-team/marimo/blob/b52faf3caf9aa73f4902791306d02a20968fc4be/marimo/_plugins/ui/_impl/altair_chart.py#L175 https://github.com/marimo-team/marimo/blob/b52faf3caf9aa73f4...

akshayka 3y ago
1. We don't have a public plugin API yet, but we will in the future. Our (internal) plugins are represented as custom elements: Python writes the HTML (e.g., `<marimo-vega ...>` and the frontend instantiates it. In the meantime, maybe we can help you port your table widget and make it a marimo plugin. You can reach us in our Discord (https://discord.gg/JE7nhX6mD8 https://discord.gg/JE7nhX6mD8) or at Github. 2. marimo was built from scratch, it doesn't depend on Jupyter or IPython at all. 3. marimo doesn't interact with the Jupyter ecosystem. We have brainstormed the possibility of a compatibility layer that allows Jupyter widgets to be used as marimo plugins, but right now that's just an idea.

aredox 3y ago

Awesome! What would be the best way to use it locally in a minimal, self-contained install?

derHackerman 3y ago
Try using pipx!

bravura 3y ago

I am most intrigued by the annotation demo you showed, since annotation is painful to set up for small projects. Can you talk about it in more detail? Can I tell who the user is so I can have multiple annotators? Can I use gold data to determine which annotators aren't paying attention? Where do I learn more about how to build this kind of tool? Overall, kudos, I signed up for the waitlist.

ametrau 3y ago

Thank you. Jupyter has me taking my hair out a lot of the time. Some completely bizarre design decisions

bitsrat 3y ago

I read in a comment that Marimo is an alternate to Jupyter. Does it not depend on Jupyter Server or ipykernel ? Is it a replacement for Jupyter lab ? I am thinking of Jupyter as all the components in this diagram - https://docs.jupyter.org/en/latest/projects/architecture/content-architecture.html https://docs.jupyter.org/en/latest/projects/architecture/con... Sorry did not get to look into the codebase yet

mscolnick 3y ago
Correct, it does not depend on Jupyter. It’s built from the ground up with different principles in mind

chris_nielsen 3y ago

I love this, but Im using DataSpell from JetBrains at the moment because it has 2 killer features: 1. Variable viewer so I can see the current value of all variables in scope. 2. Interactive debugger Maybe the variable viewer is only important because Jupyter notebooks don’t track and rerun dependencies? So I wouldn’t need it with Marimo. But the interactive debugger is priceless. Any plan to add debugging?

mscolnick 3y ago
1. We do have a variable viewer. We have a few helper panels in the bottom left. 2. PDB support is planned and was scoped out yesterday. Appreciate the feedback!
- chris_nielsen 3y ago
  That's awesome, ok I'm going to go check it out. Great work!

jwilber 3y ago

This is amazing!

ingenieroariel 3y ago

The list of dependencies seems very short, apart from tornado it does not seem like the other ones pull in a lot of other deps. Congrats, this looks very useful and awesome. dependencies = [ # cli "click>=8.0,<9", # python 3.8 compatibility "importlib_resources>=5.10.2; python_version < \"3.9\"", # code completion "jedi>=0.18.0", # compile markdown to html "markdown>=3.4,<4", # add features to markdown "pymdown-extensions>=9.0,<11", # syntax highlighting of code in markdown "pygments>=2.13,<3", # for reading, writing configs "tomlkit>= 0.12.0", # web server "tornado>=6.1,<7", # python <=3.9 compatibility "typing_extensions>=4.4.0; python_version < \"3.10\"", # for cell formatting; if user version is not compatible, no-op "black", ]

rossjudson 3y ago

Arrggghh. Now I have to learn Python, which I've been actively resisting and making jokes about for years.

nnx 3y ago

Very interesting project, a breeze of fresh air and welcome competition to Jupyter. I guess it's still very early but the onboarding for Mario VSCode is not great at the moment, no idea how to actually start writing a Marimo notebook (no "Create: New Marimo notebook" option like Jupyter's). Then I then tried clone the cookbook repo, and get "module not found" errors that are even less friendly than when it happens on Jupyter: have to figure out which cell the error actually comes from to even know which module is missing.

petters 3y ago

Looks really impressive! But state is not tracked perfectly. Sometimes you have to manually re-run the cell. For example if one cell defines a dataclass d and another cell changes d.x = "new value". Then other cells using d.x will not know that it has changed.

wisty 3y ago

Cool. On a side note, I think the old Jupytext extension is hugely underrated. It lets Jupyter run a .py file (with markdown notes as comment in the file, displayed as notes in the web page). Both of these solve the most important part of this problems in iPython - horrible git interaction, horrible programming practice to discouraging writing library files, though Jupyter fixes most of the weird non-deterministic behaviour by forcing you to rerun the script every time you load it (rather than reactive techniques). State is OK for power users but it's known to be a massive pain for people who are just learning programming, and an issue in large projects or with interaction. With this new project having reactive updates I think it's definitely going to be great for beginners, or in gnarly projects. I wonder if it runs on pyodide (a cPython compiled to run in the browser, with matplotlib and scipy bundled).

mvelbaum 3y ago

Does this allow to run a long running task in the background so that a user can close & reopen the tab and continue seeing all the output that has been produced thus far? This is currently being worked on in Jupyter: https://github.com/jupyterlab/jupyterlab/pull/15448 https://github.com/jupyterlab/jupyterlab/pull/15448

Onawa 3y ago

Aren't many of the issues with Jupyter being mentioned in this thread solved by Quarto? I have been advocating for it's use more at work, and NIH has even started offering classes on it through the NIH library.

TtjM 3y ago
Exactly my thoughts too, especially regarding reproducibility issues Quarto has been great in the past for projects at my workplace. I have yet to try Marimo but synchronised code cells are what seems to set it apart. Quarto + jupyter-cache [1] was the closest I have managed to get to that experience but that approach has its constraints. [1]: https://github.com/executablebooks/jupyter-cache https://github.com/executablebooks/jupyter-cache

petters 3y ago

Defining the same variable more than once is an error. The reason for this is obvious. But if the variable is never used in a cell that does not first write to it, reusing variable name should be possible. Allowing that would be good, because many notebook cells start with "fig, ax = plt.subplots(2, 2)" and this is currently not allowed more than once.

jkl5xx 3y ago
Does the local underscore variables feature solve this? Or the approach outlined in the plots tutorial? IMO, not allowing redeclaration is more valuable than supporting this use case. A slight paradigm shift away from your example gives you the significant benefits of a reactive environment with fewer edge cases/quirks. I'd much rather have a notebook error out instead of silently overwriting a value. You save so much time debugging.
- bluish29 3y ago
  > Does the local underscore variables feature solve this I tried this yesterday trying to convert a Jupyter notebook with a log of fig, axs, and it was very annoying converting all of them. I tried local _ with fig_ and ax1_ …etc. but it is considered a variable that cannot be reused too. Furthermore, I expected local vs global variables to be cell based somehow, but that was naive on my part. It does static analysis, not dynamic, so defining something like _suffix and add it to all reused variables and assign different values for each cell will need a dynamical analysis to work.
- petters 3y ago
  Yes, but what I proposed seems like no risk of silently overwriting? If there are dependencies between cells there will still be an error

stuaxo 3y ago

This is good, I've been waiting for something like this to solve the issue of determinism in notebooks.

robsh 3y ago

It would be amazing if it could be deployed with pyodide/wasm as an alternative to a Python web server. Truly a standalone interactive notebook, hosted with plain html.

mbrezu 3y ago

Marimo looks and feels great! Have you considered adding support for mermaid.js in the markdown? I tried including some mermaid.js in a `mo.md` invocation, but it didn't render the diagram :-) https://mermaid.js.org/ https://mermaid.js.org/

mscolnick 3y ago
We’ve been thinking about it (but had no requests for it yet). I will look into adding it this week. If you would want to make the contribution, feel free to jump/chat in the discord.

rrr_oh_man 3y ago

Do you guys have anything resembling RStudio-style doc-aware code completion? [1] I swear it's the bane of my existence whenever doing anything inside Jupyter. Coming from RStudio it always feels like operating in a vacuum. [1] https://rstudioblog.files.wordpress.com/2015/02/s3.png https://rstudioblog.files.wordpress.com/2015/02/s3.png

mscolnick 3y ago
Yes, we do!

oakhaven 3y ago

At first I thought that in effect, this project only removes a couple of Ctrl+Enter keystrokes in Jupyter-notebook workflow. But after trying out the intro I think it looks good, I really like the simple convert to a webapp. I wonder if the state/data in the generated app are stored server-side of sent to browser. I went through the slider example in the intro and I noticed that when I change the icon, the slider position goes back to 1. I tried to fix it so that the slider-selected value is preserved over icon changes, but didn't manage, it doesn't seem straightforward.

zhenyakovalyov 3y ago

this is very impressive. the only one bit that is in my muscle memory from using Jupyter is 'A' (add above), 'B' (add below) and 'D-D' (delete) shortcuts. kudos for adding polars support!

garfieldnate 3y ago

The readme says that I can convert Jupyter notebooks, but to what extent does this actually work? What if I've imported custom JS to render mathjax or added custom CSS? What if I've added inline graphics or videos?