5 ms·

Ruby in Jupyter Notebook

181 points by Alifatisk 2y ago

DagsEoress 2y ago
Is any of this usable on hosted GPU notebooks (e.g. paperspace)? I'm currently diving into Machine Learning using Python + Scikit-learn, and I'd love to one day replace Python with Ruby. But looking at the current ML ecosystem I don't see that happening. Does anyone have experience building (Supervised / Unsupervised) models using something other than Python (including deployment)?
- qsort 2y ago
  Nbviewer is just a service that lets you host notebooks, there is no computation there. > Does anyone have experience building (Supervised / Unsupervised) models using something other than Python (including deployment)? XGBoost/LightGBM have a C API and can be used from pretty much anything, deployment is not a problem. Practically building models is more about dealing with data, the ecosystem tends to revolve around Python and R for that reason.
- thibaut_barrere 2y ago
  Since you wrote "using something other than Python" and not necessarily only Ruby, definitely look into Livebook and Elixir, and the whole ecosystem around it, including: - https://github.com/elixir-nx/axon https://github.com/elixir-nx/axon Nx-powered Neural Networks - https://github.com/elixir-nx/nx https://github.com/elixir-nx/nx Multi-dimensional arrays (tensors) and numerical definitions for Elixir - https://github.com/elixir-nx/scholar https://github.com/elixir-nx/scholar Traditional machine learning on top of Nx - https://github.com/elixir-nx/bumblebee https://github.com/elixir-nx/bumblebee Pre-trained Neural Network models in Axon (+ Models integration) - https://github.com/elixir-explorer/explorer https://github.com/elixir-explorer/explorer Series (one-dimensional) and dataframes (two-dimensional) for fast and elegant data exploration in Elixir - https://fly.io/blog/rethinking-serverless-with-flame/ https://fly.io/blog/rethinking-serverless-with-flame/ (for offloading large work to remote containers) - https://www.youtube.com/watch?v=RABXu7zqnT0 https://www.youtube.com/watch?v=RABXu7zqnT0 InstructorEx And of course Livebook (https://livebook.dev https://livebook.dev) Old but interesting video https://www.youtube.com/watch?v=g3oyh3g1AtQ https://www.youtube.com/watch?v=g3oyh3g1AtQ (Bumblebee: GPT2, Stable Diffusion, and more in Elixir) A talk on the upcoming Elixir conference (https://2024.elixirconf.com/schedule/#schedules https://2024.elixirconf.com/schedule/#schedules) is actually titled "Livebook in the cloud: GPUs and clustered workflows in seconds".
  - DagsEoress 2y ago
    Very interesting, thanks for sharing! I didn't know Elixir was so invested in Machine Learning. I have a background in Erlang so I'll definitely dive in. Do you know of any individuals / companies that have had success with Elixir+ML in production?
    - thibaut_barrere 2y ago
      You welcome. Afaik it has been a long-time bet done by José Valim (former Rubyist & author of e.g. Devise) and others, this is not something that is going away anytime. Each library has been building on top of the previous libraries & abstractions (including transpiling Elixir instructions into GPU code, see "defn" etc). Since you mention Erlang, there is even a Machine Learning Working Group at https://erlef.org/wg/machine-learning https://erlef.org/wg/machine-learning. The most iconic & advertised case I'm aware of is the work done at Amplified https://www.amplified.ai https://www.amplified.ai ; this has been the topic of a Keynote at ElixirConf EU this year, which you can find at https://www.youtube.com/watch?v=5FlZHkc4Mq4 https://www.youtube.com/watch?v=5FlZHkc4Mq4. I am also starting to use ML + Elixir in production and I'm aware of other individuals doing so. I do not have a registry of companies doing so, but we're seeing more and more experienced ML practitioners mentioning they are coming from Python and willing to try something different (e.g. https://elixirforum.com/t/data-science-and-machine-learning-workflows-in-elixir/53659 https://elixirforum.com/t/data-science-and-machine-learning-... and other posts on Elixir Forum). Hope this helps!
      - nextos 2y ago
        I had looked into Elixir/Erlang + ML before, and I keep revisiting it for Python alternatives. How developed is Elixir/Erlang in this area? What are the key advantages?
- michaelmior 2y ago
  It's certainly possible, but the provider would need to add support for the kernel. That is, I'm pretty sure you can't install a new kernel in these environments yourself.
- xipho 2y ago
  Perhaps not quite what you're looking for but Ankane does (a lot) of great work, e.g. https://github.com/ankane/torch.rb https://github.com/ankane/torch.rb.
- crabbone 2y ago
  R and Julia have had integration with Jupyter for a long time. Even so, trying to avoid Python in the world of Jupyter will put you in a very tough spot. In general, doesn't matter how much you dislike it, there's no real way around it. You'll have to face it in some capacity whether you like it or not.
isodev 2y ago
I love the idea. It's similar to what the Elixir folks have been working on with Livebook https://livebook.dev https://livebook.dev which seems somewhat more refined on the UI side + the benefits of distributed erlang/elixir (e.g. a livebook can interface with a live system and interact with the remote application/gpu etc).
- michaelmior 2y ago
  Jupyter notebooks have been around for almost a decade now. The interesting piece here is adding Ruby as a language that can be used instead of the original Python. (There have been many other languages integrated with Jupyter as well.)
  - lucasoshiro 2y ago
    Jupyter means "Julia, Python, R", the three original languages that it was created for. Nowadays you can add support for many languages by using other kernels. If you want to see the available kernels for each language you can check it here: https://github.com/jupyter/jupyter/wiki/Jupyter-kernels https://github.com/jupyter/jupyter/wiki/Jupyter-kernels
    - dpflan 2y ago
      Ruby fits in here nicely then...
    - fifilura 2y ago
      JuPytR aaaah! TIL!
    - michaelmior 2y ago
      Didn't know that. I always thought of Jupyter as originally a Python thing. Thanks for sharing that :)
      - jcheng 2y ago
        Jupyter is a rebrand of iPython, so you were right the first time!
BaculumMeumEst 2y ago
Does anyone have a way to use generative AI to generate Jupyter notebooks? I've tried with prompts but it chokes on the markup, and also I'm wondering if the markup wastes too much context for that to be a good idea anyways. Right now I just use Cursor or Claude, copy replies, and chop them up into blocks manually.
- prasoonds 2y ago
  We've made an open source fork of Jupyter - kind of like Cursor but for Jupyter. See GH: https://github.com/pretzelai/pretzelai/ https://github.com/pretzelai/pretzelai/ You can install it with pip install pretzelai (in a new environment preferably) - then run it with pretzel lab. You can bring your own keys or use the default free (for now) AI server. We also have a hosted version to make it easy to try it out: https://pretzelai.app https://pretzelai.app Would love to get your feedback!
  - BaculumMeumEst 2y ago
    Can it reliably generate interspersed blocks of markup and code with a single prompt?
    - prasoonds 2y ago
      Hmm do you mean you want to create multiple cells from a single prompt - some code cells, then some markdown cells, then some code cells and so on? The sidebar can certainly produce code mixed with markdown but right now, we process the markdown and show visually. https://imgur.com/a/bpYu8yN https://imgur.com/a/bpYu8yN The cell level Cmd + K shortcut only works on a given cell to create or edit code and fix errors. Just tested it and it generates markdown well (just start your prompt with "this is a markdown cell") https://imgur.com/VuDciQN https://imgur.com/VuDciQN In the sidebar/chat window, it should be trivial to not parse the markdown and just show it raw. I'll work on it. In the main notebook, it's a bit harder but we are planning to allow multi-cell insertions but it will probably take 2-3 weeks.
      - 2y ago
engine_y 2y ago
[flagged]
- ironmagma 2y ago
  I was talking with a coworker recently who was asking the question of why Ruby wasn’t the data science language of choice. I am still not sure the answer. Why not? Ruby seems like it would be great for math.
  - engine_y 2y ago
    Well maybe it should be. I was thinking the same about languages which are better suited for data wrangling. But to me, the reason python is the DS language shows how good evolution is and survival of the fittest. You and I would probably argue that a DS language should have strong static typing but in nature what prevailed was the opposite. A nimble, easy, no fuss, comedy oriented named language.
    - short_sells_poo 2y ago
      From my experience, python won out the research crowd exactly for the reasons you mentioned, but also because it is a bit of a swiss army knife platform. Python has plenty of libraries for doing pretty much anything these days. Computer Vision? Can do. ML? Can do. Dashboards? Can do. And the list goes on. This means that a researcher can do their research in python and then also do the implementation in the same language and platform they are familiar with. If someone told me to use rust for doing any sort of interactive/iterative numerical research, I'd send them for the hills. And I love Rust. But when I want to quickly learn about the properties of some data, or try to use dataset A to predict dataset B, I don't give a rats ass about types, or type safety, or proper programming idioms. I just want to look at my data. Then, if I convince myself that I have some sort of model that I want to use, I could theoretically reach for Rust, but generally I don't need to. I can just take the output of the research - more or less verbatim - and start using it in production. Often with minimal changes. Yes, sometimes circumstances require to use something more performant, but these days you also have Numba or Nuitka to optimize some hot loops. For the longest time I wanted Julia to become a success, because I felt that it combined in one language and platform the possibility to write easy no-fuss, scalable, performant and robust code. A language which could evolve a codebase from initial research analysis to strongly typed production code. Unfortunately it never seems to have gained traction, so we are left with the next best thing: python. It's not really good at anything (as a language), but it's the least bad out of everything out there and the libraries are excellent.
Kalanos 2y ago
i thought there already was a ruby kernel. are you the original developer or a new maintainer of iruby?
nomilk 2y ago
Rubyter notebook =)
joshdavham 2y ago
This looks great! I’m already running Node as well as python in Jupyter notebooks so to see more languages being added is always great
- nichochar 2y ago
  Do you like using Node in Jupyter? Are there any downsides / problems that you find with it? We're trying to build a TypeScript notebook and I'm very interested in what people's current tooling for this looks like today.
  - joshdavham 2y ago
    I like being able to use Node in Jupyter, but I can't say that how I'm using it feels as natural as it does with Python. I'm way stronger at python than node so it was pretty hard to get set up with it. There's also probably a way to do it, but I haven't figured out how to use ESModule imports in Jupyter, so that's been a bit annoying also. It also hasn't crossed my mind until literally now, that one could write typescript in jupyter notebooks given that typescript needs to be compiled. Do you have a link to your current work? I think I'd be interested
    - nichochar 2y ago
      Imports is the thing that makes it useless. You cannot import stuff normally in the kernel.. Yes, my current project is https://github.com/srcbookdev/srcbook https://github.com/srcbookdev/srcbook
  - quackware 2y ago
    Deno has support for Jupyter notebooks which is quite nice. https://docs.deno.com/runtime/manual/tools/jupyter/ https://docs.deno.com/runtime/manual/tools/jupyter/
    - joshdavham 2y ago
      Cool maybe I'll take a look at that too
DwnVoteHoneyPot 2y ago
This is a dead project. Everything in the SciRuby/sciruby-notebooks Github repository hasn't been updated in 8,9 years.
- bunderbunder 2y ago
  This page is talking about a project called iRuby, which I assume is unrelated or successor. The repository this page links had a 0.8.0 release exactly one month ago. The same org does also have a sciruby project that hasn't been touched in 8 or 9 years like you say.
- keithxm23 2y ago
  While the example-notebooks repo you've mentioned is not actively maintained it looks like the IRuby kernel used by Jupyter still is maintained: https://github.com/SciRuby/iruby https://github.com/SciRuby/iruby
bmacho 2y ago
Needs (2016)
vidarh 2y ago
This renders better for me straight from Github than it does on nbviewer.org: https://github.com/SciRuby/sciruby-notebooks/blob/master/getting_started.ipynb https://github.com/SciRuby/sciruby-notebooks/blob/master/get...
turriblegrapes 2y ago
nbviewer has nothing to do with ruby specifically, this example links to a notebook using existing iruby kernel > nbviewer is an open source project under the larger Project Jupyter initiative > along with other projects like Jupyter Notebook, JupyterLab, and JupyterHub. https://nbviewer.org/faq#what-is-nbviewer https://nbviewer.org/faq#what-is-nbviewer
kinduff 2y ago
There is also jupyter_on_rails [0] which integrates both. Using it feels so good, I love how an app can suddently become a playground or a sandbox. [0]: https://github.com/Yuki-Inoue/jupyter_on_rails https://github.com/Yuki-Inoue/jupyter_on_rails
- Alifatisk 2y ago
  That's so bizzare yet cool!
portpecos 2y ago
Finally!!

181 points by Alifatisk 2y ago

Is any of this usable on hosted GPU notebooks (e.g. paperspace)? I'm currently diving into Machine Learning using Python + Scikit-learn, and I'd love to one day replace Python with Ruby. But looking at the current ML ecosystem I don't see that happening. Does anyone have experience building (Supervised / Unsupervised) models using something other than Python (including deployment)?

qsort 2y ago
Nbviewer is just a service that lets you host notebooks, there is no computation there. > Does anyone have experience building (Supervised / Unsupervised) models using something other than Python (including deployment)? XGBoost/LightGBM have a C API and can be used from pretty much anything, deployment is not a problem. Practically building models is more about dealing with data, the ecosystem tends to revolve around Python and R for that reason.
thibaut_barrere 2y ago
Since you wrote "using something other than Python" and not necessarily only Ruby, definitely look into Livebook and Elixir, and the whole ecosystem around it, including: - https://github.com/elixir-nx/axon https://github.com/elixir-nx/axon Nx-powered Neural Networks - https://github.com/elixir-nx/nx https://github.com/elixir-nx/nx Multi-dimensional arrays (tensors) and numerical definitions for Elixir - https://github.com/elixir-nx/scholar https://github.com/elixir-nx/scholar Traditional machine learning on top of Nx - https://github.com/elixir-nx/bumblebee https://github.com/elixir-nx/bumblebee Pre-trained Neural Network models in Axon (+ Models integration) - https://github.com/elixir-explorer/explorer https://github.com/elixir-explorer/explorer Series (one-dimensional) and dataframes (two-dimensional) for fast and elegant data exploration in Elixir - https://fly.io/blog/rethinking-serverless-with-flame/ https://fly.io/blog/rethinking-serverless-with-flame/ (for offloading large work to remote containers) - https://www.youtube.com/watch?v=RABXu7zqnT0 https://www.youtube.com/watch?v=RABXu7zqnT0 InstructorEx And of course Livebook (https://livebook.dev https://livebook.dev) Old but interesting video https://www.youtube.com/watch?v=g3oyh3g1AtQ https://www.youtube.com/watch?v=g3oyh3g1AtQ (Bumblebee: GPT2, Stable Diffusion, and more in Elixir) A talk on the upcoming Elixir conference (https://2024.elixirconf.com/schedule/#schedules https://2024.elixirconf.com/schedule/#schedules) is actually titled "Livebook in the cloud: GPUs and clustered workflows in seconds".
- DagsEoress 2y ago
  Very interesting, thanks for sharing! I didn't know Elixir was so invested in Machine Learning. I have a background in Erlang so I'll definitely dive in. Do you know of any individuals / companies that have had success with Elixir+ML in production?
  - thibaut_barrere 2y ago
    You welcome. Afaik it has been a long-time bet done by José Valim (former Rubyist & author of e.g. Devise) and others, this is not something that is going away anytime. Each library has been building on top of the previous libraries & abstractions (including transpiling Elixir instructions into GPU code, see "defn" etc). Since you mention Erlang, there is even a Machine Learning Working Group at https://erlef.org/wg/machine-learning https://erlef.org/wg/machine-learning. The most iconic & advertised case I'm aware of is the work done at Amplified https://www.amplified.ai https://www.amplified.ai ; this has been the topic of a Keynote at ElixirConf EU this year, which you can find at https://www.youtube.com/watch?v=5FlZHkc4Mq4 https://www.youtube.com/watch?v=5FlZHkc4Mq4. I am also starting to use ML + Elixir in production and I'm aware of other individuals doing so. I do not have a registry of companies doing so, but we're seeing more and more experienced ML practitioners mentioning they are coming from Python and willing to try something different (e.g. https://elixirforum.com/t/data-science-and-machine-learning-workflows-in-elixir/53659 https://elixirforum.com/t/data-science-and-machine-learning-... and other posts on Elixir Forum). Hope this helps!
    - nextos 2y ago
      I had looked into Elixir/Erlang + ML before, and I keep revisiting it for Python alternatives. How developed is Elixir/Erlang in this area? What are the key advantages?
michaelmior 2y ago
It's certainly possible, but the provider would need to add support for the kernel. That is, I'm pretty sure you can't install a new kernel in these environments yourself.
xipho 2y ago
Perhaps not quite what you're looking for but Ankane does (a lot) of great work, e.g. https://github.com/ankane/torch.rb https://github.com/ankane/torch.rb.
crabbone 2y ago
R and Julia have had integration with Jupyter for a long time. Even so, trying to avoid Python in the world of Jupyter will put you in a very tough spot. In general, doesn't matter how much you dislike it, there's no real way around it. You'll have to face it in some capacity whether you like it or not.

isodev 2y ago

I love the idea. It's similar to what the Elixir folks have been working on with Livebook https://livebook.dev https://livebook.dev which seems somewhat more refined on the UI side + the benefits of distributed erlang/elixir (e.g. a livebook can interface with a live system and interact with the remote application/gpu etc).

michaelmior 2y ago
Jupyter notebooks have been around for almost a decade now. The interesting piece here is adding Ruby as a language that can be used instead of the original Python. (There have been many other languages integrated with Jupyter as well.)
- lucasoshiro 2y ago
  Jupyter means "Julia, Python, R", the three original languages that it was created for. Nowadays you can add support for many languages by using other kernels. If you want to see the available kernels for each language you can check it here: https://github.com/jupyter/jupyter/wiki/Jupyter-kernels https://github.com/jupyter/jupyter/wiki/Jupyter-kernels
  - dpflan 2y ago
    Ruby fits in here nicely then...
  - fifilura 2y ago
    JuPytR aaaah! TIL!
  - michaelmior 2y ago
    Didn't know that. I always thought of Jupyter as originally a Python thing. Thanks for sharing that :)
    - jcheng 2y ago
      Jupyter is a rebrand of iPython, so you were right the first time!

BaculumMeumEst 2y ago

Does anyone have a way to use generative AI to generate Jupyter notebooks? I've tried with prompts but it chokes on the markup, and also I'm wondering if the markup wastes too much context for that to be a good idea anyways. Right now I just use Cursor or Claude, copy replies, and chop them up into blocks manually.

prasoonds 2y ago
We've made an open source fork of Jupyter - kind of like Cursor but for Jupyter. See GH: https://github.com/pretzelai/pretzelai/ https://github.com/pretzelai/pretzelai/ You can install it with pip install pretzelai (in a new environment preferably) - then run it with pretzel lab. You can bring your own keys or use the default free (for now) AI server. We also have a hosted version to make it easy to try it out: https://pretzelai.app https://pretzelai.app Would love to get your feedback!
- BaculumMeumEst 2y ago
  Can it reliably generate interspersed blocks of markup and code with a single prompt?
  - prasoonds 2y ago
    Hmm do you mean you want to create multiple cells from a single prompt - some code cells, then some markdown cells, then some code cells and so on? The sidebar can certainly produce code mixed with markdown but right now, we process the markdown and show visually. https://imgur.com/a/bpYu8yN https://imgur.com/a/bpYu8yN The cell level Cmd + K shortcut only works on a given cell to create or edit code and fix errors. Just tested it and it generates markdown well (just start your prompt with "this is a markdown cell") https://imgur.com/VuDciQN https://imgur.com/VuDciQN In the sidebar/chat window, it should be trivial to not parse the markdown and just show it raw. I'll work on it. In the main notebook, it's a bit harder but we are planning to allow multi-cell insertions but it will probably take 2-3 weeks.
    - 2y ago

engine_y 2y ago

[flagged]

ironmagma 2y ago
I was talking with a coworker recently who was asking the question of why Ruby wasn’t the data science language of choice. I am still not sure the answer. Why not? Ruby seems like it would be great for math.
- engine_y 2y ago
  Well maybe it should be. I was thinking the same about languages which are better suited for data wrangling. But to me, the reason python is the DS language shows how good evolution is and survival of the fittest. You and I would probably argue that a DS language should have strong static typing but in nature what prevailed was the opposite. A nimble, easy, no fuss, comedy oriented named language.
  - short_sells_poo 2y ago
    From my experience, python won out the research crowd exactly for the reasons you mentioned, but also because it is a bit of a swiss army knife platform. Python has plenty of libraries for doing pretty much anything these days. Computer Vision? Can do. ML? Can do. Dashboards? Can do. And the list goes on. This means that a researcher can do their research in python and then also do the implementation in the same language and platform they are familiar with. If someone told me to use rust for doing any sort of interactive/iterative numerical research, I'd send them for the hills. And I love Rust. But when I want to quickly learn about the properties of some data, or try to use dataset A to predict dataset B, I don't give a rats ass about types, or type safety, or proper programming idioms. I just want to look at my data. Then, if I convince myself that I have some sort of model that I want to use, I could theoretically reach for Rust, but generally I don't need to. I can just take the output of the research - more or less verbatim - and start using it in production. Often with minimal changes. Yes, sometimes circumstances require to use something more performant, but these days you also have Numba or Nuitka to optimize some hot loops. For the longest time I wanted Julia to become a success, because I felt that it combined in one language and platform the possibility to write easy no-fuss, scalable, performant and robust code. A language which could evolve a codebase from initial research analysis to strongly typed production code. Unfortunately it never seems to have gained traction, so we are left with the next best thing: python. It's not really good at anything (as a language), but it's the least bad out of everything out there and the libraries are excellent.

Kalanos 2y ago

i thought there already was a ruby kernel. are you the original developer or a new maintainer of iruby?

nomilk 2y ago

Rubyter notebook =)

joshdavham 2y ago

This looks great! I’m already running Node as well as python in Jupyter notebooks so to see more languages being added is always great

nichochar 2y ago
Do you like using Node in Jupyter? Are there any downsides / problems that you find with it? We're trying to build a TypeScript notebook and I'm very interested in what people's current tooling for this looks like today.
- joshdavham 2y ago
  I like being able to use Node in Jupyter, but I can't say that how I'm using it feels as natural as it does with Python. I'm way stronger at python than node so it was pretty hard to get set up with it. There's also probably a way to do it, but I haven't figured out how to use ESModule imports in Jupyter, so that's been a bit annoying also. It also hasn't crossed my mind until literally now, that one could write typescript in jupyter notebooks given that typescript needs to be compiled. Do you have a link to your current work? I think I'd be interested
  - nichochar 2y ago
    Imports is the thing that makes it useless. You cannot import stuff normally in the kernel.. Yes, my current project is https://github.com/srcbookdev/srcbook https://github.com/srcbookdev/srcbook
- quackware 2y ago
  Deno has support for Jupyter notebooks which is quite nice. https://docs.deno.com/runtime/manual/tools/jupyter/ https://docs.deno.com/runtime/manual/tools/jupyter/
  - joshdavham 2y ago
    Cool maybe I'll take a look at that too

DwnVoteHoneyPot 2y ago

This is a dead project. Everything in the SciRuby/sciruby-notebooks Github repository hasn't been updated in 8,9 years.

bunderbunder 2y ago
This page is talking about a project called iRuby, which I assume is unrelated or successor. The repository this page links had a 0.8.0 release exactly one month ago. The same org does also have a sciruby project that hasn't been touched in 8 or 9 years like you say.
keithxm23 2y ago
While the example-notebooks repo you've mentioned is not actively maintained it looks like the IRuby kernel used by Jupyter still is maintained: https://github.com/SciRuby/iruby https://github.com/SciRuby/iruby

bmacho 2y ago

Needs (2016)

vidarh 2y ago

This renders better for me straight from Github than it does on nbviewer.org: https://github.com/SciRuby/sciruby-notebooks/blob/master/getting_started.ipynb https://github.com/SciRuby/sciruby-notebooks/blob/master/get...

turriblegrapes 2y ago

nbviewer has nothing to do with ruby specifically, this example links to a notebook using existing iruby kernel > nbviewer is an open source project under the larger Project Jupyter initiative > along with other projects like Jupyter Notebook, JupyterLab, and JupyterHub. https://nbviewer.org/faq#what-is-nbviewer https://nbviewer.org/faq#what-is-nbviewer

kinduff 2y ago

There is also jupyter_on_rails [0] which integrates both. Using it feels so good, I love how an app can suddently become a playground or a sandbox. [0]: https://github.com/Yuki-Inoue/jupyter_on_rails https://github.com/Yuki-Inoue/jupyter_on_rails

Alifatisk 2y ago
That's so bizzare yet cool!

portpecos 2y ago

Finally!!