113 ms·

Hi everyone, glad to be back here with the official announcement! It is late here but feel free to drop questions and I will answer them when I am up. Meanwhil

Hi everyone, glad to be back here with the official announcement! It is late here but feel free to drop questions and I will answer them when I am up. Meanwhile I hope you will enjoy the content we put out: the announcements, example apps, and sample notebooks!

tommica 4y ago
Its really interesting- need to play with this!
losvedir 4y ago
What's the motivation for the name? I get "numbat" to "numerical", but not immediately seeing any connection for bumblebee. Maybe a more serious question: is anyone using Elixir ML in production? I'm absolutely gobsmacked at the quantity and quality of development effort that's gone into it (and use Livebook daily, though not for ML stuff). It's clearly a major focus for the team. I'm wondering if it's ready for production adoption, and if so, if anyone has used it "in anger" yet.
- seanmor5 4y ago
  We are all really into bumblebees, specifically the kind that can transform into 1977 Yellow Camaros
- afhammad 4y ago
  > Maybe a more serious question: is anyone using Elixir ML in production? Checkout this recent talk about how https://www.amplified.ai/ https://www.amplified.ai/ moved from Python to an Elixir ML stack: https://www.youtube.com/watch?v=Y2Nr4dNu6hI https://www.youtube.com/watch?v=Y2Nr4dNu6hI
- cigrainger 4y ago
  We use it in production at Amplified! It’s been a joy. We train our own large language models and have fine tuned and deployed using Elixir. I talked about it at ElixirConf this year (https://m.youtube.com/watch?v=Y2Nr4dNu6hI https://m.youtube.com/watch?v=Y2Nr4dNu6hI). We were able to completely eliminate a few python services and consolidate to all Elixir for ETL and ML.
pbowyer 4y ago
Congratulations on this release! If I have a Huggingface model that I've finetuned, can I load it using Bumblebee? I fine-tuned ConvNext and changed it into a multi-label classifier and saved it as a PyTorch model. It works great but being able to use it in LiveBook instead of Jupyter Notebook would be fantastic. I think I'd have to convert the format, but what then?
- seanmor5 4y ago
  No need to convert the format! Bumblebee supports loading parameters directly from PyTorch. You can just specify `{:local, path/to/checkpoint}` in `Bumblebee.load_model`
chem83 4y ago
Congrats on the launch! Muito legal! Alternatively to EXLA/Torchx, any thoughts on supporting an ML compiler frontend like Google IREE/MLIR by generating StableHLO or LinAlg? This could pave the way towards supporting multiple hardware targets (Vulkan-based, RVV-based, SME-based etc.) with minimal effort from the framework.
- seanmor5 4y ago
  There is a project in the ecosystem that builds on top of MLIR and can target any MLIR dialect, but it is not mature enough yet. I think it would be great to see!
- josevalim 4y ago
  Yes, we have been eyeing IREE but no work has started yet. However, our compiler architecture was designed exactly to make this possible, so it will be definitely welcome if someone decides to trail ahead on this road!
AlphaWeaver 4y ago
Congrats on the launch! Been eagerly sitting in the #elixir IRC channel all day :) How easy would it be to support OpenAI's new Whisper transcription model in Bumblebee?
- seanmor5 4y ago
  It should not be too hard! From what I saw Whisper is similar to Bart, and we have Bart. The missing piece is a library for audio processing to tensors.
  - tylrbrkr 4y ago
    Would Nx Signal be that missing piece? https://github.com/polvalente/nx-signal https://github.com/polvalente/nx-signal
    - seanmor5 4y ago
      For some of it! I was thinking more something for loading different audio encodings to Nx tensors…so doing everything up until Nx
- ricketycricket 4y ago
  I don't have the ML chops to make it happen, but whisper would be insanely useful to have in the tool chest.
shay_ker 4y ago
Congrats on the launch, this is fantastic. I love seeing these releases. They bring me so much joy & excitement. Can you provide some thoughts on the benefits of doing ML on Elixir vs. Python? Is the benefit the language semantics? Is it much easier to get distributed work in Elixir ML vs Python ML? Are the tools better/smoother? Are there ML ops improvements? Perhaps there’s a blog post I missed :)
- josevalim 4y ago
  I have written a bit about potential benefits in the initial announcement of Nx [0] and then on its first release [1]. In a nutshell, there has been trends in Python (such as JAX and Thinc.ai) that argue functional programming can provide better abstractions and more composable building blocks for deep learning libraries. And I believe Elixir, as a functional language with Lisp-style macros, is in an excellent position to exploit that - as seen in "numerical definitions" which compile a subset of Elixir to the CPU/GPU. I also think the Erlang VM, with its distribution and network capabilities, can provide exciting developments in the realm of federated and distributed learning. We aren't exploring those aspects yet but we are getting closer to having the foundation to do so. Regarding ML ops, I believe one main advantage is explained in this video announcement. When deploying a ML model with Nx, you can embed the model within your applications: you don't need a 3rd-party service because we batch and route requests from multiple cores and multiple nodes withing Erlang/Elixir. This can be specially beneficial for projects like Nerves [5] and we will see how it evolves in the long term (as we _just_ announced it). Finally, one of the benefits on starting from scratch after Python has paved the way is that we can learn from its ecosystem and provide a unified experience. You can think of Nx as Numpy+JAX+TFServing all in one place and we hope that doing so streamlines the developer experience. This also means libraries like Scholar [2] (which aims to serve a similar role as SciPy) and Meow [3] (for Genetic Algorithms) get to use the same abstractions and compile to the CPU/GPU. The latter can show an order of magnitude improvement over other currently used frameworks [4]. [0]: https://dashbit.co/blog/nx-numerical-elixir-is-now-publicly-available https://dashbit.co/blog/nx-numerical-elixir-is-now-publicly-... [1]: https://dashbit.co/blog/elixir-and-machine-learning-nx-v0.1 https://dashbit.co/blog/elixir-and-machine-learning-nx-v0.1 [2]: https://github.com/elixir-nx/scholar/ https://github.com/elixir-nx/scholar/ [3]: https://github.com/jonatanklosko/meow https://github.com/jonatanklosko/meow [4]: https://dl.acm.org/doi/10.1145/3512290.3528753 https://dl.acm.org/doi/10.1145/3512290.3528753 [5]: https://www.nerves-project.org/ https://www.nerves-project.org/

tommica 4y ago

Its really interesting- need to play with this!

losvedir 4y ago

What's the motivation for the name? I get "numbat" to "numerical", but not immediately seeing any connection for bumblebee. Maybe a more serious question: is anyone using Elixir ML in production? I'm absolutely gobsmacked at the quantity and quality of development effort that's gone into it (and use Livebook daily, though not for ML stuff). It's clearly a major focus for the team. I'm wondering if it's ready for production adoption, and if so, if anyone has used it "in anger" yet.

seanmor5 4y ago
We are all really into bumblebees, specifically the kind that can transform into 1977 Yellow Camaros
afhammad 4y ago
> Maybe a more serious question: is anyone using Elixir ML in production? Checkout this recent talk about how https://www.amplified.ai/ https://www.amplified.ai/ moved from Python to an Elixir ML stack: https://www.youtube.com/watch?v=Y2Nr4dNu6hI https://www.youtube.com/watch?v=Y2Nr4dNu6hI
cigrainger 4y ago
We use it in production at Amplified! It’s been a joy. We train our own large language models and have fine tuned and deployed using Elixir. I talked about it at ElixirConf this year (https://m.youtube.com/watch?v=Y2Nr4dNu6hI https://m.youtube.com/watch?v=Y2Nr4dNu6hI). We were able to completely eliminate a few python services and consolidate to all Elixir for ETL and ML.

pbowyer 4y ago

Congratulations on this release! If I have a Huggingface model that I've finetuned, can I load it using Bumblebee? I fine-tuned ConvNext and changed it into a multi-label classifier and saved it as a PyTorch model. It works great but being able to use it in LiveBook instead of Jupyter Notebook would be fantastic. I think I'd have to convert the format, but what then?

seanmor5 4y ago
No need to convert the format! Bumblebee supports loading parameters directly from PyTorch. You can just specify `{:local, path/to/checkpoint}` in `Bumblebee.load_model`

chem83 4y ago

Congrats on the launch! Muito legal! Alternatively to EXLA/Torchx, any thoughts on supporting an ML compiler frontend like Google IREE/MLIR by generating StableHLO or LinAlg? This could pave the way towards supporting multiple hardware targets (Vulkan-based, RVV-based, SME-based etc.) with minimal effort from the framework.

seanmor5 4y ago
There is a project in the ecosystem that builds on top of MLIR and can target any MLIR dialect, but it is not mature enough yet. I think it would be great to see!
josevalim 4y ago
Yes, we have been eyeing IREE but no work has started yet. However, our compiler architecture was designed exactly to make this possible, so it will be definitely welcome if someone decides to trail ahead on this road!

AlphaWeaver 4y ago

Congrats on the launch! Been eagerly sitting in the #elixir IRC channel all day :) How easy would it be to support OpenAI's new Whisper transcription model in Bumblebee?

seanmor5 4y ago
It should not be too hard! From what I saw Whisper is similar to Bart, and we have Bart. The missing piece is a library for audio processing to tensors.
- tylrbrkr 4y ago
  Would Nx Signal be that missing piece? https://github.com/polvalente/nx-signal https://github.com/polvalente/nx-signal
  - seanmor5 4y ago
    For some of it! I was thinking more something for loading different audio encodings to Nx tensors…so doing everything up until Nx
ricketycricket 4y ago
I don't have the ML chops to make it happen, but whisper would be insanely useful to have in the tool chest.

shay_ker 4y ago

Congrats on the launch, this is fantastic. I love seeing these releases. They bring me so much joy & excitement. Can you provide some thoughts on the benefits of doing ML on Elixir vs. Python? Is the benefit the language semantics? Is it much easier to get distributed work in Elixir ML vs Python ML? Are the tools better/smoother? Are there ML ops improvements? Perhaps there’s a blog post I missed :)

josevalim 4y ago
I have written a bit about potential benefits in the initial announcement of Nx [0] and then on its first release [1]. In a nutshell, there has been trends in Python (such as JAX and Thinc.ai) that argue functional programming can provide better abstractions and more composable building blocks for deep learning libraries. And I believe Elixir, as a functional language with Lisp-style macros, is in an excellent position to exploit that - as seen in "numerical definitions" which compile a subset of Elixir to the CPU/GPU. I also think the Erlang VM, with its distribution and network capabilities, can provide exciting developments in the realm of federated and distributed learning. We aren't exploring those aspects yet but we are getting closer to having the foundation to do so. Regarding ML ops, I believe one main advantage is explained in this video announcement. When deploying a ML model with Nx, you can embed the model within your applications: you don't need a 3rd-party service because we batch and route requests from multiple cores and multiple nodes withing Erlang/Elixir. This can be specially beneficial for projects like Nerves [5] and we will see how it evolves in the long term (as we _just_ announced it). Finally, one of the benefits on starting from scratch after Python has paved the way is that we can learn from its ecosystem and provide a unified experience. You can think of Nx as Numpy+JAX+TFServing all in one place and we hope that doing so streamlines the developer experience. This also means libraries like Scholar [2] (which aims to serve a similar role as SciPy) and Meow [3] (for Genetic Algorithms) get to use the same abstractions and compile to the CPU/GPU. The latter can show an order of magnitude improvement over other currently used frameworks [4]. [0]: https://dashbit.co/blog/nx-numerical-elixir-is-now-publicly-available https://dashbit.co/blog/nx-numerical-elixir-is-now-publicly-... [1]: https://dashbit.co/blog/elixir-and-machine-learning-nx-v0.1 https://dashbit.co/blog/elixir-and-machine-learning-nx-v0.1 [2]: https://github.com/elixir-nx/scholar/ https://github.com/elixir-nx/scholar/ [3]: https://github.com/jonatanklosko/meow https://github.com/jonatanklosko/meow [4]: https://dl.acm.org/doi/10.1145/3512290.3528753 https://dl.acm.org/doi/10.1145/3512290.3528753 [5]: https://www.nerves-project.org/ https://www.nerves-project.org/