2 ms·
Since you wrote "using something other than Python" and not necessarily only Ruby, definitely look into Livebook and Elixir, and the whole ecosystem around it,
by thibaut_barrere 2y ago
Since you wrote "using something other than Python" and not necessarily only Ruby, definitely look into Livebook and Elixir, and the whole ecosystem around it, including:
- https://github.com/elixir-nx/axon https://github.com/elixir-nx/axon Nx-powered Neural Networks
- https://github.com/elixir-nx/nx https://github.com/elixir-nx/nx Multi-dimensional arrays (tensors) and numerical definitions for Elixir
- https://github.com/elixir-nx/scholar https://github.com/elixir-nx/scholar Traditional machine learning on top of Nx
- https://github.com/elixir-nx/bumblebee https://github.com/elixir-nx/bumblebee Pre-trained Neural Network models in Axon (+ Models integration)
- https://github.com/elixir-explorer/explorer https://github.com/elixir-explorer/explorer Series (one-dimensional) and dataframes (two-dimensional) for fast and elegant data exploration in Elixir
- https://fly.io/blog/rethinking-serverless-with-flame/ https://fly.io/blog/rethinking-serverless-with-flame/ (for offloading large work to remote containers)
- https://www.youtube.com/watch?v=RABXu7zqnT0 https://www.youtube.com/watch?v=RABXu7zqnT0 InstructorEx
And of course Livebook (https://livebook.dev https://livebook.dev)
Old but interesting video https://www.youtube.com/watch?v=g3oyh3g1AtQ https://www.youtube.com/watch?v=g3oyh3g1AtQ (Bumblebee: GPT2, Stable Diffusion, and more in Elixir)
A talk on the upcoming Elixir conference (https://2024.elixirconf.com/schedule/#schedules https://2024.elixirconf.com/schedule/#schedules) is actually titled "Livebook in the cloud: GPUs and clustered workflows in seconds".
- DagsEoress 2y agoVery interesting, thanks for sharing! I didn't know Elixir was so invested in Machine Learning. I have a background in Erlang so I'll definitely dive in. Do you know of any individuals / companies that have had success with Elixir+ML in production?
- thibaut_barrere 2y agoYou welcome. Afaik it has been a long-time bet done by José Valim (former Rubyist & author of e.g. Devise) and others, this is not something that is going away anytime. Each library has been building on top of the previous libraries & abstractions (including transpiling Elixir instructions into GPU code, see "defn" etc). Since you mention Erlang, there is even a Machine Learning Working Group at https://erlef.org/wg/machine-learning https://erlef.org/wg/machine-learning. The most iconic & advertised case I'm aware of is the work done at Amplified https://www.amplified.ai https://www.amplified.ai ; this has been the topic of a Keynote at ElixirConf EU this year, which you can find at https://www.youtube.com/watch?v=5FlZHkc4Mq4 https://www.youtube.com/watch?v=5FlZHkc4Mq4. I am also starting to use ML + Elixir in production and I'm aware of other individuals doing so. I do not have a registry of companies doing so, but we're seeing more and more experienced ML practitioners mentioning they are coming from Python and willing to try something different (e.g. https://elixirforum.com/t/data-science-and-machine-learning-workflows-in-elixir/53659 https://elixirforum.com/t/data-science-and-machine-learning-... and other posts on Elixir Forum). Hope this helps!
- nextos 2y agoI had looked into Elixir/Erlang + ML before, and I keep revisiting it for Python alternatives. How developed is Elixir/Erlang in this area? What are the key advantages?