4 ms·

Hi everyone, contributor to Livebook and Nx here. When we started the Numerical Elixir effort, we were excited about the possibilities of mixing projects like

Hi everyone, contributor to Livebook and Nx here. When we started the Numerical Elixir effort, we were excited about the possibilities of mixing projects like Google XLA's (from Tensorflow) and LibTorch (from PyTorch) with the Erlang VM abilities to run concurrent, distributed, and fault-tolerant software. I am very glad we are at a point where those ideas are coming to life and I explore part of it in the video. My favorite bit: making the tensor serving implementation cluster distributed took only 400 LOC (including docs and tests!): https://github.com/elixir-nx/nx/pull/1090 https://github.com/elixir-nx/nx/pull/1090 I'll be glad to answer questions about Nx or anything from Livebook's launch week!

smw 4y ago
Looks amazing! Small user feedback: Even reading the blog, after installing the windows app it's not obvious how to get to the machine learning demos. Also, after I found the +smart button from another page, on windows it fails due to lack of make (and presumably a set of compiler tools). This was frustrating trying to demo for someone on their computer.
dpflan 4y ago
Thanks for sharing! The distributed ML currently seems focused on model execution. I see another commenter's excitement about "Looking forward to NX transformations that take distributed training next level." -- which I agree, will be quite interesting. Where / how do you see Nx being used effectively in distributed training? Is distributed training a reality for open-sourced models to compete against big tech models?
- josevalim 4y ago
  For distributed training, one important feature is to be able to do GPU-to-GPU communication, such as allreduce, allgather, and all2all. Those are not supported at the moment but they are in our roadmap. At this level, however, it seems the language runtime itself plays a reduced role, so I don't expect the experience to be much different to, say, Python/JAX. For the second question, my understanding is that all big tech models rely on distributed training, so distributed training is a requisite for competing really.
DrewADesign 4y ago
Nothing specific about this project, and don't feel obligated to respond, but I just wanted to thank you for all the work you've done with Elixir and the related ecosystem. Great language, great tools, and a helpful, welcoming community. It was a perfect introduction to practical functional programming. Haven't found a big project for it yet, but I've done a bunch of little side projects since a friend who worked at Appcues gave me the hard sell on it around 2018.
Dowwie 4y ago
Is anyone working on audio libraries that will enable streaming audio chunks for Whisper processing? Saving audio files into a local file system, running ffmpeg to chunk, and then sending them off to Whisper is very tactical..
- brentjanderson 4y ago
  You may be able to incorporate the [Membrane Framework](https://membrane.stream/ https://membrane.stream/) to do that. Built in Elixir, deals in those types of multimedia problems. I'm not an expert here, but I'd expect that capturing a sample using Membrane and piping it into Whisper should be doable.
- josevalim 4y ago
  The current pipeline expects PCM audio blobs and, if data is coming from a microphone in the browser, you can do the initial processing and conversion in the browser (see the JS in this single file Phoenix app speech to text example [0]). On the other hand, if you expect a variety of formats (mp3, wav, etc), then shelling out or embedding ffmpeg is probably the quickest path to achieve something. The Membrane Framework[1] is an option here too which includes streaming. I believe Lars is going to do a cool demo with Membrane and ML at ElixirConf EU next week. [0]: https://github.com/elixir-nx/bumblebee/blob/main/examples/phoenix/speech_to_text.exs https://github.com/elixir-nx/bumblebee/blob/main/examples/ph... [1]: https://membrane.stream/ https://membrane.stream/
  - ricketycricket 4y ago
    > I believe Lars is going to do a cool demo with Membrane and ML at ElixirConf EU next week. Yes, the relevant part of his demo with the membrane pipeline appears to be here: https://github.com/lawik/lively/blob/master/lib/lively/media/pipeline.ex https://github.com/lawik/lively/blob/master/lib/lively/media...
    - Dowwie 4y ago
      limited in usefulness.. seems that Lars kept a MembraneTranscript library dependency private
pejrich 4y ago
> contributor to Livebook and Nx here While accurate, it's a bit of an understatement :) Thanks for all your work, Jose.
dangerboysteve 4y ago
As everyone is chiming in on, fantastic work by you and your team.
andy_ppp 4y ago
Do you ever think about why you’re probably a 100x programmer, is it just working memory and pure intelligence or some strategy or tactics that make you so good at this. Asking for a friend :-)

smw 4y ago

Looks amazing! Small user feedback: Even reading the blog, after installing the windows app it's not obvious how to get to the machine learning demos. Also, after I found the +smart button from another page, on windows it fails due to lack of make (and presumably a set of compiler tools). This was frustrating trying to demo for someone on their computer.

dpflan 4y ago

Thanks for sharing! The distributed ML currently seems focused on model execution. I see another commenter's excitement about "Looking forward to NX transformations that take distributed training next level." -- which I agree, will be quite interesting. Where / how do you see Nx being used effectively in distributed training? Is distributed training a reality for open-sourced models to compete against big tech models?

josevalim 4y ago
For distributed training, one important feature is to be able to do GPU-to-GPU communication, such as allreduce, allgather, and all2all. Those are not supported at the moment but they are in our roadmap. At this level, however, it seems the language runtime itself plays a reduced role, so I don't expect the experience to be much different to, say, Python/JAX. For the second question, my understanding is that all big tech models rely on distributed training, so distributed training is a requisite for competing really.

DrewADesign 4y ago

Nothing specific about this project, and don't feel obligated to respond, but I just wanted to thank you for all the work you've done with Elixir and the related ecosystem. Great language, great tools, and a helpful, welcoming community. It was a perfect introduction to practical functional programming. Haven't found a big project for it yet, but I've done a bunch of little side projects since a friend who worked at Appcues gave me the hard sell on it around 2018.

Dowwie 4y ago

Is anyone working on audio libraries that will enable streaming audio chunks for Whisper processing? Saving audio files into a local file system, running ffmpeg to chunk, and then sending them off to Whisper is very tactical..

brentjanderson 4y ago
You may be able to incorporate the [Membrane Framework](https://membrane.stream/ https://membrane.stream/) to do that. Built in Elixir, deals in those types of multimedia problems. I'm not an expert here, but I'd expect that capturing a sample using Membrane and piping it into Whisper should be doable.
josevalim 4y ago
The current pipeline expects PCM audio blobs and, if data is coming from a microphone in the browser, you can do the initial processing and conversion in the browser (see the JS in this single file Phoenix app speech to text example [0]). On the other hand, if you expect a variety of formats (mp3, wav, etc), then shelling out or embedding ffmpeg is probably the quickest path to achieve something. The Membrane Framework[1] is an option here too which includes streaming. I believe Lars is going to do a cool demo with Membrane and ML at ElixirConf EU next week. [0]: https://github.com/elixir-nx/bumblebee/blob/main/examples/phoenix/speech_to_text.exs https://github.com/elixir-nx/bumblebee/blob/main/examples/ph... [1]: https://membrane.stream/ https://membrane.stream/
- ricketycricket 4y ago
  > I believe Lars is going to do a cool demo with Membrane and ML at ElixirConf EU next week. Yes, the relevant part of his demo with the membrane pipeline appears to be here: https://github.com/lawik/lively/blob/master/lib/lively/media/pipeline.ex https://github.com/lawik/lively/blob/master/lib/lively/media...
  - Dowwie 4y ago
    limited in usefulness.. seems that Lars kept a MembraneTranscript library dependency private

pejrich 4y ago

> contributor to Livebook and Nx here While accurate, it's a bit of an understatement :) Thanks for all your work, Jose.

dangerboysteve 4y ago

As everyone is chiming in on, fantastic work by you and your team.

andy_ppp 4y ago

Do you ever think about why you’re probably a 100x programmer, is it just working memory and pure intelligence or some strategy or tactics that make you so good at this. Asking for a friend :-)