3 ms·

"Adding ML to X" is a powerful way to probe what X is really made of and maybe even give direction to efforts to improve X. That is because "ML" is a proxy for

"Adding ML to X" is a powerful way to probe what X is really made of and maybe even give direction to efforts to improve X. That is because "ML" is a proxy for an entire domain (data science) that has both very distinct requirements and patterns developed over decades and is now evolving very fast as it broke out of the whitecoat lab and (for good or bad) into our lives. The ascent of python is as the article noted "accidental" but that doesnt mean it is suboptimal given all the requirements of the domain and the available alternatives (here and now). Its natural to focus on an exceptional attribute (like concurrency) to motivate even having the discussion but this addresses only one of three critical requirements if you decide to do everything within one langauge. 1) Ability to experiment interactively (REPL and friends), expressivity, human oriented language design etc 2) Ability to build production scale models (training or estimation phase) utilizing heterogeneous and fiddly compute 3) Ability to deploy to diverse devices and footprint / latency requirements both natively and as part of a web application. NB: Its only here that the intrinsic qualities of erlang might put it above the rest Historically there was never an ecosystem than could do all that optimally within just one language. Some of the requirements are arguably even incompatible. The current preferred solution is not "python" but python/c++ which does some of that well (1, 2) and 3 only passably. The only declared single language contender is julia but its very far from being a complete ecosystem in the above sense. Another emerging possibility is mojo, as a superset of python. But that is not even available yet. Back to the erlang/elixir combo. Its not even clear to me if it can address all of the above in a passable way. But on the other hand ML is really shaking the game and what is mayne more imprortant is the vitality and resourcefulness of different communities to develop in the direction of the new rewuirement. We have not seen the end of ecosystem evolution.

josevalim 3y ago
Hello, creator of Elixir here (and co-creator of Nx). I like your breakdown of the requirements in three abilities. I think Elixir provides all three of them (with a caveat) and I will refer to two videos for those who want to learn more: * This video shows how you can use Livebook (open-source computational notebook environment) to explore pre-trained neural networks (from Hugging Face), learn how they work, and then deploy them as part of a Phoenix web application: https://news.livebook.dev/announcing-bumblebee-gpt2-stable-diffusion-and-more-in-elixir-3Op73O https://news.livebook.dev/announcing-bumblebee-gpt2-stable-d... * This video shows how to scale a neural network serving, first to run it concurrently, and then have it run distributed over multiple CPUs and GPUs: https://news.livebook.dev/distributed2-machine-learning-notebooks-with-elixir-and-livebook---launch-week-1---day-2-1aIlaw https://news.livebook.dev/distributed2-machine-learning-note... The caveat is about your second ability. The production scale models are, at the end of the day running native/cuda code, using the same backends as Python (Google XLA and LibTorch). What we do is that we compile your Elixir code into a graph which is then compiled to the CPU/GPU just-in time (in an approach similar to JAX). You can find more information here: https://github.com/elixir-nx/#why-elixir https://github.com/elixir-nx/#why-elixir
- nologic01 3y ago
  Personally I would love to see your vision succeed. Monocultures are intrinsically risky and limiting. Pretty soon we are all fish in a bowl, unable to see the water. What will it take? I am just an armchair philosopher in this respect. Like they say languages are only in part about communicating with the computer, they are about communicating with other people. So these three major tasks are really about winning over three different types of people and their peculiar brain wirings, motivations and aspirations. 1) and 2) are more introvert: mathematical types that want to express their inner abstract worlds in computer instructions in the least painful way. The famous data point is that pytorch became the tool of choice of academic ML publishing over the first mover tensorflow. Then you have the true computer nerds that love making the most of hardware and see in ML a major new stage of computing: HPC becoming mass market. 3) is more extrovert (keeping in mind the norms of the tech industry :-) developing and scaling delightful products and apps to be used by non-technical people. Erlang/elixir clearly offers something special in terms of 3) and that is important because in the end its all down to serving useful software to people.

josevalim 3y ago

Hello, creator of Elixir here (and co-creator of Nx). I like your breakdown of the requirements in three abilities. I think Elixir provides all three of them (with a caveat) and I will refer to two videos for those who want to learn more: * This video shows how you can use Livebook (open-source computational notebook environment) to explore pre-trained neural networks (from Hugging Face), learn how they work, and then deploy them as part of a Phoenix web application: https://news.livebook.dev/announcing-bumblebee-gpt2-stable-diffusion-and-more-in-elixir-3Op73O https://news.livebook.dev/announcing-bumblebee-gpt2-stable-d... * This video shows how to scale a neural network serving, first to run it concurrently, and then have it run distributed over multiple CPUs and GPUs: https://news.livebook.dev/distributed2-machine-learning-notebooks-with-elixir-and-livebook---launch-week-1---day-2-1aIlaw https://news.livebook.dev/distributed2-machine-learning-note... The caveat is about your second ability. The production scale models are, at the end of the day running native/cuda code, using the same backends as Python (Google XLA and LibTorch). What we do is that we compile your Elixir code into a graph which is then compiled to the CPU/GPU just-in time (in an approach similar to JAX). You can find more information here: https://github.com/elixir-nx/#why-elixir https://github.com/elixir-nx/#why-elixir

nologic01 3y ago
Personally I would love to see your vision succeed. Monocultures are intrinsically risky and limiting. Pretty soon we are all fish in a bowl, unable to see the water. What will it take? I am just an armchair philosopher in this respect. Like they say languages are only in part about communicating with the computer, they are about communicating with other people. So these three major tasks are really about winning over three different types of people and their peculiar brain wirings, motivations and aspirations. 1) and 2) are more introvert: mathematical types that want to express their inner abstract worlds in computer instructions in the least painful way. The famous data point is that pytorch became the tool of choice of academic ML publishing over the first mover tensorflow. Then you have the true computer nerds that love making the most of hardware and see in ML a major new stage of computing: HPC becoming mass market. 3) is more extrovert (keeping in mind the norms of the tech industry :-) developing and scaling delightful products and apps to be used by non-technical people. Erlang/elixir clearly offers something special in terms of 3) and that is important because in the end its all down to serving useful software to people.