6 ms·
The reason why Python dominates is that modern ML applications don't exist in a vacuum. They aren't the standalone C/FORTRAN/MATLAB scripts of yore that load in
by MontyCarloHall 1y ago
The reason why Python dominates is that modern ML applications don't exist in a vacuum. They aren't the standalone C/FORTRAN/MATLAB scripts of yore that load in some simple, homogeneous data, crunch some numbers, and spit out a single result. Rather, they are complex applications with functionality extending far beyond the number crunching, which requires a robust preexisting software ecosystem.
For example, a modern ML application might need an ETL pipeline to load and harmonize data of various types (text, images, video, etc., all in different formats) from various sources (local filesystem, cloud storage, HTTP, etc.) The actual computation then must leverage many different high-level functionalities, e.g. signal/image processing, optimization, statistics, etc. All of this computation might be too big for one machine, and so the application must dispatch jobs to a compute cluster or cloud. Finally, the end results might require sophisticated visualization and organization, with a GUI and database.
There is no single language with a rich enough ecosystem that can provide literally all of the aforementioned functionality besides Python. Python's numerical computing libraries (NumPy/PyTorch/JAX etc.) all call out to C/C++/FORTRAN under the hood and are thus extremely high-performance, and for functionality they don't implement, Python's C/C++ FFIs (e.g. Python.h, NumPy C integration, PyTorch/Boost C++ integration) are not perfect, but are good enough that implementing the performance-critical portions of code in C/C++ is much easier compared to re-implementing entire ecosystems of packages in another language like Julia.
- Hizonner 1y agoThis guy is worried about GPU kernels, which are never, ever written in Python. As you point out, Python is a glue language for ML. > There is no single language with a rich enough ecosystem that can provide literally all of the aforementioned functionality besides Python. That may be true, but some of us are still bitter that all that grew up around an at-least-averagely-annoying language rather than something nicer.
- MontyCarloHall 1y ago>This guy is worried about GPU kernels Then the title should be "why GPU kernel programming needs a new programming language." I can get behind that; I've written CUDA C and it was not fun (though this was over a decade ago and things may have since improved, not to mention that the code I wrote then could today be replaced by a couple lines of PyTorch). That said, GPU kernel programming is fairly niche: for the vast majority of ML applications, the high-level API functions in PyTorch/TensorFlow/JAX/etc. provide optimal GPU performance. It's pretty rare that one would need to implement custom kernels. >which are never, ever written in Python. Not true! Triton is a Python API for writing kernels, which are JIT compiled.
- catgary 1y agoI agree with you that writing kernels isn’t necessarily the most important thing for most ML devs. I think an MLIR-first workflow with robust support for the StableHLO and LinAlg dialects is the best path forward for ML/array programming, so on one hand I do applaud what Mojo is doing. But I’m much more interested in how MLIR opens the door to “JAX in <x>”. I think Julia is moving in that direction with Reactant.jl, and I think there’s a Rust project doing something similar (I think burn.dev may be using ONNX has an even higher-level IR). In my ideal world, I would be able to write an ML model and training loop in some highly verified language and call it from Python/Rust for training.
- pama 1y agoAlthough they are incredibly useful, the ML applications, or MLops, or ML Devops, or whatever other production-related application tech stack terminology may come to mind, are providing critical scaffolding infrastructure and glue but are not strictly what comes to mind when you only use the term “Machine Learning”. The key to machine learning is the massively parallel ability to efficiently train large neural network models, and the key to using the benefits of these trained models is the ability to rapidly evaluate them. Yes you need to get data for the training and you need complex infrastructure for the applications but the conferences, papers, studies on machine learning dont refer to these (other than in passing), in part because they are solvable, and in part because they are largely orthogonal to ML. Another way to think about it is that the nvidia price is what goes to infinity in this recent run and not the database or disk drive providers. So if someone finds a way to make the core ML part better with a programming language solution that is certainly very welcome and the title is appropriate. (The fact that GPU programming is considered niche in the current state of ML is a strong argument for keeping the title as is.)
- seanmcdirmid 1y agoThere are a lot of tricks for writing GPU code in a high level language and using some sort of meta programming to make it work out (I think Conal Elliott first did this in Haskell, where he also does symbolic differentiation in the same paper!).
- ModernMech 1y ago> This guy is worried about GPU kernels, which are never, ever written in Python. As you point out, Python is a glue language for ML. That's kind of the point of Mojo, they're trying to solve the so-called "two language problem" in this space. Why should you need two languages to write your glue code and kernel code? Why can't there be a language which is both as easy to write as Python, but can still express GPU kernels for ML applications? That's what Mojo is trying to be through clever use of LLVM MLIR.
- nostrademons 1y agoIt's interesting, people have been trying to solve the "two language problem" since before I started professionally programming 25 years ago, and in that time period two-language solutions have just gotten even more common. Back in the 90s they were usually spoken about only in reference to games and shell programming; now the pattern of "scripting language calls out to highly-optimized C or CUDA for compute-intensive tasks" is common for webapps, ML, cryptocurrency, drones, embedded, robotics, etc. I think this is because many, many problem domains have a structure that lends themselves well to two-language solutions. They have a small homogenous computation structure on lots of data that needs to run extremely fast. And they also have a lot of configuration and data-munging that is basically quick one-time setup but has to be specified somewhere, and the more concisely you can specify it, the less human time development takes. The requirements on a language designed to run extremely fast are going to be very different from one that is designed to be as flexible and easy to write as possible. You usually achieve quick execution by eschewing flexibility and picking a programming model that is fairly close to the machine model, but you achieve flexibility by having lots of convenience features built into the language, most of which will have some cost in memory or indirections. There've been a number of attempts at "one language to rule them all", notably PL/1, C++, Julia (in the mathematical programming subdomain), and Common Lisp, but it often feels like the "flexible" subset is shoehorned in to fit the need for zero-cost abstractions, and/or the "compute-optimized" subset is almost a whole separate language that is bolted on with similar but more verbose syntax.
- soVeryTired 1y ago
- jimbokun 1y ago> That may be true, but some of us are still bitter that all that grew up around an at-least-averagely-annoying language rather than something nicer. Don't worry. If you stick around this industry long enough you'll see this happen several more times.
- almostgotcaught 1y ago> which are never, ever written in Python nah never ever ever ever ever ... except https://github.com/FlagOpen/FlagGems https://github.com/FlagOpen/FlagGems https://github.com/linkedin/Liger-Kernel https://github.com/linkedin/Liger-Kernel https://github.com/meta-pytorch/applied-ai https://github.com/meta-pytorch/applied-ai https://github.com/gpu-mode/triton-index https://github.com/gpu-mode/triton-index https://github.com/AlibabaPAI/FLASHNN https://github.com/AlibabaPAI/FLASHNN https://github.com/unslothai/unsloth https://github.com/unslothai/unsloth the number of people commenting on this stuff that don't know what they're actually talking about grows by leaps and bounds every day...
- Hizonner 1y agoI stand corrected. I should have known people would be doing that in Python. How many of the world's total FLOPs are going through those?
- almostgotcaught 1y agoTriton is a backend for PyTorch. Lately it is the backend. So it's definitely double digits percentage if not over 50%.
- lairv 1y agoIt's the backend for torch.compile, pytorch eager mode will still use cuBLAS/cuDNN/custom CUDA kernels, not sure what's the usage of torch.compile
- almostgotcaught 1y ago> not sure what's the usage of torch.compile consider that at minimum both FB and OAI themselves definitely make heavy use of the Triton backend in PyTorch.
- cavisne 1y agoDoesn’t triton write its own intermediate language that then compiles to PTX?
- anothernewdude 1y ago> rather than something nicer. Python was the something nicer. A lot of the other options were so much worse.
- benzible 1y agoPython's ecosystem is hard to beat, but Elixir/Nx already does a lot of what Mojo promises. EXLA gives you GPU/TPU compilation through XLA with similar performance to Mojo's demos, Explorer handles dataframes via Polars, and now Pythonx lets you embed Python when you need those specialized libraries. The real difference is that Elixir was built for distributed systems from day one. OTP/BEAM gives the ability to handle millions of concurrent requests as well as coordinating across GPU nodes. If you're building actual ML services (not just optimizing kernels), having everything from Phoenix / LiveView to Nx in one stack built for extreme fault-tolerance might matter more than getting the last bit of performance out of your hardware.
- melodyogonna 1y agoWho uses this Exla in production?
- benzible 1y agoThese guys, for one: https://www.amplified.ai https://www.amplified.ai See: https://www.youtube.com/watch?v=5FlZHkc4Mq4 https://www.youtube.com/watch?v=5FlZHkc4Mq4
- devbug 1y agoI recently built out training and inference at a FinTech (for fraud and risk) using Elixir and tried this very approach… We’re now using Python for training and forking Ortex (ONNX) for inference. The ecosystem just isn’t there, especially for training. It’s a little better for inference but still has significant gaps. I will eventually have time to push contributions upstream but Python has so much momentum behind it. Livebooks are amazing though and a better experience than anything a Python offers sans libraries.
- goatlover 1y ago> There is no single language with a rich enough ecosystem that can provide literally all of the aforementioned functionality besides Python. Have a hard time believing C++ and Java don't have rich enough ecosystems. Not saying they make for good glue languages, but everything was being written in those languages before Python became this popular.
- j2kun 1y agoYeah the OP here listed a bunch of Python stuff that all ends up shelling out to C++. C++ is rich enough, period, but people find it unpleasant to work in (which I agree with). It's not about "richness," it's about giving a language ecosystem for people who don't really want to do the messy, low-level parts of software, and which can encapsulate the performance-critical parts with easy glue
- FuckButtons 1y agoI mean, you’ve basically described why people use Python, it’s a way to use C/C++ without having to write it.
- anakaine 1y agoAnd ill take that reason every single day. I could spend days or more working out particular issues in C++, or I could use a much nicer to use glue language with a great ecosystem and a huge community driving it and get the same task done in minutes to hours.
- lairv 1y agoI tried to statically link DuckDB to one of my C++ project earlier this year and it took me 3 days to have something working on Windows/Linux/MacOS (just to be able to use the dependency) While I'm not a C++ expert, doing the same in Python is just one pip install away, so yeah both "richness" and "ease of use" of the ecosystem matters
- j2kun 1y ago
- halayli 1y agoFortunately, Chris knows what he's doing. https://docs.modular.com/mojo/manual/python/ https://docs.modular.com/mojo/manual/python/
- benreesman 1y agoI'm in kind of a different place with it on the inference side. I've got these crazy tuned up CUDA kernels that are relatively straightforward to build in isolation and really where all the magic happens, and there's this new CUTLASS 3 stuff and modern C++ can call it all trivially. And then there's this increasingly thin film of torch crap that's just this side of unbuildable and drags in this reference counting and broken setup.py and it's a bunch of up and down projections to the real hacker shit. I'm thinking I'm about one squeeze of the toothpaste tube from just squuezing that junk out and having a nice, clean, well-groomed C++ program that can link anything and link into anything.
- pjmlp 1y agoCUTLASS 4 has first class support for Python.
- saagarjha 1y agoIn fact I doubt the C++ API will be getting much love moving forward
- pjmlp 1y agoAt GTC 2025, NVidia introduced two major changes in CUDA ecosystem. First class support for Python JIT/DSLs across the whole ecosystem. Change the way C++ is used and taught, more focused on standard C++ support and libraries, than low level CUDA extensions. So in a way, I think you're kind of right.
- benreesman 1y agoNah, their people are way involved in mdarray and ROCm is looking to have the "oh no its broken again" bit flipped off in the RDNA 4/5 cycle. NVIDIA wants Python and C++ people, they want a new thing to moat up on, and they know it has to be legitimately good to defy economic gravity on chips a lot of companies can design and fan now.
- fellowmartian 1y agoIronically Python is the worst language for everything you’ve described. Packaging is pain, wheels are pain, everything breaks all the time. It’s only great for those standalone scripts. Nobody in their right mind would design Python the way it turned out if the goal was to be the main ML language.
- nialv7 1y agoYou argument is circular. Python has all this ecosystem _because_ it have been the language of choice for ML for a decade. At this point it's difficult to beat, but doesn't explain why it was chosen all those years ago.
- chickenzzzzu 1y agoNot only is their argument circular but it is wrong. There is no need to use 50 million lines of Python, Pytorch, Numpy, Linux, Cmake, CUDA, and god knows how many other layers of madness to do inference. It is literally on the order of tens of thousands of lines of code, instead of tens of millions, to do Vulkan ML, especially if you strip out the parts of the kernel you don't need.
- 317070 1y agoI was there when it was chosen all those years ago. At the time (2007-2009), Matlab was the application of choice for what would become "deep" learning research. Though it had its warts, and licensing issues. It was easy for students to get started with and to use, also as a lot of them were not from computer science backgrounds, but often from statistics, engineering or neuroscience. When autograd came (this was even before gpu's), people needed something more powerful than matlab, yet familiar. Numpy already existed, and python+numpy+matplotlib give you an environment and a language very similar to matlab. The biggest hurdle was that python is zero-indexed. If things went slightly different, I reckon we might have ended up using Octave or lua. I reckon Octave was too restrictive and poorly documented for autograd. On the other hand, lua was too dissimilar to matlab. I think it was Theano, the first widely used python autograd, and then later PyTorch, that really sealed the deal for python.
- nickpeterson 1y agoYou were there 30 years ago, when the strength of men failed?
- breuleux 1y agoWe chose Python for Theano because Python was already the language of choice for our research lab. If it had been my choice, I would probably have picked Scheme (I was really into macros at that time) or Ruby (I think it's better designed than Python). But if we had done it in another language than Python, frankly, I'm not sure it would have taken off in the first place. Python already had quite a bit of inertia, likely thanks to numpy and matplotlib.
- pjmlp 1y agoI was there when Perl and Tcl were the main actors, that is why VTK used Tcl originally. Python dominates, because 25 years ago places like CERN started to adopt Python as their main scripting language, and eventually got used for more tasks than empowered shell scripts. It is like arguing why C dominates and nothing else can ever replace it.
- cdavid 1y agoI agree ability to use python to "script HPC" was key factor, but by itself would not have been enough. What really made it dominate is numpy/scipy/matplotlib becoming good enough to replace matlab 20 years ago, and enabled an explosion of tools on top of it: pandas, scikit learn, and the DL stuff ofc. This is what differentiates python from other "morally equivalent" scripting languages.