4 ms·

uv is great, but I think the real fix is just abandoning Python. The culture that language maintains is rather hostile to maintainable development, easier to j

uv is great, but I think the real fix is just abandoning Python. The culture that language maintains is rather hostile to maintainable development, easier to just switch to Rust and just write better code by default.

airza 1y ago
There's not really another game in town if you want to do fast ML development :/
- DiabloD3 1y ago
  Dunno, almost all of the people I know anywhere in the ML space are on the C and Rust end of the spectrum. Lack of types, lack of static analysis, lack of ... well, lack of everything Python doesn't provide and fights users on costs too much developer time. It is a net negative to continue pouring time and money into anything Python-based. The sole exclusion I've seen to my social circle is those working at companies that don't directly do ML, but provide drivers/hardware/supporting software to ML people in academia, and have to try to fix their cursed shit for them. Also, fwiw, there is no reason why Triton is Python. I dislike Triton for a lot of reasons, but its just a matmul kernel DSL, there is nothing inherent in it that has to be, or benefits from, being Python.... it takes DSL in, outputs shader text out, then has the vendor's API run it (ie, CUDA, ROCm, etc). It, too, would benefit from becoming Rust.
  - nkozyra 1y ago
    > Dunno, almost all of the people I know anywhere in the ML space are on the C and Rust end of the spectrum. I wish this were broadly true. But there's too much legacy Python sunk cost for most people though. Just so much inertia behind Python for people to abandon it and try to rebuild an extensive history of ML tooling. I think ML will fade away from Python eventually but right now it's still everywhere.
    - DiabloD3 1y ago
      A lot of what I see in ML is all focused around Triton, which is why I mentioned it. If someone wrote a Triton impl that is all Rust instead, that would do a _lot_ of the heavy lifting on switching... most of their hard code is in Triton DSL, not in Python, the Python is all boring code that calls Triton funcs. That changes the argument on cost for a lot of people, but sadly not all.
  - mountainriver 1y ago
    I love Rust and C, I write quite a bit of both. I am an ML engineer by trade. To say most ML people are using Rust and C couldn’t be further from the truth
    - Narishma 1y ago
      They said most people they knew, not most people.
      - mountainriver 1y ago
        It was obviously implied
  - airza 1y ago
    Okay. Humor me. I want to write a transformer-based classifier for a project. I am accustomed to the pytorch and tensorflow libraries. What is the equivalent using C?
    - adastra22 1y ago
      You do know that tensorflow was written in C++ and the Python API bolted on top?
      - wolvesechoes 1y ago
        It could be written in mix of Cobol and APL. No one cares. People saying "oh those Python libraries are just C/C++ libraries with Python API, every language can have them" have one problem - no other language has them (with such extensive documentation, tutorials etc.)
        adastra22 1y ago
        Tensorflow has extensive documentation of its C++ interface, as that is the primary interface for the library (the Python API is a wrapper on top).
        wolvesechoes 1y ago
        I hoped it was quite obvious that by "other languages" I meant "other than Python and C/C++ in which they are written". At least sibling actually mentioned Java.
        adastra22 1y ago
        Scroll up this thread and the other poster was asking if you can use pytorch and tensorflow from C. Both are C++ libraries, so accessing them from C/C++ is pretty trivial and has first-class support.
        1y ago
  - wolvesechoes 1y ago
    > It, too, would benefit from becoming Rust. Yet it was created for Python. Someone took that effort and did it. No one took that effort in Rust. End of the story of crab's superiority. Python community is constantly creating new, great, highly usable packages that become de facto industry standards, and maintain old ones for years, creating tutorials, trainings and docs. Commercial vendors ship Python APIs to their proprietary solutions. Whereas Rust community is going through forums and social media telling them that they should use Rust instead, or that they "cheated" because those libraries are really C/C++ libraries (and BTW those should be done in Rust as well, because safety).
- deleted 1y ago
  [deleted]
- pjmlp 1y ago
  PyTorch also supports C++ and Java, Tensorflow also does C++ and Java, Apple AI is exposing ML libraries via Swift, Microsoft is exposing their AI stuff via .NET and Java as well, then there is Julia and Mojo is coming along. It is happening.
  - famouswaffles 1y ago
    TensorFlow is a C++ library with a python wrapping, yet nobody (obviously exaggeration) actually uses tensorflow (or torch) in C++ for ML R&D. It's like people just don't get it. The ML ecosystem in python didn't just spring from the ether. People wanted to interface in python badly, that's why you have all these libraries with substantial code in another language yet development didn't just shift to that language. If python was fast enough, most would be fine to ditch the C++ backends and have everything in python, but the reverse isn't true. The C++ interface exists, and no-one is using it.
    - pjmlp 1y ago
      The existing C++ API is done according to that "beautiful" Google guidelines, thus it could be much better. However people are definitely using it, as Android doesn't do Python, neither does ChromeOS.
      - famouswaffles 1y ago
        >However people are definitely using it, as Android doesn't do Python, neither does ChromeOS. That's not really a reason to think people are using it for that when things like onnxruntime and executorch exist. In fact, they are very likely not using it for that, if only because the torch runtime is too heavy for distribution on the edge anyway (plus android can run python). Regardless, that's just inference of existing models (which yes I'm sure happens in other languages), not research and/or development of new models (what /u/airza was concerned about), which is probably 99% in python.
        pjmlp 1y ago
trklausss 1y ago
Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative. Also, tons of CAE platforms have Python bindings, so you are "forced" to work on Python. Sometimes the solution is not just "abandoning a language". If it fits your purpose, knock yourself out, for others that may be reading: uv is great for Python dependency management on development, I still have to test it for deployment :)
- aeve890 1y ago
  >Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative. I'd say Go is a better alternative if you want to replace python scripting. Less friction and much faster compilation times than Rust.
  - DiabloD3 1y ago
    I am not a huge fan of Go, but if all the world's "serious" Python became Go, the average code quality would skyrocket, so I think I can agree to this proposal.
  - physicsguy 1y ago
    Go performance is terrible for numeric stuff though, no SIMD support.
    - pclmulqdq 1y ago
      There are Go SIMD libraries now, and there's also easy use of C libraries via Cgo.
    - DiabloD3 1y ago
      (given the context of LLMs) Unless you're doing CPU-side inference for corner cases where GPU inference is worse, lack of SIMD isn't a huge issue. There are libraries to write SIMD in Go now, but I think the better fix is being able to autovectorize during the LLVM IR optimization stage, so its available with multiple languages. I think LLVM has it now, its just not super great yet.
Exuma 1y ago
i hate python, but the idea of replacing python with rust is absurd
pjmlp 1y ago
I know Python since version 1.6. It is great for learning on how to program (BASIC replacement), OS scripting tasks as Perl replacement, and embedded scripting in GUI applications. Additionally understand PYTHONPATH, and don't mess with anything else. All the other stuff that is supposed to fix Python issues, I never bothered with them. Thankfully, other languages are starting to also have bindings to the same C and C++ compute libraries.
WhereIsTheTruth 1y ago
abandoning Python for Rust in AI would cripple the field, not rescue it the disease is the cargo cult addiction (which Rust is full of) to micro libraries, not the language that carries 90% of all peer reviewed papers, datasets, and models published in the last decade every major breakthrough, from AlphaFold to Stable Diffusion, ships with a Python reference implementation because that is the language researchers can read, reproduce, and extend, remove Python and you erase the accumulated, executable knowledge of an entire discipline overnight, enforcing Rust would sabotage the field more than anything on the topic of uv, it will do more harm than good by enabling and empowering cargo cults on a systemic level the solution has always been education, teaching juniors to value simplicity, portability and maintainability
- stonemetal12 1y ago
  Nah, it would be like going from chemistry to chemical engineering. Doing chemical reactions in the lab by hand is great for learning but you aren't going to run a fleet of cars on hand made gas. Getting ML out of the lab and into production needs that same mental conversion from CS to SE.
wavemode 1y ago
Rust is not a viable replacement for Python except in a few domains.

airza 1y ago

There's not really another game in town if you want to do fast ML development :/

DiabloD3 1y ago
Dunno, almost all of the people I know anywhere in the ML space are on the C and Rust end of the spectrum. Lack of types, lack of static analysis, lack of ... well, lack of everything Python doesn't provide and fights users on costs too much developer time. It is a net negative to continue pouring time and money into anything Python-based. The sole exclusion I've seen to my social circle is those working at companies that don't directly do ML, but provide drivers/hardware/supporting software to ML people in academia, and have to try to fix their cursed shit for them. Also, fwiw, there is no reason why Triton is Python. I dislike Triton for a lot of reasons, but its just a matmul kernel DSL, there is nothing inherent in it that has to be, or benefits from, being Python.... it takes DSL in, outputs shader text out, then has the vendor's API run it (ie, CUDA, ROCm, etc). It, too, would benefit from becoming Rust.
- nkozyra 1y ago
  > Dunno, almost all of the people I know anywhere in the ML space are on the C and Rust end of the spectrum. I wish this were broadly true. But there's too much legacy Python sunk cost for most people though. Just so much inertia behind Python for people to abandon it and try to rebuild an extensive history of ML tooling. I think ML will fade away from Python eventually but right now it's still everywhere.
  - DiabloD3 1y ago
    A lot of what I see in ML is all focused around Triton, which is why I mentioned it. If someone wrote a Triton impl that is all Rust instead, that would do a _lot_ of the heavy lifting on switching... most of their hard code is in Triton DSL, not in Python, the Python is all boring code that calls Triton funcs. That changes the argument on cost for a lot of people, but sadly not all.
- mountainriver 1y ago
  I love Rust and C, I write quite a bit of both. I am an ML engineer by trade. To say most ML people are using Rust and C couldn’t be further from the truth
  - Narishma 1y ago
    They said most people they knew, not most people.
    - mountainriver 1y ago
      It was obviously implied
- airza 1y ago
  Okay. Humor me. I want to write a transformer-based classifier for a project. I am accustomed to the pytorch and tensorflow libraries. What is the equivalent using C?
  - adastra22 1y ago
    You do know that tensorflow was written in C++ and the Python API bolted on top?
    - wolvesechoes 1y ago
      It could be written in mix of Cobol and APL. No one cares. People saying "oh those Python libraries are just C/C++ libraries with Python API, every language can have them" have one problem - no other language has them (with such extensive documentation, tutorials etc.)
      - adastra22 1y ago
        Tensorflow has extensive documentation of its C++ interface, as that is the primary interface for the library (the Python API is a wrapper on top).
        wolvesechoes 1y ago
        I hoped it was quite obvious that by "other languages" I meant "other than Python and C/C++ in which they are written". At least sibling actually mentioned Java.
        adastra22 1y ago
        Scroll up this thread and the other poster was asking if you can use pytorch and tensorflow from C. Both are C++ libraries, so accessing them from C/C++ is pretty trivial and has first-class support.
        1y ago
- wolvesechoes 1y ago
  > It, too, would benefit from becoming Rust. Yet it was created for Python. Someone took that effort and did it. No one took that effort in Rust. End of the story of crab's superiority. Python community is constantly creating new, great, highly usable packages that become de facto industry standards, and maintain old ones for years, creating tutorials, trainings and docs. Commercial vendors ship Python APIs to their proprietary solutions. Whereas Rust community is going through forums and social media telling them that they should use Rust instead, or that they "cheated" because those libraries are really C/C++ libraries (and BTW those should be done in Rust as well, because safety).
deleted 1y ago
[deleted]
pjmlp 1y ago
PyTorch also supports C++ and Java, Tensorflow also does C++ and Java, Apple AI is exposing ML libraries via Swift, Microsoft is exposing their AI stuff via .NET and Java as well, then there is Julia and Mojo is coming along. It is happening.
- famouswaffles 1y ago
  TensorFlow is a C++ library with a python wrapping, yet nobody (obviously exaggeration) actually uses tensorflow (or torch) in C++ for ML R&D. It's like people just don't get it. The ML ecosystem in python didn't just spring from the ether. People wanted to interface in python badly, that's why you have all these libraries with substantial code in another language yet development didn't just shift to that language. If python was fast enough, most would be fine to ditch the C++ backends and have everything in python, but the reverse isn't true. The C++ interface exists, and no-one is using it.
  - pjmlp 1y ago
    The existing C++ API is done according to that "beautiful" Google guidelines, thus it could be much better. However people are definitely using it, as Android doesn't do Python, neither does ChromeOS.
    - famouswaffles 1y ago
      >However people are definitely using it, as Android doesn't do Python, neither does ChromeOS. That's not really a reason to think people are using it for that when things like onnxruntime and executorch exist. In fact, they are very likely not using it for that, if only because the torch runtime is too heavy for distribution on the edge anyway (plus android can run python). Regardless, that's just inference of existing models (which yes I'm sure happens in other languages), not research and/or development of new models (what /u/airza was concerned about), which is probably 99% in python.
      - pjmlp 1y ago

trklausss 1y ago

Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative. Also, tons of CAE platforms have Python bindings, so you are "forced" to work on Python. Sometimes the solution is not just "abandoning a language". If it fits your purpose, knock yourself out, for others that may be reading: uv is great for Python dependency management on development, I still have to test it for deployment :)

aeve890 1y ago
>Every tool for the right job. If you are doing tons of scripting (for e.g. tests on platforms different than Rust), Python can be a solid valid alternative. I'd say Go is a better alternative if you want to replace python scripting. Less friction and much faster compilation times than Rust.
- DiabloD3 1y ago
  I am not a huge fan of Go, but if all the world's "serious" Python became Go, the average code quality would skyrocket, so I think I can agree to this proposal.
- physicsguy 1y ago
  Go performance is terrible for numeric stuff though, no SIMD support.
  - pclmulqdq 1y ago
    There are Go SIMD libraries now, and there's also easy use of C libraries via Cgo.
  - DiabloD3 1y ago
    (given the context of LLMs) Unless you're doing CPU-side inference for corner cases where GPU inference is worse, lack of SIMD isn't a huge issue. There are libraries to write SIMD in Go now, but I think the better fix is being able to autovectorize during the LLVM IR optimization stage, so its available with multiple languages. I think LLVM has it now, its just not super great yet.

Exuma 1y ago

i hate python, but the idea of replacing python with rust is absurd

pjmlp 1y ago

I know Python since version 1.6. It is great for learning on how to program (BASIC replacement), OS scripting tasks as Perl replacement, and embedded scripting in GUI applications. Additionally understand PYTHONPATH, and don't mess with anything else. All the other stuff that is supposed to fix Python issues, I never bothered with them. Thankfully, other languages are starting to also have bindings to the same C and C++ compute libraries.

WhereIsTheTruth 1y ago

abandoning Python for Rust in AI would cripple the field, not rescue it the disease is the cargo cult addiction (which Rust is full of) to micro libraries, not the language that carries 90% of all peer reviewed papers, datasets, and models published in the last decade every major breakthrough, from AlphaFold to Stable Diffusion, ships with a Python reference implementation because that is the language researchers can read, reproduce, and extend, remove Python and you erase the accumulated, executable knowledge of an entire discipline overnight, enforcing Rust would sabotage the field more than anything on the topic of uv, it will do more harm than good by enabling and empowering cargo cults on a systemic level the solution has always been education, teaching juniors to value simplicity, portability and maintainability

stonemetal12 1y ago
Nah, it would be like going from chemistry to chemical engineering. Doing chemical reactions in the lab by hand is great for learning but you aren't going to run a fleet of cars on hand made gas. Getting ML out of the lab and into production needs that same mental conversion from CS to SE.

wavemode 1y ago

Rust is not a viable replacement for Python except in a few domains.