3 ms·

Mmm I don't buy it. Not many projects use setup.py now anyway and pip is still super slow. > Plenty of tools are written in Rust without being notably fast. T

by IshKebab 11mo ago

Mmm I don't buy it. Not many projects use setup.py now anyway and pip is still super slow. > Plenty of tools are written in Rust without being notably fast. This also hasn't been my experience. Most tools written in Rust are notably fast.

zahlman 11mo ago
> Not many projects use setup.py now anyway and pip is still super slow. Yes, but that's still largely not because of being written in Python. The architecture is really just that bad. Any run of pip that touches the network will end up importing more than 500 modules and a lot of that code will simply not be used. For example, one of the major dependencies is Rich, which includes things like a 3600-entry mapping of string names to emoji; Rich in turn depends on Pygments which normally includes a bunch of rules for syntax highlighting in dozens of programming languages (but this year they've finished trimming those parts of the vendored Pygments). Another thing is that pip's cache is an HTTP cache. It literally doesn't know how to access its own package download cache without hitting the network, and it does that access through wrappers that rely on cachecontrol and Requests.
- IshKebab 11mo ago
  > Any run of pip that touches the network will end up importing more than 500 modules and a lot of that code will simply not be used. That's a property of Python though. The fact that it isn't compiled (and that importing is very slow). > a 3600-entry mapping of string names to emoji Which can easily be zero-cost in Rust. > It literally doesn't know how to access its own package download cache without hitting the network This is the only example you've given that actually fits with your thesis.
  - zahlman 11mo ago
    > That's a property of Python though. The fact that it isn't compiled (and that importing is very slow). Bytecode compilation is compilation. There are many things that could be used to improve import speed that I never even see discussed, let alone implemented. But most importantly, pip doesn't need to have all these modules imported. They already proved they could defer the Requests imports; but the actual network calls aren't that hard to do with the standard library anyway. (As nice as it would be to have Requests in the standard library, but I digress.) Most of the stuff it imports up-front from Rich will go entirely unused. > Which can easily be zero-cost in Rust. Which is irrelevant to the point. > This is the only example you've given that actually fits with your thesis. No. My thesis is that pip doesn't have to be the way it is in order to actually solve the problem of installing Python packages. Everything I mentioned is an example of a thing pip doesn't have to do in order to install packages, and slows it down unnecessarily.
    - IshKebab 11mo ago
      > Bytecode compilation is compilation. It was pretty clear I meant AoT compilation. > Which is irrelevant to the point. Sorry, the fact that typical Rust programming style is much faster than typical Python style is irrelevant to the fact that a Rust tool is faster than its Python competitor? Riiight. > My thesis is that pip doesn't have to be the way it is in order to actually solve the problem of installing Python packages. Ok but the actual debate was about whether `uv` is faster because it is written in Rust. The answer is yes because when you put a normal amount of effort into writing Python or Rust you run into all these problems in Python that you don't run into in Rust. Sure you can defer imports, but that's a pretty uncommon thing to do in Python. It's like, you wouldn't say "bicycles are just as fast as cars; as long as you attach a jet engine to the bicycle!". Yes, technically possible, but nobody does that.
scottlamb 11mo ago
Mine either. Choosing Rust by no means guarantees your tool will be fast—you can of course still screw it up with poor algorithms. But I think most people who choose Rust do so in part because they aspire for their tool to be "blazing fast". Memory safety is a big factor of course, but if you didn't care about performance, you might have gotten that via a GCed (and likely also interpreted or JITed or at least non-LLVM-backend) language.
- IshKebab 11mo ago
  Yeah sometimes you get surprisingly fast Python programs or surprisingly slow Rust programs, but if you put in a normal amount of effort then in the vast majority of cases Rust is going to be 10-200x faster. I actually rewrote a non-trivial Python program in Rust once because it was so slow (among other reasons), and got a 50x speedup. It was mostly just running regexes over logs too, which is the sort of thing Python people say is an ideal case (because it's mostly IO or implemented in C).

zahlman 11mo ago

> Not many projects use setup.py now anyway and pip is still super slow. Yes, but that's still largely not because of being written in Python. The architecture is really just that bad. Any run of pip that touches the network will end up importing more than 500 modules and a lot of that code will simply not be used. For example, one of the major dependencies is Rich, which includes things like a 3600-entry mapping of string names to emoji; Rich in turn depends on Pygments which normally includes a bunch of rules for syntax highlighting in dozens of programming languages (but this year they've finished trimming those parts of the vendored Pygments). Another thing is that pip's cache is an HTTP cache. It literally doesn't know how to access its own package download cache without hitting the network, and it does that access through wrappers that rely on cachecontrol and Requests.

IshKebab 11mo ago
> Any run of pip that touches the network will end up importing more than 500 modules and a lot of that code will simply not be used. That's a property of Python though. The fact that it isn't compiled (and that importing is very slow). > a 3600-entry mapping of string names to emoji Which can easily be zero-cost in Rust. > It literally doesn't know how to access its own package download cache without hitting the network This is the only example you've given that actually fits with your thesis.
- zahlman 11mo ago
  > That's a property of Python though. The fact that it isn't compiled (and that importing is very slow). Bytecode compilation is compilation. There are many things that could be used to improve import speed that I never even see discussed, let alone implemented. But most importantly, pip doesn't need to have all these modules imported. They already proved they could defer the Requests imports; but the actual network calls aren't that hard to do with the standard library anyway. (As nice as it would be to have Requests in the standard library, but I digress.) Most of the stuff it imports up-front from Rich will go entirely unused. > Which can easily be zero-cost in Rust. Which is irrelevant to the point. > This is the only example you've given that actually fits with your thesis. No. My thesis is that pip doesn't have to be the way it is in order to actually solve the problem of installing Python packages. Everything I mentioned is an example of a thing pip doesn't have to do in order to install packages, and slows it down unnecessarily.
  - IshKebab 11mo ago
    > Bytecode compilation is compilation. It was pretty clear I meant AoT compilation. > Which is irrelevant to the point. Sorry, the fact that typical Rust programming style is much faster than typical Python style is irrelevant to the fact that a Rust tool is faster than its Python competitor? Riiight. > My thesis is that pip doesn't have to be the way it is in order to actually solve the problem of installing Python packages. Ok but the actual debate was about whether `uv` is faster because it is written in Rust. The answer is yes because when you put a normal amount of effort into writing Python or Rust you run into all these problems in Python that you don't run into in Rust. Sure you can defer imports, but that's a pretty uncommon thing to do in Python. It's like, you wouldn't say "bicycles are just as fast as cars; as long as you attach a jet engine to the bicycle!". Yes, technically possible, but nobody does that.

scottlamb 11mo ago

Mine either. Choosing Rust by no means guarantees your tool will be fast—you can of course still screw it up with poor algorithms. But I think most people who choose Rust do so in part because they aspire for their tool to be "blazing fast". Memory safety is a big factor of course, but if you didn't care about performance, you might have gotten that via a GCed (and likely also interpreted or JITed or at least non-LLVM-backend) language.

IshKebab 11mo ago
Yeah sometimes you get surprisingly fast Python programs or surprisingly slow Rust programs, but if you put in a normal amount of effort then in the vast majority of cases Rust is going to be 10-200x faster. I actually rewrote a non-trivial Python program in Rust once because it was so slow (among other reasons), and got a 50x speedup. It was mostly just running regexes over logs too, which is the sort of thing Python people say is an ideal case (because it's mostly IO or implemented in C).