4 ms·

But what was wrong with pip, venv and pyproject.toml in the first place? I just keep a system installation of python for my personal things and an environment f

by FattiMei 11mo ago

But what was wrong with pip, venv and pyproject.toml in the first place? I just keep a system installation of python for my personal things and an environment for every project I'm working on. I'd get suspicious if a developer is picky about python versions or library versions like what crazy programs are you writing?

wrs 11mo ago
The pytorch ecosystem, for one, is notorious for very specific version dependencies between libraries.
jvanderbot 11mo ago
What was wrong was that you needed to do that. How many commands are required to build up a locally consistent workspace? Modern package managers do that for you.
oblio 11mo ago
How do pip and venv integrate with pyproject.toml? At least pip doesn't even use it.
- zahlman 11mo ago
  As of half a year ago with pip 25.1, it can install from "dependency groups" listed in pyproject.toml: https://ichard26.github.io/blog/2025/04/whats-new-in-pip-25.1/ https://ichard26.github.io/blog/2025/04/whats-new-in-pip-25.... Pip also generates PEP 751 lockfiles, and installing from those is on the roadmap still (https://github.com/pypa/pip/issues/13334 https://github.com/pypa/pip/issues/13334). venv is lower-level tooling. Literally all it does is create a virtual environment — the same kind that uv creates and manages. There's nothing to "integrate".
johnfn 11mo ago
As mostly a Python outsider, in the infrequent times that I do use python package management, uv just works. When I use pip I’d get all sorts of obscure error messages that I’d have to go track down, probably because I got some obscure environment detail wrong. With uv I never run into that nonsense.
the8472 11mo ago
What's wrong? Having modify the shell environment, no lockfile, slow download/installation, lack of a standard dependency dir, ... > I'd get suspicious if a developer is picky about python versions or library versions Certain library versions only support certain python versions. And they also break API. So moving up/down the python versions also means moving library versions which means stuff no longer works.
- zahlman 11mo ago
  You don't have to modify the environment (this is provided as an option for convenience). The alternatives are to use higher-level management like uv does, or to specify the path to executables in the virtual environment directly. But uv works by creating virtual environments that are essentially the same as what you get with `python -m venv --without-pip` (although they reimplemented the venv creation logic). Pip can install from dependency groups in a pyproject.toml file, and can write PEP 751 lockfiles, and work is under way to allow it to install from those lockfiles as well. I don't know what you mean about a "standard dependency dir". When you make a venv yourself, you can call it what you want, and put it where you want. If you want to put it in a "standard" place, you can trivially make a shell alias to do so. (You can also trivially make a shell alias for "activate the venv at a hard-coded relative path", and use that from your project root.) Yes, pip installation is needlessly slow for a variety of reasons (that mostly do not have to do with being implemented in Python rather than Rust). Resolving dependencies is also slow (and Rust may be more relevant here; I haven't done detailed testing). But your download speed is still going to be primarily limited by your internet connection to PyPI.
  - the8472 11mo ago
    I'm confused by this reply. > The alternatives are to use higher-level management like uv does, The question was specifically what's wrong with pip, venv and pyproject toml, i.e. what issues uv is trying to address. Well of course the thing trying to address the problem addresses the problem.... > I don't know what you mean about a "standard dependency dir". like node's node_modules, or cargo's ~/.cargo/registry. You shouldn't have to manually create and manage that. installing/building should just create it. Which is what uv does and pip doesn't. > the same as what you get with `python -m venv --without-pip` The thing that should be automatic. And even if it is not it should at least be less arcane. An important command like that should have been streamlined long ago. One of the many improvements uv brings to the table. > and work is under way to allow it to install from those lockfiles as well. Yeah well, the lack up until now is one of those "what is wrong" things. > But your download speed is still going to be primarily limited by your internet connection to PyPI. Downloading lots of small packages dependencies serially leaves a lot of performance on the table due to latency and non-instantaneous response from congestion controllers. Downloading and installing concurrently reduces walltime further.
zahlman 11mo ago
Design-wise, nothing, IMO. But I don't fault people who prefer the uv workflow, either. Chacun a son gout. Implementation-wise, there's nothing wrong in my view with venv. Or rather, everything is compelled to use virtual environments, including uv, and venv is just a simple tool for doing so manually. Pip, on the other hand, is slow and bulky due to poor architecture, a problem made worse by the expectation (you can work around it, but it requires additional understanding and setup, and isn't a perfect solution) of re-installing it into each virtual environment. (The standard library venv defaults to such installation; you can disable this, but then you have to have a global pip set up, and you have to direct it to install into the necessary environment. One sneaky way to do this is to install Pipx, and then set up some script wrappers that use Pipx's vendored copy of pip. I describe my techniques for this in https://zahlman.github.io/posts/2025/01/07/python-packaging-2/ https://zahlman.github.io/posts/2025/01/07/python-packaging-....) Edit: by "design" above I meant the broad strokes of how you use pip, installing single packages with their transitive dependencies etc. There's a lot I would change about the CLI syntax, and other design issues like that.
pornel 11mo ago
https://xkcd.com/1987/ https://xkcd.com/1987/

wrs 11mo ago

The pytorch ecosystem, for one, is notorious for very specific version dependencies between libraries.

jvanderbot 11mo ago

What was wrong was that you needed to do that. How many commands are required to build up a locally consistent workspace? Modern package managers do that for you.

oblio 11mo ago

How do pip and venv integrate with pyproject.toml? At least pip doesn't even use it.

zahlman 11mo ago
As of half a year ago with pip 25.1, it can install from "dependency groups" listed in pyproject.toml: https://ichard26.github.io/blog/2025/04/whats-new-in-pip-25.1/ https://ichard26.github.io/blog/2025/04/whats-new-in-pip-25.... Pip also generates PEP 751 lockfiles, and installing from those is on the roadmap still (https://github.com/pypa/pip/issues/13334 https://github.com/pypa/pip/issues/13334). venv is lower-level tooling. Literally all it does is create a virtual environment — the same kind that uv creates and manages. There's nothing to "integrate".

johnfn 11mo ago

As mostly a Python outsider, in the infrequent times that I do use python package management, uv just works. When I use pip I’d get all sorts of obscure error messages that I’d have to go track down, probably because I got some obscure environment detail wrong. With uv I never run into that nonsense.

the8472 11mo ago

What's wrong? Having modify the shell environment, no lockfile, slow download/installation, lack of a standard dependency dir, ... > I'd get suspicious if a developer is picky about python versions or library versions Certain library versions only support certain python versions. And they also break API. So moving up/down the python versions also means moving library versions which means stuff no longer works.

zahlman 11mo ago
You don't have to modify the environment (this is provided as an option for convenience). The alternatives are to use higher-level management like uv does, or to specify the path to executables in the virtual environment directly. But uv works by creating virtual environments that are essentially the same as what you get with `python -m venv --without-pip` (although they reimplemented the venv creation logic). Pip can install from dependency groups in a pyproject.toml file, and can write PEP 751 lockfiles, and work is under way to allow it to install from those lockfiles as well. I don't know what you mean about a "standard dependency dir". When you make a venv yourself, you can call it what you want, and put it where you want. If you want to put it in a "standard" place, you can trivially make a shell alias to do so. (You can also trivially make a shell alias for "activate the venv at a hard-coded relative path", and use that from your project root.) Yes, pip installation is needlessly slow for a variety of reasons (that mostly do not have to do with being implemented in Python rather than Rust). Resolving dependencies is also slow (and Rust may be more relevant here; I haven't done detailed testing). But your download speed is still going to be primarily limited by your internet connection to PyPI.
- the8472 11mo ago
  I'm confused by this reply. > The alternatives are to use higher-level management like uv does, The question was specifically what's wrong with pip, venv and pyproject toml, i.e. what issues uv is trying to address. Well of course the thing trying to address the problem addresses the problem.... > I don't know what you mean about a "standard dependency dir". like node's node_modules, or cargo's ~/.cargo/registry. You shouldn't have to manually create and manage that. installing/building should just create it. Which is what uv does and pip doesn't. > the same as what you get with `python -m venv --without-pip` The thing that should be automatic. And even if it is not it should at least be less arcane. An important command like that should have been streamlined long ago. One of the many improvements uv brings to the table. > and work is under way to allow it to install from those lockfiles as well. Yeah well, the lack up until now is one of those "what is wrong" things. > But your download speed is still going to be primarily limited by your internet connection to PyPI. Downloading lots of small packages dependencies serially leaves a lot of performance on the table due to latency and non-instantaneous response from congestion controllers. Downloading and installing concurrently reduces walltime further.

zahlman 11mo ago

Design-wise, nothing, IMO. But I don't fault people who prefer the uv workflow, either. Chacun a son gout. Implementation-wise, there's nothing wrong in my view with venv. Or rather, everything is compelled to use virtual environments, including uv, and venv is just a simple tool for doing so manually. Pip, on the other hand, is slow and bulky due to poor architecture, a problem made worse by the expectation (you can work around it, but it requires additional understanding and setup, and isn't a perfect solution) of re-installing it into each virtual environment. (The standard library venv defaults to such installation; you can disable this, but then you have to have a global pip set up, and you have to direct it to install into the necessary environment. One sneaky way to do this is to install Pipx, and then set up some script wrappers that use Pipx's vendored copy of pip. I describe my techniques for this in https://zahlman.github.io/posts/2025/01/07/python-packaging-2/ https://zahlman.github.io/posts/2025/01/07/python-packaging-....) Edit: by "design" above I meant the broad strokes of how you use pip, installing single packages with their transitive dependencies etc. There's a lot I would change about the CLI syntax, and other design issues like that.

pornel 11mo ago

https://xkcd.com/1987/ https://xkcd.com/1987/