4 ms·
(I will likely base a blog post in my packaging series off this comment later.) What people seem to miss about Pip is that it's by design, not a package manage
by zahlman 2y ago
(I will likely base a blog post in my packaging series off this comment later.)
What people seem to miss about Pip is that it's by design, not a package manager. It's a package installer, only. Of course it doesn't handle the environment setup for you; it's not intended for that. And of course it doesn't keep track of what you've installed, or make lock files, or update your `pyproject.toml`, or...
What it does do is offer a hideously complex set of options for installing everything under the sun, from everywhere under the sun. (And that complexity has led to long-standing, seemingly unfixable issues, and there are a lot of design decisions made that I think are questionable at best.)
Ideas like "welllll I use poetry but pyenv works or you could use conda too" are incoherent. They're for different purposes and different users, with varying bits of overlap. The reason people are unsatisfied is because any given tool might be missing one of the specific things they want, unless it's really all-in-one like Uv seems like it intends to be eventually.
But once you have a truly all-in-one tool, you notice how little of it you're using, and how big it is, and how useless it feels to have to put the tool name at the start of every command, and all the specific little things you don't like about its implementation of whatever individual parts. Not to mention the feeling of "vendor lock-in". Never mind that I didn't pay money for it; I still don't want to feel stuck with, say, your build back-end just because I'm using your lock-file updater.
In short, I don't want a "package manager".
I want a solid foundation (better than Pip) that handles installing (not managing) applications and packages, into either a specified virtual environment or a new one created (not managed, except to make it easy to determine the location, so other tools can manage it) for the purpose. In other words, something that fully covers the needs of users (making it possible to run the code), while providing only the bare minimum on top of that for developers - so that other developer tools can cooperate with that. And then I want specialized tools for all the individual things developers need to do.
The specialized tools I want for my own work all exist, and the tools others want mostly exist too. Twine uploads stuff to PyPI; `build` is a fine build front-end; Setuptools would do everything I need on the back-end (despite my annoyances with it). I don't need a lockfile-driven workflow and don't readily think in those terms. I use pytest from the command line for testing and I don't want a "package manager" nor "workflow tool" to wrap that for me. If I needed any wrapping there I could do a few lines of shell script myself. If anything, the problem with these tools is doing too much, rather than too little.
The base I want doesn't exist yet, so I've started making it myself. Pipx is a big step in the right direction, but it has some arbitrary limitations (I discuss these and some workarounds in my recent blog post https://zahlman.github.io/posts/2025/01/07/python-packaging-2/ https://zahlman.github.io/posts/2025/01/07/python-packaging-... ) and it's built on Pip so it inherits those faults and is that much bigger. Uv is even bigger still for the compiled binary, and I would only be using the installation parts.
- botanical76 2y agoDo you feel that Npm, mix, cargo went the wrong way, doing too much? It seems like their respective communities _love_ the standard tooling and all that it does. Or is Python fundamentally different?
- zahlman 2y agoPython is fundamentally used in different ways, and in particular is much more often used in conjunction with code in other languages that has to be compiled separately and specially interfaced with. It also has a longer history, and a userbase that's interested in very different ways of using Python than the "development ecosystem" model where you explicitly create a project with the aim of adding it to the same package store where you got your dependencies from. Different users have wildly different needs, and tons of the things workflow tools like uv/Poetry/PDM/Hatch/Flit do are completely irrelevant to lots of them. Tons of users don't want to "manage dependencies"; they want to have a single environment containing every package that any of their projects ever uses (and a shoulder to cry on if a version conflict ever arises from that). Tons of users don't want to make a "project" and they especially don't want to set up their Python code as a separate, reusable thing, isolated from the data they're working on with it right now. Tons of users think they know better than some silly "Pip" tool about exactly where each of their precious .py files ought to go on your hard drive. Tons of developers want their program to look and feel like a stand-alone, independent application, that puts itself in C:\Program Files and doesn't expect users to know what Python is. People more imaginative than I could probably continue this train of thought for quite a bit. For many of the individual tasks, there are small Unix-philosophy tools that work great. Why is typing `poetry publish` better than typing `twine upload`? (And it would be just `twine`, except that there's a `register` command that PyPI doesn't need in the first place, and a `check` command that's only to make sure that other tools did their job properly - and PyPI will do server-side checks anyway.) Why is typing `poetry run ...` better than activating the venv with the script it provided itself, and then doing the `...` part normally? An all-in-one tool works when people agree on the scope of "all", and you can therefore offer just one of them and not get complaints.
- imtringued 2y agoVirtualenv should have never existed in the first place. So you claiming that UV or whatever tool is doing too much, sounds to me like you're arguing based on "traditionalist" or "conservative" reasons rather than doing any technical thinking here. Node.js's replacement for virtualenv is literally just a folder named "node_modules". Meanwhile python has an entire tool with strange ideosyncracies that you have to pay attention to otherwise pip does the wrong thing by default. It is as if python is pretending to be a special snowflake where installing libraries into a folder is this super hyper mega overcomplicated thing that necessitates a whole dedicated tool just to manage, when in reality in other programming languages nobody is really thinking about that the fact that the libraries end up in their build folders. It just works. So again you're pretending that this is such a big deal that it needs a whole other tool, when the problem in question is so trivial that another tool is adding mental overhead with regard to the microscopic problem at hand.