3 ms·

The main advantage these tools (poetry, pipenv, uv, ...) offer is, that they let you create lock files, which make your venv reproducible. Without that, you are

The main advantage these tools (poetry, pipenv, uv, ...) offer is, that they let you create lock files, which make your venv reproducible. Without that, you are kind of living in the wild west, where tomorrow you project can break. These tools help with projects, that are supposed to do more than "runs on my machine".

gauge_field 2y ago
Just to give a concrete example, it helped me with my projects involving langchain and several langchain extensions. I have a clear and consistent source of version record for each dependency.
These335 2y ago
Maybe I'm not fully grasping lock files then, but why isn't it sufficient to just pin the versions in the requirements.txt? Obviously it doesn't handle the python version itself but I just use pyenv for that. So my stance is just pyenv + venv seems to solve those problems. But then I see people singing the praises of these newer tools and I wonder what I am not getting
- thunky 2y ago
  Look into concrete vs abstract dependencies: https://martin-thoma.com/python-requirements/ https://martin-thoma.com/python-requirements/
- zahlman 2y ago
  I was in the same position myself and had to learn the answers myself. It still doesn't really matter very much for me - I do more library than application development, and my applications would probably generally be fine with a wide range of dependency versions - if they even have dependencies. In short, different people have very different use cases, and you might simply not have the use cases that drive so many others to the tools in question. But it's still useful to understand the theory. >why isn't it sufficient to just pin the versions in the requirements.txt? Because your dependencies have dependencies. You can, in fact, pin those as well explicitly, and as long as what you pin is a valid solution, Pip will (to my understanding; I haven't done a thorough, explicit test) happily grab exactly what you asked for. And as long as the version number is enough information, that will work for your application. But some people also want to ensure they use specific exact builds of a package, and verify their hashes. Some of them might be using private indexes (or even mixing and matching with PyPI) and need to worry about supply chain attacks. And at any rate they don't want to account for all the transitive dependencies manually. In principle, a lock file is a record of such a solution, including all the transitive dependencies, plus file hashes, etc. There isn't perfect agreement about what needs to be in them, which is why discussion of a standard lock file format has been tried a few times and is still ongoing (the current effort is https://peps.python.org/pep-0751/ https://peps.python.org/pep-0751/ ; see the linked threads for just some of the related discussion of the concept - there is much more, in the abstract).
thunky 2y ago
> create lock files That's where pip-tools comes in.
- the_mitsuhiko 2y ago
  And you can use uv as a full replacement for that with less bugs, faster performance and just generally a more pleasant experience. pip-compile -> uv pip compile and pip-sync -> uv pip sync. (Though I think the high level interface is the better thing to use)
  - thunky 2y ago
    No disagreement here, just pointing out that it's also possible to do with existing tools.
    - zelphirkalt 2y ago
      "existing tools" -- What do you mean by that? pip-tools [1] seem to be just another PyPI package. What makes them more available than any of the other tools? The other tools "exist" as well. [1]: https://pypi.org/project/pip-tools/ https://pypi.org/project/pip-tools/
      - thunky 2y ago
        My point was that there is a way to have lock files and keep using pip (using pip-tools), without full switching to poetry or uv or anything else.
        the_mitsuhiko 2y ago
        And uv's first release was just a drop-in replacement for pip-tools, just faster and less buggy.
        thunky 2y ago
        I know. Maybe you should read the comment I replied to so you understand the context: https://news.ycombinator.com/item?id=42415924 https://news.ycombinator.com/item?id=42415924 You will notice that it is about the ability to create lock files and nothing else. And btw I've been using pip-tools for years and it's worked great for me.

gauge_field 2y ago

Just to give a concrete example, it helped me with my projects involving langchain and several langchain extensions. I have a clear and consistent source of version record for each dependency.

These335 2y ago

Maybe I'm not fully grasping lock files then, but why isn't it sufficient to just pin the versions in the requirements.txt? Obviously it doesn't handle the python version itself but I just use pyenv for that. So my stance is just pyenv + venv seems to solve those problems. But then I see people singing the praises of these newer tools and I wonder what I am not getting

thunky 2y ago
Look into concrete vs abstract dependencies: https://martin-thoma.com/python-requirements/ https://martin-thoma.com/python-requirements/
zahlman 2y ago
I was in the same position myself and had to learn the answers myself. It still doesn't really matter very much for me - I do more library than application development, and my applications would probably generally be fine with a wide range of dependency versions - if they even have dependencies. In short, different people have very different use cases, and you might simply not have the use cases that drive so many others to the tools in question. But it's still useful to understand the theory. >why isn't it sufficient to just pin the versions in the requirements.txt? Because your dependencies have dependencies. You can, in fact, pin those as well explicitly, and as long as what you pin is a valid solution, Pip will (to my understanding; I haven't done a thorough, explicit test) happily grab exactly what you asked for. And as long as the version number is enough information, that will work for your application. But some people also want to ensure they use specific exact builds of a package, and verify their hashes. Some of them might be using private indexes (or even mixing and matching with PyPI) and need to worry about supply chain attacks. And at any rate they don't want to account for all the transitive dependencies manually. In principle, a lock file is a record of such a solution, including all the transitive dependencies, plus file hashes, etc. There isn't perfect agreement about what needs to be in them, which is why discussion of a standard lock file format has been tried a few times and is still ongoing (the current effort is https://peps.python.org/pep-0751/ https://peps.python.org/pep-0751/ ; see the linked threads for just some of the related discussion of the concept - there is much more, in the abstract).

thunky 2y ago

> create lock files That's where pip-tools comes in.

the_mitsuhiko 2y ago
And you can use uv as a full replacement for that with less bugs, faster performance and just generally a more pleasant experience. pip-compile -> uv pip compile and pip-sync -> uv pip sync. (Though I think the high level interface is the better thing to use)
- thunky 2y ago
  No disagreement here, just pointing out that it's also possible to do with existing tools.
  - zelphirkalt 2y ago
    "existing tools" -- What do you mean by that? pip-tools [1] seem to be just another PyPI package. What makes them more available than any of the other tools? The other tools "exist" as well. [1]: https://pypi.org/project/pip-tools/ https://pypi.org/project/pip-tools/
    - thunky 2y ago
      My point was that there is a way to have lock files and keep using pip (using pip-tools), without full switching to poetry or uv or anything else.
      - the_mitsuhiko 2y ago
        And uv's first release was just a drop-in replacement for pip-tools, just faster and less buggy.
        thunky 2y ago
        I know. Maybe you should read the comment I replied to so you understand the context: https://news.ycombinator.com/item?id=42415924 https://news.ycombinator.com/item?id=42415924 You will notice that it is about the ability to create lock files and nothing else. And btw I've been using pip-tools for years and it's worked great for me.