3 ms·

requirements.txt has a lot of deficiencies for package management (compare to the various features Yarn or Cargo offer). The biggest weakness is transitive depe

requirements.txt has a lot of deficiencies for package management (compare to the various features Yarn or Cargo offer). The biggest weakness is transitive dependency locking: even if you specify the exact version of the dependencies you want, Pip can still resolve to a different version of their dependencies, and even then there’s no verification that the content matches what is expected beyond the version number (other tools use a content checksum). This can cause a long tail of reproducibility and maintenance issues. It’s also a tiny subset of what the parent describes.

bpicolo 6y ago
It also can't separate dev vs test vs prod dependencies, for example. Maintaining separate files for those is not exactly a convenient solution
Sir_Substance 6y ago
When I first set up my requirements.txt's, I usually set up a venv, install the things that I need and then run "pip freeze" to get a list of all deps including transitive deps, and put them all in the requirements.txts. I do sometimes feel that people are making rube goldberg machines out of their package management in an attempt to avoid just writing down all their deps.
gruez 6y ago
>The biggest weakness is transitive dependency locking: even if you specify the exact version of the dependencies you want, Pip can still resolve to a different version of their dependencies Isn't that what pip freeze is for?
- lmm 6y ago
  The trouble there is that if you freeze you can't unfreeze. A project needs the equivalent of both gemfile and gemfile.lock (unless you're using a system like maven that got things right the first time), but there's no standard way of having both of them in Python.
HALtheWise 6y ago
I strongly recommend https://github.com/jazzband/pip-tools https://github.com/jazzband/pip-tools to solve this. It provides a simple script to take a requirements file and "compile" a full specification of all transitive dependencies. You check both files into the repo, point pip at the generated file, and manually modify the other one. It means you often don't need to pin requirements manually at all, and the versions will be explicitly updated whenever you choose to recompile your requirements.

bpicolo 6y ago

It also can't separate dev vs test vs prod dependencies, for example. Maintaining separate files for those is not exactly a convenient solution

Sir_Substance 6y ago

When I first set up my requirements.txt's, I usually set up a venv, install the things that I need and then run "pip freeze" to get a list of all deps including transitive deps, and put them all in the requirements.txts. I do sometimes feel that people are making rube goldberg machines out of their package management in an attempt to avoid just writing down all their deps.

gruez 6y ago

>The biggest weakness is transitive dependency locking: even if you specify the exact version of the dependencies you want, Pip can still resolve to a different version of their dependencies Isn't that what pip freeze is for?

lmm 6y ago
The trouble there is that if you freeze you can't unfreeze. A project needs the equivalent of both gemfile and gemfile.lock (unless you're using a system like maven that got things right the first time), but there's no standard way of having both of them in Python.

HALtheWise 6y ago

I strongly recommend https://github.com/jazzband/pip-tools https://github.com/jazzband/pip-tools to solve this. It provides a simple script to take a requirements file and "compile" a full specification of all transitive dependencies. You check both files into the repo, point pip at the generated file, and manually modify the other one. It means you often don't need to pin requirements manually at all, and the versions will be explicitly updated whenever you choose to recompile your requirements.