5 ms·

The main problems with just "pip install -r requirements.txt" are that you might get a different result the next time you run it if a (transitive) dependency ha

by jks 4y ago

The main problems with just "pip install -r requirements.txt" are that you might get a different result the next time you run it if a (transitive) dependency has released a new version, and that upgrading dependencies is an error-prone manual task. Both pipenv and poetry try to solve these problems.

julienpalard 4y ago
Upgrading dependencies is also a way for me to resolve issues, so I don't fight it.
bvrmn 4y ago
pip freeze > stable-reqs.txt ??
- throwaway234232 4y ago
  https://github.com/pypa/pip/issues/4732 https://github.com/pypa/pip/issues/4732
  - camgunz 4y ago
    This thread is just about pip freeze not including hashes, and while it's true that would provide a lot of reassurance, you can still pin transitive version dependencies w/o them.
- potatochup 4y ago
  This kinda works, but it isn't a cross-platform solution.
traverseda 4y ago
That's a feature, you should be using the latest minor/patch release when developing. If you really need to lock a dependency to a specific version it is possible, but you shouldn't be doing it. If a dependency is constantly breaking on minor/patch releases you shouldn't be using it. I think it's a lot of javascript developers who end up having this problem, and all I can say is that the python package ecosystem is not the same. Please don't lock your dependencies to specific minor/patch versions, and please don't use so many dependencies that it becomes tedious to deal with changes. Especially when you're writing a library. If you're locking to anything other than a major release or a minimum minor/patch revision when producing a library things have gone very wrong.
- Hackbraten 4y ago
  > That's a feature, you should be using the latest minor/patch release when developing. You’re not wrong, but developers still need precise control about which version is pinned so they have a chance to figure out whether a bug is in their code or a regression in a dependency. > If a dependency is constantly breaking on minor/patch releases you shouldn't be using it. A more common example would be a dependency of decent quality, but there will still be bugs because there’s no such thing as bug-free software. And when the inevitable issue comes up, it’s super useful to have a switch that lets you pin down the regression and helps you figure out whether it’s on you or on the upstream project to fix the issue. > Please don't lock your dependencies to specific minor/patch versions Even if you allow a version range of "*" in your dependency file, it may still be a good practice for your app to have the resolved version number pinned in a dependency lock file, and even commit that lock file to version control. > Especially when you're writing a library. If you're locking to anything other than a major release or a minimum minor/patch revision when producing a library things have gone very wrong. You’re referring to the dependency file, not the dependency lock file, right? I couldn’t agree more. At the same time, it may still make sense for some libraries to have a lock file during the development process, and even commit that lock file to version control. You just can’t include that lock file in your release.
- zarzavat 4y ago
  I don't agree. Repeatability is more important. Every developer working on a project should have the same version of each dependency. The testing server should have the same version. Every deployment should have the same version. I don't want my tools "helpfully" upgrading me to a different version, I don't care if it's "minor". It's different and that's a potential source of heisenbugs. Any change to dependencies should be an explicit action and it should be a commit in SCM. By all means show an ugly warning message to nag people to upgrade but computers live to serve us, not the other way around. The moment you start devolving these decisions to a computer you've lost control of your core competency, which is ultimately what code is running on the machine.
- mirashii 4y ago
  I really couldn’t disagree with this more. Reproducibility is far more important than integrating unknown bug fixes, and upgrading dependencies should be a manual process during which you review the changes and verify that they don’t break your system. My time and the time of my peers is too valuable to be debugging minor incompatibilities between a library 5 layers down the transitive dependency stack because versions aren’t pinned. It’s too important that if I go back to a 3 year old checkout of the code that it still runs and isn’t a wild goose chase of running down bugs and library incompatibilities. You can always upgrade overzealously when dependencies are pinned, falling back to the workflow you’ve described, but the inverse isn’t true. If your tools don’t allow you to pin, you’re signing yourself up for churn at unknown and unpredictable times.
ptx 4y ago
This is solved in pip nowadays with constraint files [1], so you can put your top-level dependencies in requirements.txt and all the transient ones in constraints.txt: pip install -r requirements.txt -c constraints.txt [1] https://pip.pypa.io/en/stable/user_guide/#constraints-files https://pip.pypa.io/en/stable/user_guide/#constraints-files

by jks 4y ago

julienpalard 4y ago

Upgrading dependencies is also a way for me to resolve issues, so I don't fight it.

bvrmn 4y ago

pip freeze > stable-reqs.txt ??

throwaway234232 4y ago
https://github.com/pypa/pip/issues/4732 https://github.com/pypa/pip/issues/4732
- camgunz 4y ago
  This thread is just about pip freeze not including hashes, and while it's true that would provide a lot of reassurance, you can still pin transitive version dependencies w/o them.
potatochup 4y ago
This kinda works, but it isn't a cross-platform solution.

traverseda 4y ago

That's a feature, you should be using the latest minor/patch release when developing. If you really need to lock a dependency to a specific version it is possible, but you shouldn't be doing it. If a dependency is constantly breaking on minor/patch releases you shouldn't be using it. I think it's a lot of javascript developers who end up having this problem, and all I can say is that the python package ecosystem is not the same. Please don't lock your dependencies to specific minor/patch versions, and please don't use so many dependencies that it becomes tedious to deal with changes. Especially when you're writing a library. If you're locking to anything other than a major release or a minimum minor/patch revision when producing a library things have gone very wrong.

Hackbraten 4y ago
> That's a feature, you should be using the latest minor/patch release when developing. You’re not wrong, but developers still need precise control about which version is pinned so they have a chance to figure out whether a bug is in their code or a regression in a dependency. > If a dependency is constantly breaking on minor/patch releases you shouldn't be using it. A more common example would be a dependency of decent quality, but there will still be bugs because there’s no such thing as bug-free software. And when the inevitable issue comes up, it’s super useful to have a switch that lets you pin down the regression and helps you figure out whether it’s on you or on the upstream project to fix the issue. > Please don't lock your dependencies to specific minor/patch versions Even if you allow a version range of "*" in your dependency file, it may still be a good practice for your app to have the resolved version number pinned in a dependency lock file, and even commit that lock file to version control. > Especially when you're writing a library. If you're locking to anything other than a major release or a minimum minor/patch revision when producing a library things have gone very wrong. You’re referring to the dependency file, not the dependency lock file, right? I couldn’t agree more. At the same time, it may still make sense for some libraries to have a lock file during the development process, and even commit that lock file to version control. You just can’t include that lock file in your release.
zarzavat 4y ago
I don't agree. Repeatability is more important. Every developer working on a project should have the same version of each dependency. The testing server should have the same version. Every deployment should have the same version. I don't want my tools "helpfully" upgrading me to a different version, I don't care if it's "minor". It's different and that's a potential source of heisenbugs. Any change to dependencies should be an explicit action and it should be a commit in SCM. By all means show an ugly warning message to nag people to upgrade but computers live to serve us, not the other way around. The moment you start devolving these decisions to a computer you've lost control of your core competency, which is ultimately what code is running on the machine.
mirashii 4y ago
I really couldn’t disagree with this more. Reproducibility is far more important than integrating unknown bug fixes, and upgrading dependencies should be a manual process during which you review the changes and verify that they don’t break your system. My time and the time of my peers is too valuable to be debugging minor incompatibilities between a library 5 layers down the transitive dependency stack because versions aren’t pinned. It’s too important that if I go back to a 3 year old checkout of the code that it still runs and isn’t a wild goose chase of running down bugs and library incompatibilities. You can always upgrade overzealously when dependencies are pinned, falling back to the workflow you’ve described, but the inverse isn’t true. If your tools don’t allow you to pin, you’re signing yourself up for churn at unknown and unpredictable times.

ptx 4y ago

This is solved in pip nowadays with constraint files [1], so you can put your top-level dependencies in requirements.txt and all the transient ones in constraints.txt: pip install -r requirements.txt -c constraints.txt [1] https://pip.pypa.io/en/stable/user_guide/#constraints-files https://pip.pypa.io/en/stable/user_guide/#constraints-files