4 ms·
That probably works for smaller projects without many dependencies, but it’s just going to install the sub-dependency versions that satisfy whatever comes last
by postpawl 5y ago
That probably works for smaller projects without many dependencies, but it’s just going to install the sub-dependency versions that satisfy whatever comes last in the requirements file. The pip docs describe that situation here: https://pip.pypa.io/en/latest/topics/dependency-resolution/ https://pip.pypa.io/en/latest/topics/dependency-resolution/
The pip docs also suggest using pip-tools to create lock files. Pip-tools is only for creating lock files (it’s not trying to fix virtualenvs like poetry is), and it works great.
- kbumsik 5y agoInteresting. So use `pip-compile` instead of `pip freeze > requirements.txt`?
- postpawl 5y agoSort of - you need to make a requirements.in file that specifies only your top level dependencies, then you run pip-compile on that file to generate the requirements.txt.
- kbumsik 5y agoSo another requirements.in file ignoring PEP621? I'm confused... Maybe I will just try first. pip-tools seems to acknowledging PEP621 https://pip-tools.readthedocs.io/en/latest/changelog/?highlight=pyproject#id5 https://pip-tools.readthedocs.io/en/latest/changelog/?highli... https://github.com/jazzband/pip-tools/issues/1047 https://github.com/jazzband/pip-tools/issues/1047
- Groxx 5y agopip-tools predates PEP621 by ~5-8 years, depending on how you count. So yes, it "ignores" it.
- edgyquant 5y agoYeah our codebase has a requirements file that takes over an hour to install with the new dependency resolver (and over 10 minutes using the deprecated resolver.) This is on a 6c/12t ryzen with 32g of ram and a gigabit connection.
- klyrs 5y agoThis has been my experience, too, and with a moderately simple set of dependencies. I don't understand why this is considered acceptable enough to deprecate the old resolver.
- minusf 5y agoit seems like you are posting on the wrong platform, github is that --> way :} the pip team has introduced strict performance checks to ensure the new resolver was not significantly slower than the old. they solicited requirements files to weed out situations exactly like the one you describe. for example https://github.com/pypa/pip/issues/8664 https://github.com/pypa/pip/issues/8664 if you want it improved, please report it.
- Groxx 5y agoI believe pipenv (not pyenv!) is also a viable option for correct versioning, though I'm not actually sure whether it or pip-tools is more actively developed these days. Last I used pipenv though (~2 years ago), it was a nicer virtualenv + pip-tools combination, but had worse version resolution / less useful verbose output for no apparent reason (since iirc it shares tons of code with pip-tools). Putting that aside though: yes, 100% pip-tools or an equivalent (which pyenv is not). It's the only sane way to both freeze dependencies for reproducibility, and maintain upgradability. I've used pip-tools for years on many large, complex projects, and it has been a huge benefit every single time. And it routinely revealed significant library-incompatibility problems that teams had only luckily dodged due to not using feature X yet, because pip's resolver has been so braindead for forever.
- aschampion 5y agoPipenv quickly becomes unusably slow. As in half hour to change one dependency. Poetry is strictly better, and I say that believing poetry is not a great solution either. There aren't many good, general recommendations to give people regarding Python package management, but "not pipenv" is one of the few I'm confident in.
- Groxx 5y agoGood to know, thanks :)
- d0mine 5y agoHow long ago did you try pipenv? I don't use it for large projects (just a few dozens dependencies), it is not fast but it works (I expect it to be just a convenience wrapper around corresponding functionality provided by pip/pip-tools (there were substantial changes in pip's dependency resolution).