3 ms·
I've been experimenting with a new requirements.txt and requirements.txt.lock route. Pip allows your requirements file to omit versions. That means: you can ma
by nurettin 4y ago
I've been experimenting with a new requirements.txt and requirements.txt.lock route.
Pip allows your requirements file to omit versions. That means: you can manually add your root requirements into your requirements.txt. Then you can
pip install -r requirements.txt
Then you can do
pip freeze > rewuirements.txt.lock
commit that, and now everyone can use the prepared lock file. If you want to refresh your package versions, try
pip install -U -r requirements.txt
and it will update your dependencies. If anything breaks, you can uninstall it. If it all works out, you can lock again.
This way I can stay within pip tools and speed up package installs. One negative is having to manually fix dependency conflicts. But that is a price I'd be willing to pay for ease if development and deployment.
Edit: Also note that the article makes an excellent point concerning supply chain attacks which pipenv would have successfully mitigated in this particular case.
- Dunedan 4y agoCheck out pip-tools [1] which does exactly that, albeit in a slightly more polished way. [1]: https://github.com/jazzband/pip-tools https://github.com/jazzband/pip-tools
- toyg 4y agoPretty much all stuff that tools like pipenv do, is something that anyone proficient with pip can do on their own. Their "advantage" lies in standardizing, so people don't have to find out if you called the lockfile "requirements.txt.lock", "requirement.lock", or something else. Unfortunately they create another learning curve, another leaky abstraction layer, another point of failure, etc etc...
- nurettin 4y agoTo add to that, the suspicious steps are: manually adding root dependencies. What if you forget? Not locking. What if you forget? Locking invalid versions. What if you don't check all is compatible first? How do you handle dev requirements? requirements_dev? but then how do you lock? The good thing is: you got rid of dependency walking and slow locking. As with everything, there is a tradeoff that I was willing to take.
- deleted 4y ago[deleted]
- 323 4y ago> and it will update your dependencies. I'm pretty sure you need an extra step, since pip freeze will put "==" constraints in the lock file. I do the same thing, but I have this extra step to replace "==" with ">=" sed -i "s/==/>=/g" requirements.txt.lock
- ansgri 4y agoIsn't the purpose of the lock file to lock precise versions for max reproducibility?
- nurettin 4y agoThere is a misunderstanding. This is why I don't use the lock file to update dependencies in the first place. You have to use the requirements.txt for updates. The lock is for deploying, or initializing to last stable revision. requirements.txt also ensures that you only install what you need.
- 323 4y agoBut pip -U will only update first level dependencies (the ones directly listed in requirements.txt). It will not update secondary dependencies. This is why every 3 months or so I completely delete the venv and recreate it.
- nurettin 4y agoI see, yes unfortunately even if we lock the requirements, people just yank versions or update them in place. It all feels a little bit finicky pardon the fun.
- Nextgrid 4y agoJust FYI this doesn't actually pin hashes, unlike pipenv and pip-tools. If this functionality is important to you I suggest you use the latter.