4 ms·
# Ensure we always have an up to date lock file. if ! test -f uv.lock || ! uv lock --check 2>/dev/null; then uv lock fi Doesn't this defeat the purpose
by gchamonlive 1y ago
# Ensure we always have an up to date lock file.
if ! test -f uv.lock || ! uv lock --check 2>/dev/null; then
uv lock
fi
Doesn't this defeat the purpose of having a lock file? If it doesn't exist or if it's invalid something catastrophic happened to the lock file and it should be handled by someone familiar with the project. Otherwise, why have a lock file at all? The CI will silently replace the lock file and cause potential confusion.
- freetonik 1y agoIn the Python world, I often see lockfiles treated a one "weird step in the installation process", and not committed to version control.
- burnt-resistor 1y agoIn the almost every world, Ruby and elsewhere too, constraints in library package metadata are supposed to express the full supported possibilities of allowed constraints while lock files represent current specific state. That's why they're not committed in that case to allow greater flexibility/interoperability for downstream users. For applications, it's recommended (but still optional) to commit lock files so that very specific and consistent dependencies are maintained to prevent arbitrary, unsupervised package upgrades leading to breakage.
- MrJohz 1y agoI know Cargo recommended your approach for a while, but ended up recommending that all projects always check in a lock file. This is also the norm in most other ecosystems I've used including Javascript and other Python package managers. When you're developing a library, you still want consistent, reproducible dependency installs. You don't want, for example, a random upgrade to a testing library to break your CI pipelines or cause delays while releasing. So you check in the lock file for the people working on the library. But when someone installs the library via a package manager, that package manager will ignore the lock file and just use the constraints in the package metadata. This avoids any interoperability issues for downstream users. I've heard of setups where there are even multiple lock files checked in so different combinations of dependency can be tested in CI, but I've not seen that in practice, and I imagine it's very much dependent on how the ecosystem as a whole operates.
- oceansky 1y agoIt's what I used to do with package-lock.json when I had little production experience.
- slau 1y agoIn my experience, this is fundamentally untrue. pip-tools has extensive support for recording the explicit version numbers, package hashes and whatnot directly in the requirements.txt based on requirements.in and constraints files. There are many projects that use pip-compile to lock things down. You couldn’t use python in a regulated environment if you didn’t. I’ve written many Makefiles that explicitly forbid CI from ever creating or updating the actual requirements.txt. It has to be reviewed by a human, or more.
- Hasnep 1y agoThey're not saying that's how it's supposed to be used, they're saying that's how it's often used by people who are unfamiliar with lock files
- MrJohz 1y agoThere are lots of tools that allow you to generate what are essentially lock files. But I think what the previous poster is saying is that most people either don't use these tools or don't use them correctly. That certainly matches my experience, where I've seen some quite complicated projects get put into production without any sort of dependency locking whatsoever - and where I've also seen the consequences of that where random dependencies have upgraded and broken everything and it's been almost impossible to figure out why. To me, one of the big advantages of UV (and similar tools) is that they make locked dependencies the default, rather than something you need to learn about and opt into. These sorts of better defaults are sorely needed in the Python ecosystem.
- bckr 1y agoThis is kinda how I treat it. I figured that I have already set the requirements in the pyproject.toml file. Should I be committing the lock file?
- gcarvalho 1y agoIf your pyproject.toml does not list all your dependencies (including dependencies of your dependencies) and a fixed version for each, you may get different versions of the dependencies in future installs. A lock file ensures all installations resolve the same versions, and the environment doesn’t differ simply because installations were made on different dates. Which is usually what you want for an application running in production.
- robertlagrant 1y agoWould strongly recommend a lockfile if these things sound like a good idea: - (fairly) reproducable builds in that you don't want dependencies blind-updating without knowing about it - removing "works on my machine" issues caused by different dependency versions - being able to cache dependency download folders in CI and use the lockfile as the cache key
- 9dev 1y agoWhat are the possible remediation steps, however? If there is no lock file at all, this is likely the first run, or it will be overwritten from a git upstream later on anyway; if it's broken, chances are high someone messed up a package installation and creating a fresh lock file seems like the only sensible thing to do. I also feel like this handles rare edge cases, but it seems like a pretty straightforward way to do so.
- globular-toast 1y agoThe fix is to generate the lockfile and commit it to the repository. Every build should be based on the untouched lockfile from the repo. It's the entire point of it.
- stavros 1y agoIf there's no lock file at all, you haven't locked your dependencies, and you should just install whatever is current (don't create a lockfile). If it's broken, you have problems, and you need to abort the deploy. There is never a reason for an automated system to create a lockfile.
- ealexhudson 1y agoThe reason is simple: it allows you to do the install using "sync" in all cases, whether the lockfile exists or not. Where the lockfile doesn't exist, it creates it from whatever current is, and the lockfile then gets thrown away later. So it's equivalent to what you're saying, it just avoids having two completely separate install paths. I think it's the correct approach.
- stavros 1y agoI don't understand, you can already run `uv sync` if the lockfile doesn't exist. It just creates a new one. Why do it explicitly, like here?
- JimDabell 1y agoIf the lock file is missing the only sensible thing to do is require human intervention. Either it’s the unusual case of somebody initialising a project but never syncing it, or something has gone seriously wrong – with potential security implications. The upside to automating this is negligible and the downside is large.
- silvester23 1y agoThis is actually covered by the --locked option that uv sync provides. If you do `uv sync --locked` it will not succeed if the lock file does not exist or is out of date. Edit: I slightly misread your comment. I strongly agree that having no lock file or a lockfile that does not match your specified dependencies is a case where a human should intervene. That's why I suggest you should always use the --locked option in your build.
- remram 1y agoYes this is a major bug in the process. I came to the comments to say this as well. They say this but do the exact opposite as you point out: > The --frozen flag ensures the lock file doesn’t get updated. That’s exactly what we want because we expect the lock file to have a complete list of exact versions we want to use for all dependencies that get installed.
- nickjj 1y agoHi author here. If you end up with an invalid lock file, it doesn't silently fail and move on with a generated lock file. The `uv lock` command fails with a helpful message and then errexit from the shell script kicks in. The reason I redirected the uv lock --check command's errors to /dev/null is because `uv lock` throws the same error and I wanted to avoid outputting it twice. For example, I made my lock file invalid by manually switching one of the dependencies to a version that doesn't match the expected SHA. Then I ran the same script you partially quoted and it yields this error which blocks the build and gives a meaningful message that a human can react to: 1.712 Using CPython 3.13.3 interpreter at: /usr/local/bin/python3 1.716 error: Failed to parse `uv.lock` 1.716 Caused by: The entry for package `amqp` v5.3.4 has wheel `amqp-5.3.1-py3-none-any.whl` with inconsistent version: v5.3.1 ------ failed to solve: process "/bin/sh -c chmod 0755 bin/* && bin/uv-install" did not complete successfully: exit code: 2 This error is produced from `uv lock` when the if condition evaluates to true. With that said, this logic would be much clearer which I just commit and pushed: if test -f uv.lock; then uv lock --check else uv lock fi As for a missing lock file, yep it will generate one but we want that. The expectation there is we have nothing to base things off of, so let's generate a fresh one and use it moving forward. The human expectation in a majority of the cases is to generate one in this spot and then you can commit it so moving forward one exists.
- deleted 1y ago[deleted]
- remram 1y agoI see you just changed your article from what it was when we commented: if ! test -f uv.lock || ! uv lock --check 2>/dev/null; then uv lock; fi Your new version no longer has the bug we are talking about. I don't know why you are trying to pretend it was never there though?
- nickjj 1y ago> Your new version no longer has the bug we are talking about. I don't know why you are trying to pretend it was never there though? I'm not sure I understand what you mean? 1. I posted the article last week on my site 2. I noticed it was on HN today (yay) 3. I looked at the parent's comment 4. The parent's description isn't what happens with the original code 5. I made the comment you're replying to on HN to address their concerns and included a refactored version of the original condition for clarity then said I pushed the updates 6. I pushed the updates to both git and my site so both match up There's nothing to pretend about and there's no bug because both versions of the code do the same thing, the 2nd version is just easier to read and requires less `uv` knowledge to know what happens when `uv lock` runs with an invalid lock file. The history is in the HN comment I wrote and git history. It doesn't make sense to leave the original code in the blog post and then write a wall of text to explain how it worked fine but here's a modified version for clarity. Both versions of the code have the same outcome which is ensuring there's a valid lock file before syncing. What would you have done differently? I saw feedback, saw room for improvement, left an audit trail in the comments and moved on. Here's the commits https://github.com/nickjj/docker-flask-example/commit/d1b7b92868039d2b429f636c015b08a0bceaf6c0 https://github.com/nickjj/docker-flask-example/commit/d1b7b9... and https://github.com/nickjj/docker-django-example/commit/a12e2fbbc8570616bc66e4271348709d0a3a0dc1 https://github.com/nickjj/docker-django-example/commit/a12e2... btw.