4 ms·

Lockfiles and version pinning are bandaid solutions to be applied in the absence of resources for supporting new versions of libraries, especially given that, u

Lockfiles and version pinning are bandaid solutions to be applied in the absence of resources for supporting new versions of libraries, especially given that, unless you maintain your own internal package repositories, you're then dependent on those ancient versions remaining available through your usual channels for the unforeseeable future. On my team, we stick to pulling the latest versions of dependencies, but we run our CI/CD pipeline on every commit and hold onto the logs from the last few successful builds. When it's clear that a dependency upgrade has caused a failure, we check to see if we can afford the work we'd need to do to accommodate the new version, and only if it's nontrivial do we resort to pinning the dependency (but it's usually trivial). Even then, the intention is that this is a temporary workaround until we have the time and people required to make the change. Using lockfiles and version pinning by default opens you up to all sorts of issues. Breaking changes accumulate, and you'll fall further and further behind, increasing the development effort required to get up to date. If you ever want to use some hot, new library that just dropped with a must-have feature for your project, you're highly likely to run into a conflict that will force you to make all those changes you've been procrastinating. Of course, you'll also find yourself stuck with whatever security vulnerabilities exist in your pinned version of the dependency, to be discovered and exploited at some future date. This is a complete nonstarter in my field of work.

chatmasta 3y ago
> Using lockfiles and version pinning by default opens you up to all sorts of other issues Agreed this is a tradeoff. That's why you need a rigorous auto-update process. Your approach works too, it's just a question of when you want to detect failures caused by an upgrade Regarding security, you're still vulnerable until you update. In your approach, you couple those updates with ongoing development. But what if nobody's pushed code for a month? You'd probably want automated upgrades, right? So in that case why not pin your dependencies too? > you're then dependent on those ancient versions remaining available through your usual channels for the unforeseeable future It's a safe-ish bet that versions will remain available on npm since it's not possible to unpublish a package from there after 72 hours of publishing it. But you don't need to rely on the registry if you cache your dependencies, and you can take more advanced measures to hold onto that cache. In our codebase we build Docker images from every commit, and for the JS images we store the Yarn cache in a shared Docker layer. This was a bit finicky to get working, but isn't strictly necessary anyway since the final build includes the node_modules too. So we have reproducible builds at the application level, but we also hold onto all versions of Docker images (tagged by commit, and only rebuilt when their source changes), so we have reproducible deployments and - crucially - the ability to rollback to an earlier version without relying on any external dependencies. This was not an easy system to build, but it's more of a general devops problem than one that's limited to any specific packaging system. Our monorepo includes JS, Python, C, Lua, and a bunch of other languages and packages. I will say that the code for making Python properly reproducible was a lot more complicated (we use Pants and some hackery for generating a locked requirements.txt).

chatmasta 3y ago

> Using lockfiles and version pinning by default opens you up to all sorts of other issues Agreed this is a tradeoff. That's why you need a rigorous auto-update process. Your approach works too, it's just a question of when you want to detect failures caused by an upgrade Regarding security, you're still vulnerable until you update. In your approach, you couple those updates with ongoing development. But what if nobody's pushed code for a month? You'd probably want automated upgrades, right? So in that case why not pin your dependencies too? > you're then dependent on those ancient versions remaining available through your usual channels for the unforeseeable future It's a safe-ish bet that versions will remain available on npm since it's not possible to unpublish a package from there after 72 hours of publishing it. But you don't need to rely on the registry if you cache your dependencies, and you can take more advanced measures to hold onto that cache. In our codebase we build Docker images from every commit, and for the JS images we store the Yarn cache in a shared Docker layer. This was a bit finicky to get working, but isn't strictly necessary anyway since the final build includes the node_modules too. So we have reproducible builds at the application level, but we also hold onto all versions of Docker images (tagged by commit, and only rebuilt when their source changes), so we have reproducible deployments and - crucially - the ability to rollback to an earlier version without relying on any external dependencies. This was not an easy system to build, but it's more of a general devops problem than one that's limited to any specific packaging system. Our monorepo includes JS, Python, C, Lua, and a bunch of other languages and packages. I will say that the code for making Python properly reproducible was a lot more complicated (we use Pants and some hackery for generating a locked requirements.txt).