5 ms·

"We're all on the internet, and we're all constantly running npm install or equivalent." This is not at all accurate, at least in my work. I use version pinnin

"We're all on the internet, and we're all constantly running npm install or equivalent." This is not at all accurate, at least in my work. I use version pinning to be resilient to changes in the greater internet: builds don't use the internet, and instead use internally pinned and vendored sources for third party libraries. If you rely upon the internet to build and deploy your code, then someone else's outage (like github's) can become your outage, or make yours worse. If you work on a product which has high uptime requirements, this is unacceptable. Just as important, suppose any one of those dependencies breaks the social contract described in the posted article: they make a backwards-incompatible change, and your code no longer compiles or builds or (much worse!) subtly breaks. Now you're relying very strongly upon the good intentions and competency of all third party maintainers. Worst of all, suppose any of your dependencies gets compromised and now has a security vulnerability. What do you do? This is not a nitpick; I think it's central. Because I need network-free, reproducible builds of known-good code, I need to maintain a copy of third-party code somewhere. When I start doing that, I need to keep track of what version (which could be a commit, sure) I'm storing to know when I'm out of date. Then, when my cached copy is out of date, I need to know how safe it is to update to match the remote version: should I expect builds to break, or not? Semantic versioning is a way for package maintainers to signal their expectations. It's not something you can rely completely on - in these systems I'm describing, you still need automated tests to give you more confidence that upgrading is safe. But semantic versioning improves your confidence which helps avoid wasting work on an upgrade that will certainly fail, and helps identify problems in advance. It's also helpful to have minor and patch versions so applications can concisely describe which features of third-party dependencies are required to build and run your code. This makes it easier to integrate new applications - you can clearly see whether your cached copy of the third-party code is up-to-date enough.

state_less 9y ago
Agreed, plus version pinning isn't mutually exclusive to running latest, head, or whatever you call now. You can still resolve now to some version. For example, Debian stable resolves to version 9. Stable points to the dependency tree called version 9, but you can also just reference it directly if you want. You can go back in time and reference dependency tree 8 if 9 was a doozy. It's foolish to think every version is going to better than the previous, in my experience. Sometimes you have to make the present the past until the future gets its act together.
akkartik 9y ago
Author here. One thing I'm realizing I should have been clearer about is that I'm talking about problems of upgrading, not deploying. The fact that we pin versions in places like Gemfile.lock is totally fine. However, the fact that we pin versions in places like Gemfile is a smell. (You may not, and I do not[1]. OP was my attempt at finding middle ground with mainstream best practice.) This discussion gets at the distinction: https://stackoverflow.com/questions/4151495/should-gemfile-lock-be-included-in-gitignore https://stackoverflow.com/questions/4151495/should-gemfile-l... The use of left-pad as an example in OP was intended as an acknowledgement of the deployment issues you point out. But I was probably being so subtle that I veered into cuteness :) [1] See my example of "exemplary library use" at http://arclanguage.org/item?id=20221 http://arclanguage.org/item?id=20221
- azernik 9y ago
  In the second half, GP is talking about upgrading: > When I start doing that, I need to keep track of what version (which could be a commit, sure) I'm storing to know when I'm out of date. Then, when my cached copy is out of date, I need to know how safe it is to update to match the remote version: should I expect builds to break, or not? > Semantic versioning is a way for package maintainers to signal their expectations. It's not something you can rely completely on - in these systems I'm describing, you still need automated tests to give you more confidence that upgrading is safe. But semantic versioning improves your confidence which helps avoid wasting work on an upgrade that will certainly fail, and helps identify problems in advance. Putting these things in Gemfile.lock does not accurately reflect the workflow where all upgrades happen with humans in the loop, who need to be aware of and initiate any change that happens in your dependencies' code.
  - akkartik 9y ago
    I actually interpreted the second half to be agreeing with me, so I glossed over it. "I need to know how safe it is to update.." Exactly what OP is about. I agree that the best approach today is to maintain a local cache of a specific version, and to maintain unit tests to warn you of any breakage. In which case: what's the point of a package manager, again? As I briefly mentioned above, my current approach is to be extremely conservative in introducing dependencies, inline any dependencies I do introduce, and then treat them as my own code, polishing them alongside my own stuff, deleting code paths I don't need, submitting patches upstream when I discover issues from my hacking. I'm investigating better ways to write automated tests, so that when tests pass we can actually gain confidence that nothing has regressed: http://akkartik.name/about http://akkartik.name/about. If we could do this we wouldn't need compatibility at all! But that is blue-sky research; OP is my attempt at meeting the rest of the world half-way. Assume package managers are trying to do something useful. How can we build them to actually do what they advertise? > Putting these things in Gemfile.lock does not accurately reflect the workflow where all upgrades happen with humans in the loop, who need to be aware of and initiate any change that happens in your dependencies' code. I agree that upgrades should be pull not push, so I'm not sure what you're disagreeing with here. We need upgrades to be easy to perform so that we'll be more likely to perform them and so that we'll perform them more often, thus keeping our projects up-to-date on the latest vulnerabilities and bugs.

state_less 9y ago

Agreed, plus version pinning isn't mutually exclusive to running latest, head, or whatever you call now. You can still resolve now to some version. For example, Debian stable resolves to version 9. Stable points to the dependency tree called version 9, but you can also just reference it directly if you want. You can go back in time and reference dependency tree 8 if 9 was a doozy. It's foolish to think every version is going to better than the previous, in my experience. Sometimes you have to make the present the past until the future gets its act together.

akkartik 9y ago

Author here. One thing I'm realizing I should have been clearer about is that I'm talking about problems of upgrading, not deploying. The fact that we pin versions in places like Gemfile.lock is totally fine. However, the fact that we pin versions in places like Gemfile is a smell. (You may not, and I do not[1]. OP was my attempt at finding middle ground with mainstream best practice.) This discussion gets at the distinction: https://stackoverflow.com/questions/4151495/should-gemfile-lock-be-included-in-gitignore https://stackoverflow.com/questions/4151495/should-gemfile-l... The use of left-pad as an example in OP was intended as an acknowledgement of the deployment issues you point out. But I was probably being so subtle that I veered into cuteness :) [1] See my example of "exemplary library use" at http://arclanguage.org/item?id=20221 http://arclanguage.org/item?id=20221

azernik 9y ago
In the second half, GP is talking about upgrading: > When I start doing that, I need to keep track of what version (which could be a commit, sure) I'm storing to know when I'm out of date. Then, when my cached copy is out of date, I need to know how safe it is to update to match the remote version: should I expect builds to break, or not? > Semantic versioning is a way for package maintainers to signal their expectations. It's not something you can rely completely on - in these systems I'm describing, you still need automated tests to give you more confidence that upgrading is safe. But semantic versioning improves your confidence which helps avoid wasting work on an upgrade that will certainly fail, and helps identify problems in advance. Putting these things in Gemfile.lock does not accurately reflect the workflow where all upgrades happen with humans in the loop, who need to be aware of and initiate any change that happens in your dependencies' code.
- akkartik 9y ago
  I actually interpreted the second half to be agreeing with me, so I glossed over it. "I need to know how safe it is to update.." Exactly what OP is about. I agree that the best approach today is to maintain a local cache of a specific version, and to maintain unit tests to warn you of any breakage. In which case: what's the point of a package manager, again? As I briefly mentioned above, my current approach is to be extremely conservative in introducing dependencies, inline any dependencies I do introduce, and then treat them as my own code, polishing them alongside my own stuff, deleting code paths I don't need, submitting patches upstream when I discover issues from my hacking. I'm investigating better ways to write automated tests, so that when tests pass we can actually gain confidence that nothing has regressed: http://akkartik.name/about http://akkartik.name/about. If we could do this we wouldn't need compatibility at all! But that is blue-sky research; OP is my attempt at meeting the rest of the world half-way. Assume package managers are trying to do something useful. How can we build them to actually do what they advertise? > Putting these things in Gemfile.lock does not accurately reflect the workflow where all upgrades happen with humans in the loop, who need to be aware of and initiate any change that happens in your dependencies' code. I agree that upgrades should be pull not push, so I'm not sure what you're disagreeing with here. We need upgrades to be easy to perform so that we'll be more likely to perform them and so that we'll perform them more often, thus keeping our projects up-to-date on the latest vulnerabilities and bugs.