4 ms·

This: "Imagine a world where software could be tested against a version of the dependency, and then allow all version updates as long as they are minor (a new

This: "Imagine a world where software could be tested against a version of the dependency, and then allow all version updates as long as they are minor (a new functionality added in a backward-compatible manner)" is a fantasy in my experience. The Semver contract exists as a concept sure, but there's no guarantee that the publisher of a dependency you consume actually maintains compatibility, in the context of your applications usage, between minor versions. It's more of an aspiration.

jaapz 3y ago
As someone who has been bitten time and time again by transitive dependencies not adhering to semver while still claiming to be using semver, i totally agree. Dependency pinning is a must when you want to have full control, and even then some package managers allow pulling or updating specific versions of dependencies (looking at you, pip).
- User23 3y ago
  One can conceive of a solution that bisects dependencies independently to find the latest version that satisfies a comprehensive test suite. But that build and test is going to have to be reasonably quick. Even moreso if you take the exhaustive approach and transitively apply the algorithm to dependency dependencies which would also require backtracking. A more durable approach might be to require every API to formally and verifiably provide precondition and postcondition that can be checked with an SMT solver for every public interface. With current tech that is pushing a lot onto module authors, but I believe there is some research being done on inferring both preconditions and postconditions from the semantics of the component statements so perhaps we can avoid forcing programmers to carry the intolerable burden of learning the first order predicate calculus.
hinkley 3y ago
I haven’t figured out the math yet, but on my shelf if future projects, I have a part of a plan to tie semver to unit tests. The theory so far is: there’s a subset of unit tests that specify the API. If you create new tests, that’s mostly fine. If have to alter a test, that’s probably a breaking change. The main problem I can already see with this, which is why it’s on the shelf, is that some test frameworks and many tester writers cannot guarantee the invariants as stated above. You end up changing tests for minor behavioral changes. You mix tests of private fields in with tests of public ones. And even if you split them, then functional tests end up defining parts of the API. But somewhere out there in the future is a language where types are defined by tests, and certain testing patterns indicate minor version increments and others major.
- jrochkind1 3y ago
  I've thought about this too. The other issue is that tests can have bugs too, sometimes you are just fixing a test bug.
  - hinkley 3y ago
    Definitely. So in a robust system, you might also have to look at code coverage to see if the test interacts with any code changes since the last minor version increment. If I change the test but not the code, that may be a clarification. If I change both, it’s ambiguous.
davidmurdoch 3y ago
This has bitten me twice before: a transitive dep update broke my library. I now publish my npm shrinkwrap file and only officially support npm (or anything that respects the published lock file, which I think is only npm).
- no_wizard 3y ago
  I think yarn and pnpm can read these
  - davidmurdoch 3y ago
    They don't use published lock files.
- 3np 3y ago
  Why do you prefer shrinkwrap instead of npm's package-lock.json?
  - davidmurdoch 3y ago
    Package locks are ignored by npm when published.
    - 3np 3y ago
      Oh, so do I understand it right if you want your users to use specific pinned version for all of your dependencies? In that case why not pin to specific versions (as opposed to semver ranges) in package.json, which would be respected identically by npm/yarn/pnpm and again remove the need for shrinkwrap?
      - davidmurdoch 3y ago
        Yes. Because that doesn't pin dependencies of your dependencies (this is what "transitive dependencies" means).

jaapz 3y ago

As someone who has been bitten time and time again by transitive dependencies not adhering to semver while still claiming to be using semver, i totally agree. Dependency pinning is a must when you want to have full control, and even then some package managers allow pulling or updating specific versions of dependencies (looking at you, pip).

User23 3y ago
One can conceive of a solution that bisects dependencies independently to find the latest version that satisfies a comprehensive test suite. But that build and test is going to have to be reasonably quick. Even moreso if you take the exhaustive approach and transitively apply the algorithm to dependency dependencies which would also require backtracking. A more durable approach might be to require every API to formally and verifiably provide precondition and postcondition that can be checked with an SMT solver for every public interface. With current tech that is pushing a lot onto module authors, but I believe there is some research being done on inferring both preconditions and postconditions from the semantics of the component statements so perhaps we can avoid forcing programmers to carry the intolerable burden of learning the first order predicate calculus.

hinkley 3y ago

I haven’t figured out the math yet, but on my shelf if future projects, I have a part of a plan to tie semver to unit tests. The theory so far is: there’s a subset of unit tests that specify the API. If you create new tests, that’s mostly fine. If have to alter a test, that’s probably a breaking change. The main problem I can already see with this, which is why it’s on the shelf, is that some test frameworks and many tester writers cannot guarantee the invariants as stated above. You end up changing tests for minor behavioral changes. You mix tests of private fields in with tests of public ones. And even if you split them, then functional tests end up defining parts of the API. But somewhere out there in the future is a language where types are defined by tests, and certain testing patterns indicate minor version increments and others major.

jrochkind1 3y ago
I've thought about this too. The other issue is that tests can have bugs too, sometimes you are just fixing a test bug.
- hinkley 3y ago
  Definitely. So in a robust system, you might also have to look at code coverage to see if the test interacts with any code changes since the last minor version increment. If I change the test but not the code, that may be a clarification. If I change both, it’s ambiguous.

davidmurdoch 3y ago

This has bitten me twice before: a transitive dep update broke my library. I now publish my npm shrinkwrap file and only officially support npm (or anything that respects the published lock file, which I think is only npm).

no_wizard 3y ago
I think yarn and pnpm can read these
- davidmurdoch 3y ago
  They don't use published lock files.
3np 3y ago
Why do you prefer shrinkwrap instead of npm's package-lock.json?
- davidmurdoch 3y ago
  Package locks are ignored by npm when published.
  - 3np 3y ago
    Oh, so do I understand it right if you want your users to use specific pinned version for all of your dependencies? In that case why not pin to specific versions (as opposed to semver ranges) in package.json, which would be respected identically by npm/yarn/pnpm and again remove the need for shrinkwrap?
    - davidmurdoch 3y ago
      Yes. Because that doesn't pin dependencies of your dependencies (this is what "transitive dependencies" means).