5 ms·

Package registries should implement some sort of required automated integration testing. If a package owner wants to update their package with what they claim i

Package registries should implement some sort of required automated integration testing. If a package owner wants to update their package with what they claim is a non-breaking change according to semver, then the new version must pass all integration test cases of the previous. This could also include crowdsourced testing of that package, that allows users of that package to add tests over time to that particular version, so as to mitigate owners who don't test their packages well enough.

baddox 9y ago
Then you’re just defining “breaking change” as “change that breaks the tests.” Anyone with any programming experience knows that “but the tests passed” is never a valid excuse for a bug.
- jitl 9y ago
  Some verification would be better than no verification. You still need to verify and test package upgrades and pin versions; but it would make the semver contract at least specified to some degree.
jitl 9y ago
I think this is an excellent idea, particularly for typed languages. For a typed language package manager, you can run the following verifications: - for a patch version, all exported (name, type) pairs of A.B.C are identical to those of A.B.(C-1) - for a minor version, enforce first rule, but allow new types for new names. I think the Elm package manager does this, based on other comments here in this thread. Either way, that sort of automatic “exports” verification could be a base layer in a semver verification package manager, with integration tests layered on top.
- vbernat 9y ago
  For C ABI, this exists since quite some time: https://abi-laboratory.pro/tracker/timeline/libevent/ https://abi-laboratory.pro/tracker/timeline/libevent/ (though, there is not really a way to enforce).
- int_19h 9y ago
  There's a lot more to in than that in practice. For example, in a language with function overloading and implicit conversions, adding an overload may silently change the meaning of existing code. Sometimes, this can get very gnarly. For example, C# has overloading, and it also has type inference for lambdas; and it tries to make these two work seamlessly. So, consider something like this: int F(Func<int, int> g) => ... string F(Func<string, string> g) => ... This is perfectly legal - just a couple of overloads. Now, suppose we call it like so: F(x => x.ToUpper()); There's no type for x here, so compiler has to infer it; but it also has two possible overloads to deal with. The way it goes about it is by considering both, and seeing if the lambda "makes sense" - i.e. if the body compiles, and its result type (accounting for possible implicit conversions) matches that specified in the function type being considered. Now, when it tries Func<int, int>, it doesn't work, because if x is int, it doesn't have a method ToUpper, so the body of the lambda is ill-formed. On the other hand, given Func<string, string>, x is string, ToUpper() is well-formed, and returns a string, matching the function type. So that's the overload that gets chosen. Now consider what happens if someone adds a method called ToUpper to int: now the code that was compiling before, and that was calling ToUpper on string, will suddenly fail because of ambiguity. In more complicated scenarios (e.g. involving inheritance), it is even possible to have code that silently changes its meaning in such circumstances, e.g. when overloading methods across several classes in a hierarchy. It'd be rather convoluted code, but not out of the realm of possibility.

baddox 9y ago

Then you’re just defining “breaking change” as “change that breaks the tests.” Anyone with any programming experience knows that “but the tests passed” is never a valid excuse for a bug.

jitl 9y ago
Some verification would be better than no verification. You still need to verify and test package upgrades and pin versions; but it would make the semver contract at least specified to some degree.

jitl 9y ago

I think this is an excellent idea, particularly for typed languages. For a typed language package manager, you can run the following verifications: - for a patch version, all exported (name, type) pairs of A.B.C are identical to those of A.B.(C-1) - for a minor version, enforce first rule, but allow new types for new names. I think the Elm package manager does this, based on other comments here in this thread. Either way, that sort of automatic “exports” verification could be a base layer in a semver verification package manager, with integration tests layered on top.

vbernat 9y ago
For C ABI, this exists since quite some time: https://abi-laboratory.pro/tracker/timeline/libevent/ https://abi-laboratory.pro/tracker/timeline/libevent/ (though, there is not really a way to enforce).
int_19h 9y ago
There's a lot more to in than that in practice. For example, in a language with function overloading and implicit conversions, adding an overload may silently change the meaning of existing code. Sometimes, this can get very gnarly. For example, C# has overloading, and it also has type inference for lambdas; and it tries to make these two work seamlessly. So, consider something like this: int F(Func<int, int> g) => ... string F(Func<string, string> g) => ... This is perfectly legal - just a couple of overloads. Now, suppose we call it like so: F(x => x.ToUpper()); There's no type for x here, so compiler has to infer it; but it also has two possible overloads to deal with. The way it goes about it is by considering both, and seeing if the lambda "makes sense" - i.e. if the body compiles, and its result type (accounting for possible implicit conversions) matches that specified in the function type being considered. Now, when it tries Func<int, int>, it doesn't work, because if x is int, it doesn't have a method ToUpper, so the body of the lambda is ill-formed. On the other hand, given Func<string, string>, x is string, ToUpper() is well-formed, and returns a string, matching the function type. So that's the overload that gets chosen. Now consider what happens if someone adds a method called ToUpper to int: now the code that was compiling before, and that was calling ToUpper on string, will suddenly fail because of ambiguity. In more complicated scenarios (e.g. involving inheritance), it is even possible to have code that silently changes its meaning in such circumstances, e.g. when overloading methods across several classes in a hierarchy. It'd be rather convoluted code, but not out of the realm of possibility.