3 ms·
I haven’t figured out the math yet, but on my shelf if future projects, I have a part of a plan to tie semver to unit tests. The theory so far is: there’s a su
by hinkley 3y ago
I haven’t figured out the math yet, but on my shelf if future projects, I have a part of a plan to tie semver to unit tests.
The theory so far is: there’s a subset of unit tests that specify the API. If you create new tests, that’s mostly fine. If have to alter a test, that’s probably a breaking change.
The main problem I can already see with this, which is why it’s on the shelf, is that some test frameworks and many tester writers cannot guarantee the invariants as stated above. You end up changing tests for minor behavioral changes. You mix tests of private fields in with tests of public ones. And even if you split them, then functional tests end up defining parts of the API.
But somewhere out there in the future is a language where types are defined by tests, and certain testing patterns indicate minor version increments and others major.
- jrochkind1 3y agoI've thought about this too. The other issue is that tests can have bugs too, sometimes you are just fixing a test bug.
- hinkley 3y agoDefinitely. So in a robust system, you might also have to look at code coverage to see if the test interacts with any code changes since the last minor version increment. If I change the test but not the code, that may be a clarification. If I change both, it’s ambiguous.