2 ms·
This article makes some useful and clear distinctions, but it's worth pointing out that the goal of level 3 (guaranteed compatibility with new versions) is impo
by skybrian 9y ago
This article makes some useful and clear distinctions, but it's worth pointing out that the goal of level 3 (guaranteed compatibility with new versions) is impossible even in principle when most libraries and environments don't have formal specifications that they guarantee they will stick to in future versions.
For example, there's no guarantee a web app will work in a new browser or even a new version of a browser. Browser vendors try to avoid breakage for most apps but not necessarily your app. They will make backward-incompatible changes to rarely used API's.
A web app usually will work with a new browser, but this is an argument based on statistics, not logic. There is good reason to run automated level 1 tests against beta versions of browsers, to get an early warning of breakage.
Reasoning based on API specifications has a lot in common with mock-heavy testing. It can sometimes find logic bugs that are your own fault, but it's no good at finding breakage that is not your fault (but may still be your responsibility).