4 ms·
"a good system ensures even garden variety engineers will deliver." I assume this points to the idea that test procedures can replace individual expertise ? My
by waps 14y ago
"a good system ensures even garden variety engineers will deliver."
I assume this points to the idea that test procedures can replace individual expertise ? My experience is that testing systems get bypassed by "garden variety" engineers (mostly with PM/Manager support), usually in exactly the cases where you wouldn't want them to do that.
Even when this is not the case, tests, no matter the coverage, don't cover everything. The 100% test coverage demands that are in vogue these days make sure tests are usually written specifically to minimize interactions between pieces of code, which is of course exactly what a good engineer would test for.
Of course with tests testing interaction, there are pieces of code (the ones that worry me) that get tested 1000x (in one case literally iterating over all possible calls and verifying constraints afterwards, almost fuzzing), and there's pieces of code that just don't get attention (usually an if out_of_range check at the beginning of functions, since I would just test for the same range, with potentially the same mistake in both the used number and the tested number, but other things happen too. If the consequences of the function screwing up is a slight UI aberration ... well I barely test UI code at all, really. MVC for the app MC for the tests). Also there's pieces of code where I know they're potentially not thread safe and so, just to make sure big screwups get caught, I test firing off the same test 1000x in parallel. If the function works entirely based of the stack and return values, no such test is done.
- reeses 14y agoGood points. Most requirements do not include error or exception handling requirements, so when developers write their test cases (in the very few cases when they do so), they usually do not write negative test cases. Obviously, NFRs need to cover those, but in many organizations, they're seen as being "in the way of getting work done," just like comprehensive automated tests.