3 ms·
The roots of the problem you described are many. One is that there are very good unit test libraries out there, which encourages writing unit tests. That sounds
by pierrebai 5y ago
The roots of the problem you described are many. One is that there are very good unit test libraries out there, which encourages writing unit tests. That sounds good, and is a good starting point, but it often become the be-all end-all of testing.
In my experience working on many very large software, I have encountered one that was head and shoulder above the others in quality. This is not just a hunch. There were metrics tracked, thing like MTBF, and percent of sessions with clean exits, etc and this software was ridiculously better than the others.
The difference lied entirely on the fact that the better software had almost no unit test, but had 21K+ integration tests. I've worked on other software that also had more integration tests than average, and they too were of better quality.
The problem is making integration testing easy needs to be designed in from the start and requires not just one simple trick but multiple inter-locked design decisions that accumulate into making integration testing easy. In this particular case, one could write a new test just by using the software and a new test would be generated automatically. That's how you achieve writing 21k integration tests.
There are many unit test framework, but I've yet to see a generic integration framework. The reason is that integration tests are run with the whole software, so the whole software needs to be designed around them.
- rixed 5y ago> In this particular case, one could write a new test just by using the software and a new test would be generated automatically I'm interested in the topic. Could you talk more about it, or point to some resources describing in more details how this was achieved/what decisions had to be taken?
- usefulcat 5y agoI’ve written a great deal of tests over the years and this sounds very interesting to me, yet it’s not obvious to me from your comment what this would look like. I’d like it very much if you could elaborate further.
- eternityforest 5y agoIntegration tests are hard to do right, but easy to hack together. You might have to do the whole "Start with a clean DB, fake push a button, check for this record" thing, but at least you'll have some tests. They might be a maintenance nightmare but... they'll probably be better than no tests