3 ms·
Although I don't disagree that the testing of structure in the example given is not particularly good practice, I disagree with the overall message. There is a
by mikekchar 8y ago
Although I don't disagree that the testing of structure in the example given is not particularly good practice, I disagree with the overall message.
There is a difference between "unit tests" and "acceptance tests". "Does this test verify some business requirement" is a question aimed at the latter, not the former. There is nothing wrong with writing automated acceptance tests (and I highly encourage it), but they are not a replacement for unit tests.
What are unit tests for? They are for helping you reason about the effect of changes on your code. There are two main questions a unit test should answer. 1: What happens when I do X (where X is calling some code)? 2: Does the behaviour change if I modify the production code?
The first question is for helping you write the code in the first place (or for modifying it subsequently). It is not a question of the application operating properly, it's a question of the "unit" operating the way you expect. You should be able to use this information to reason quickly about what went wrong in the case that behaviour in the system changes unexpectedly: "Unit a" should behave like X, and I can see that it does so by looking at the tests. If that's the case, then "Unit y" must be at fault.
You can think of these kinds of tests as being sort of like predefined watch points in your debugger. You should expose state in your units and add tests so that you can understand the state of the unit if it doesn't behave as expected. You should be thinking, "If the behaviour of this unit is unexpected, what things would I want to look at when determining why".
You might be wondering (as many do) what a "unit" is. The answer is that it can be anything: a function, a class, a global object, a group of classes/objects acting in concert. It really doesn't matter. Some people like to fix the idea of a "unit" as a class and also add "integration tests". I have no problem with that, other than I have seen no real value in the practice. Mostly I think people find it easier to envision how to name the files that contain their tests.
Many styles of testing do a relatively good job of doing what I describe above. However many people isolate their "units" too much. I think this is an unfortunate consequence of the word "unit" which seems to imply "atomic unit" to many people. The word (IMHO) refers more to simply an identifiable chunk of code. Often people avoid using real objects as collaborators in their tests in the mistaken belief that it will help with the specificity of their tests.
The second question, "Does the behaviour change if I modify the production code" is as important as the first question. Imagine a scenario where you have a series of either manual or automated acceptance tests. You run those tests and you feel confident that the application is running as you expect. You then modify some code. At this point, it would be very nice to be told what behaviour (if anything) has changed in the application.
You might be forgiven for thinking that's the same as a regression test. However, it is not necessary to actually write regression tests to give you that information. We don't actually need to know whether or not the behaviour is correct -- we only need to know if it is different (and ideally, what is different).
As an example, I once worked on code that converted files to and from MS Word file format. I didn't write any of the original code and it was pretty stinky, TBH. Every time I made a change, I would likely break something because I couldn't reason about the code. However, we had about 2000 documents that QA had already tested and deemed acceptable. I wrote a quick hack that would allow me to compare the "accepted" output for these documents with the new outcomes. It didn't try to understand what was wrong, just that it was different. In fact, most of the time the changes were good because I was improving the conversions.
Obviously, the above is not a unit test system. However, the point is that we don't need to write complicated tests to determine if the system is working correctly. All we need is to have a known baseline for the system and then be alerted when the behaviour changes -- either in a "good" or "bad" way. Then we can have a quick look to see if the change in behaviour was what we expected.
In that way, the second criteria for good unit tests is that at least one unit test "fails" when I change the behaviour of the system. A lot of people equate TDD (Test Driven Development) with Test First Development. This is unfortunate because it steers people off course. Test First is a good way to learn TDD, but it's not by any stretch of the imagination the only way to do TDD. When I have a piece of code with good unit tests, I will often modify production code first and see what tests fail. If I have changed behaviour and no tests fail, I know I'm in big trouble. Similarly, if 200 tests fail and I have no specificity in my tests, I know I'm going to be in for a hard ride. These are the kinds of things you should be doing to test the quality of your tests.
I'll finish this with one last thought. There are 2 classes of "customer" for tests: the end user and the programmer. The user is concerned about whether or not the application is working as desired. They don't care what is technically wrong and they don't need to reason about how to change unexpected behaviour. The programmer doesn't actually need to know what the correct behaviour is for everything in the system. They need to have a known baseline of behaviour and to be told when the behaviour has changed. They also need to know if the very specific piece of code that they are currently working on is behaving as expected. The customer for unit tests are the programmer, not the end user. Having tests for the end user is a great idea, but doesn't actually help tremendously with improving the development experience. The golden rule for programming tests as a programmer: if it helps you reason about what you are doing, then it's a good idea. If it doesn't, then you should avoid it.
- aswin4 8y agoHoly sh!t this blew my mind.
- krytenboot 8y agoThis is a great comment. As a small addition to this, a good technique to assess if your unit tests are able to catch behavior changes in an application when the code changes is Mutation Testing. This is a small explanation and demo I made recently with Stryker, a framework for the JavaScript ecosystem https://github.com/peter-evans/mutation-testing https://github.com/peter-evans/mutation-testing This is a good list of available frameworks in other languages: https://github.com/theofidry/awesome-mutation-testing https://github.com/theofidry/awesome-mutation-testing