4 ms·

I have seen too many badly written tests, so I tend to agree. Most tests will just fail on any change in the UUT, giving you lots of false positives. This ult

by clktmr 6y ago

I have seen too many badly written tests, so I tend to agree. Most tests will just fail on any change in the UUT, giving you lots of false positives. This ultimately results in fixing the tests instead of the UUT and blurring the initial intention of the test. Writing testable code is not about dependency injection etc. It's about having the behaviour of a unit clearly defined. This can be done explicitly in text, but also implicitly by using the component in a certain way. If anything beyond that behaviour is tested, you start to defining the behavior in the testcase. This is fine, but often happens unintentionally leading to a very weird and very specific definition of the behaviour. For example, I work on a component that generates a configuration. I know that the order of lines of the output doesn't matter and should not matter. But the initial author wrote tests that fail if the order changes, because they just check the output against a string.

chriswarbo 6y ago
Could I ask how are you defining your terms? For example, what counts as the "unit under test" (UUT) in your opinion? What would count as "behaviour"? What counts as a "component"? I'm curious since I think there is a lot of cross-talk when it comes to the terminology around testing.
- clktmr 6y ago
  Here I used "UUT" and "component" synonymously, which is simply all the code that should be tested. With behaviour I was referring to expected output for a given input. There is definitely a lot of cross-talk. Can you make an example where my previous comment would be misguided?
  - chriswarbo 6y ago
    When I read your comment, my initial thought was that this sounds like a classic case of overly-isolated tests (e.g. mocking everything except for a single class, essentially re-implementing all those other components again and again). It's much less common to get spurious failures when testing at a higher level (what some might call "end to end tests", or "functional tests"; or even "integration tests", but that's a phrase I tend to reserve for interactions with third-party components). When I read your final paragraph I realised that I've hit that same problem many times myself, e.g. checking if two things are equal, when in fact the order doesn't matter!

by clktmr 6y ago

chriswarbo 6y ago

Could I ask how are you defining your terms? For example, what counts as the "unit under test" (UUT) in your opinion? What would count as "behaviour"? What counts as a "component"? I'm curious since I think there is a lot of cross-talk when it comes to the terminology around testing.

clktmr 6y ago
Here I used "UUT" and "component" synonymously, which is simply all the code that should be tested. With behaviour I was referring to expected output for a given input. There is definitely a lot of cross-talk. Can you make an example where my previous comment would be misguided?
- chriswarbo 6y ago
  When I read your comment, my initial thought was that this sounds like a classic case of overly-isolated tests (e.g. mocking everything except for a single class, essentially re-implementing all those other components again and again). It's much less common to get spurious failures when testing at a higher level (what some might call "end to end tests", or "functional tests"; or even "integration tests", but that's a phrase I tend to reserve for interactions with third-party components). When I read your final paragraph I realised that I've hit that same problem many times myself, e.g. checking if two things are equal, when in fact the order doesn't matter!