4 ms·

Yeah, maybe there is an approach that works well with mocks but this has always concerned me as well. It's rare (for me anyway) that I want to test which calls

Yeah, maybe there is an approach that works well with mocks but this has always concerned me as well. It's rare (for me anyway) that I want to test which calls a function makes rather than if the function returns the correct result. I can imagine scenarios where that is useful and I know how to use mocks when that comes up ... but the examples are always a system where mock based testing is strangely coupled to implementation, very fragile and have the scary failure mode you describe.

shados 10y ago
When you're testing functions in an impure language (pretty much any language, short of extremes like Elm, maybe Haskell, etc), functions are used not only for their return value but also their side effects. Side effects can be things like making an http request or writing to a file, it can be changing some global variable, but it can also be something like calling a public method of another public object. Since that function/method could also be impure, the very act of calling it is actually, in a way, part of the contract of the function under test: it returns a value, and it creates a side effect in the form of calling another function. The only real way to test for those side effects is to pass in a stub or mock and to check that the function was called correctly. That makes your tests brittle, but it's the only way to make sure it does what you think it does (even if you only call the function for an intermediate return value, the fact that changing the intermediate function could change the result of the function under test). Those tests are brittle because functions with side effects are, in fact, brittle, and favoring pure functions (where the only thing that matter is the return value, and dependencies are composed or passed as arguments) makes both your tests and your code better.
lojack 10y ago
Typically, I find that I'm concerned with what calls a function makes when those calls interface things external to my program. I.e. I'm asserting that a request to a given http endpoint is made, or that an smtp client sends an email. I assume the external libraries work and am testing my interface to them.
- taeric 10y ago
  This is where I find myself having to use these. Though, I greatly prefer to have the test merely check for the result of the call, if I can. e.g., if I have to post to a url to delete something, check to see if said something exists. In the end, it is not the call that I care about, but what the call was supposed to do. This way can lead to brittle tests, clearly. But the reality seems to be that brittle systems lead to brittle tests. There is no testing panacea.

shados 10y ago

When you're testing functions in an impure language (pretty much any language, short of extremes like Elm, maybe Haskell, etc), functions are used not only for their return value but also their side effects. Side effects can be things like making an http request or writing to a file, it can be changing some global variable, but it can also be something like calling a public method of another public object. Since that function/method could also be impure, the very act of calling it is actually, in a way, part of the contract of the function under test: it returns a value, and it creates a side effect in the form of calling another function. The only real way to test for those side effects is to pass in a stub or mock and to check that the function was called correctly. That makes your tests brittle, but it's the only way to make sure it does what you think it does (even if you only call the function for an intermediate return value, the fact that changing the intermediate function could change the result of the function under test). Those tests are brittle because functions with side effects are, in fact, brittle, and favoring pure functions (where the only thing that matter is the return value, and dependencies are composed or passed as arguments) makes both your tests and your code better.

lojack 10y ago

Typically, I find that I'm concerned with what calls a function makes when those calls interface things external to my program. I.e. I'm asserting that a request to a given http endpoint is made, or that an smtp client sends an email. I assume the external libraries work and am testing my interface to them.

taeric 10y ago
This is where I find myself having to use these. Though, I greatly prefer to have the test merely check for the result of the call, if I can. e.g., if I have to post to a url to delete something, check to see if said something exists. In the end, it is not the call that I care about, but what the call was supposed to do. This way can lead to brittle tests, clearly. But the reality seems to be that brittle systems lead to brittle tests. There is no testing panacea.