3 ms·

The mocks are the symptom. The problem is that your code doesn't restrict side effects in any way. And so you end up with integration tests for everything and s

The mocks are the symptom. The problem is that your code doesn't restrict side effects in any way. And so you end up with integration tests for everything and setting up a single test requires recreating the universe from scratch and slightly tweaking it on every run. But that's what our program does, it talks to databases and file systems and HTTP servers! Sure, and the effect of doing those things is moving data around. What does your program do with the results of these side-effects? Does it parse it? Transform it in any way? Decide whether to run effect A next with the result or effect B? This is the code that, if extracted, can be tested in isolation of databases and HTTP servers. You want as much of your code to be in this place as possible. It's a thousand times easier to test and the tests are thousands of times more reliable because they don't perform any effects. Mocks have their places but if you can't test your code without mocking out the universe then the problem is that your code is interleaving too many effects with the core logic of the program. The cure is to refactor effects to the edges of your program and run effects in one place in your code. Make the rest just plain, pure code as much as possible.

Spivak 5y ago
But what you’re describing creates tight coupling between your integration points and your internal logic that doesn’t look like tight coupling. You’re not feeding you code real data, you’re only feeding it what you think that external service will provide. And to validate that your tests are correct you need integration tests again and to enforce that your integration point code always produces consistent valid output. So yeah, the advice to avoid side effects is good but still exercise your integration points with real data and services.
- ItsMonkk 5y ago
  I'm confused as to where any mention of integration tests are? Integration tests can't use Mocks, as then they wouldn't be testing the integration. Unit tests can use Mocks, but as the article and GP and Mark Seemann[0] points out, that is always worse than writing your logic in a pure way, if you can. [0]: https://blog.ploeh.dk/2017/02/02/dependency-rejection/ https://blog.ploeh.dk/2017/02/02/dependency-rejection/
  - Spivak 5y ago
    Because you don’t gain as much as you think when you do this. Before your external external service touches your integration point and then runs some logic. You mock the external service to unit test the internal logic. Then you run integration tests to exercise the integration point. In the new world you have an external service talking to an integration point which then passes the data to your internal logic. You don’t need the mocks anymore because you can just call your logic function with your data that would have come from the mock. Great! You run integration tests again to test the integration point. But now you have another linkage at the call site. Your internal logic function now has a dependency on the output of your integration point and that’s an invariant that has to be tested to catch someone modifying one but not the other. With enough discipline you might be able to make the type system do these kinds of checks for you but IRL very few people do enough to say catch an external string field changing format slightly.
    - ItsMonkk 5y ago
      I think I was confused because I naturally do what you say few people do. My external calls all parse into an object that you can then pass into the unit. It's the integration tests job to ensure that the parse works. Then its the unit tests job to ensure that you run enough that you can be reasonably sure that you cover all cases.
    - u801e 5y ago
      > You mock the external service to unit test the internal logic You only need the output produced by that external service or the the input expected by that service. For example, I can pass in a data structure or object representing the HTTP request to the method that I'm unit testing, but I don't need to mock a client to generate those requests to test that logic.
pydry 5y ago
>Sure, and the effect of doing those things is moving data around. What does your program do with the results of these side-effects? Does it parse it? Transform it in any way? Decide whether to run effect A next with the result or effect B? This is the code that, if extracted, can be tested in isolation of databases and HTTP servers. It's very frequent that this "decision making/calculation" code: * Doesn't do very much. * Isn't even in the top 5 source of bugs for your app. * Integration tests caught those bugs just fine anyway. * The process of extraction balloons the size of the code base (all those yummy extra layers of indirection), sometimes introducing fun bugs. It's certainly the right thing to do if you have a self contained chunk of complex decision making/calculation code (e.g. a parser or pricing engine). However, if you do this as a matter of principle (and far too many do) then this advice isn't just wrong, it's dangerously wrong.
- agentultra 5y ago
  Be careful. Principled application of this advice can lead to separation of concerns. Programs that are broken down into independently verifiable modules can lead to an abundance of spare time and pursuit of new features in the absence of errors. Ensure your job security by making sure that it is difficult to test your code. In all seriousness though it's smart to take a layered approach to testing. It is plain good engineering to be able to separate your program's concerns into independent, verifiable modules and to isolate side-effects from pure code. Definitely write a few integration tests to verify that the modules work together and that global properties like configuration have the desired behaviours. And by jolly live dangerously and test in production! But don't do it dangerously. Make sure you have the right team and infrastructure to test in production safely without taking down production or frustrating customers. But test in production for reals. Integration tests are fine but nothing compares to prod. Not mocks, synthetic inputs, simulated environments, and not good intentions.

Spivak 5y ago

But what you’re describing creates tight coupling between your integration points and your internal logic that doesn’t look like tight coupling. You’re not feeding you code real data, you’re only feeding it what you think that external service will provide. And to validate that your tests are correct you need integration tests again and to enforce that your integration point code always produces consistent valid output. So yeah, the advice to avoid side effects is good but still exercise your integration points with real data and services.

ItsMonkk 5y ago
I'm confused as to where any mention of integration tests are? Integration tests can't use Mocks, as then they wouldn't be testing the integration. Unit tests can use Mocks, but as the article and GP and Mark Seemann[0] points out, that is always worse than writing your logic in a pure way, if you can. [0]: https://blog.ploeh.dk/2017/02/02/dependency-rejection/ https://blog.ploeh.dk/2017/02/02/dependency-rejection/
- Spivak 5y ago
  Because you don’t gain as much as you think when you do this. Before your external external service touches your integration point and then runs some logic. You mock the external service to unit test the internal logic. Then you run integration tests to exercise the integration point. In the new world you have an external service talking to an integration point which then passes the data to your internal logic. You don’t need the mocks anymore because you can just call your logic function with your data that would have come from the mock. Great! You run integration tests again to test the integration point. But now you have another linkage at the call site. Your internal logic function now has a dependency on the output of your integration point and that’s an invariant that has to be tested to catch someone modifying one but not the other. With enough discipline you might be able to make the type system do these kinds of checks for you but IRL very few people do enough to say catch an external string field changing format slightly.
  - ItsMonkk 5y ago
    I think I was confused because I naturally do what you say few people do. My external calls all parse into an object that you can then pass into the unit. It's the integration tests job to ensure that the parse works. Then its the unit tests job to ensure that you run enough that you can be reasonably sure that you cover all cases.
  - u801e 5y ago
    > You mock the external service to unit test the internal logic You only need the output produced by that external service or the the input expected by that service. For example, I can pass in a data structure or object representing the HTTP request to the method that I'm unit testing, but I don't need to mock a client to generate those requests to test that logic.

pydry 5y ago

>Sure, and the effect of doing those things is moving data around. What does your program do with the results of these side-effects? Does it parse it? Transform it in any way? Decide whether to run effect A next with the result or effect B? This is the code that, if extracted, can be tested in isolation of databases and HTTP servers. It's very frequent that this "decision making/calculation" code: * Doesn't do very much. * Isn't even in the top 5 source of bugs for your app. * Integration tests caught those bugs just fine anyway. * The process of extraction balloons the size of the code base (all those yummy extra layers of indirection), sometimes introducing fun bugs. It's certainly the right thing to do if you have a self contained chunk of complex decision making/calculation code (e.g. a parser or pricing engine). However, if you do this as a matter of principle (and far too many do) then this advice isn't just wrong, it's dangerously wrong.

agentultra 5y ago
Be careful. Principled application of this advice can lead to separation of concerns. Programs that are broken down into independently verifiable modules can lead to an abundance of spare time and pursuit of new features in the absence of errors. Ensure your job security by making sure that it is difficult to test your code. In all seriousness though it's smart to take a layered approach to testing. It is plain good engineering to be able to separate your program's concerns into independent, verifiable modules and to isolate side-effects from pure code. Definitely write a few integration tests to verify that the modules work together and that global properties like configuration have the desired behaviours. And by jolly live dangerously and test in production! But don't do it dangerously. Make sure you have the right team and infrastructure to test in production safely without taking down production or frustrating customers. But test in production for reals. Integration tests are fine but nothing compares to prod. Not mocks, synthetic inputs, simulated environments, and not good intentions.