4 ms·

From my own experience: Occasionally it feels awkward, but more often than not, the same patterns that make testing easier also lead to an architecture that is

by TOGoS 3y ago

From my own experience: Occasionally it feels awkward, but more often than not, the same patterns that make testing easier also lead to an architecture that is easier for me to understand and reason about, because 'having a minimal set of well-defined functions' happens to be good for both, and also leads to better composability, etc etc. And even in cases where they are not the same, I find that the confidence that automated tests give me is worth the little bit of gymnastics[1] I have to do to make the things testable. YMMV. :-P [1] And maybe I should note that I stay clear of all the fancy gizmos that 'modern' testing frameworks provide. If you need to mess around with a mocking framework, that means you did it wrong. It's hard to improve on `assertEquals( expectedOutput, doTheThing(input) )`.

PaulHoule 3y ago
Sometimes it is an improvement, sometimes it is https://dhh.dk/2014/test-induced-design-damage.html https://dhh.dk/2014/test-induced-design-damage.html
- TOGoS 3y ago
  > This presentation shows the fundamentals of the "decoupling from Rails" I 100% understand why someone would want to do that, though, for unit testing or otherwise. Rails (and every framework inspired by it) seems almost intentionally designed to force you into coding a big ball of mud where everything is held together with magic framework duct tape. I guess some people's brains are wired to enjoy working in big mud balls, but for those of us who don't, adding a decouple-from-Rails-or-Laravel-or-Spring-Boot-or-what-have-you layer is the only hope we have of continuing to receive a paycheck while keeping our sanity intact.
scubbo 3y ago
> If you need to mess around with a mocking framework, that means you did it wrong This is a debate that's been going around my professional circle for a while, I'd love to hear a bit more about your take on it. My position is fairly opposed to this - that mocks are a valuable tool allowing you to they test situations that are not reachable based purely on input (if you're not mocking, "how does my code behave in the presence of error responses from a dependency?" can only be tested by hard-coding the dependency with "when you encounter this particular input, short-circuit your logic and return an error"), and let you test otherwise-slow logic quickly. I can see, and agree with, the argument that tests which rely on mocking have their confidence bounded-above by your confidence in the correctness of your mocking (that is - if you test against incorrect expectations, your tests will pass for code that will fail in production. GIGO), but to my mind that doesn't mean that tests based on mocks are inherently bad - simply that their restrictions must be acknowledged. They have strengths and weaknesses, and are most valuable as part of a suite of testing strategies. Mocks themselves are not inherently bad, even though an over-reliance on them is.
- ramses0 3y ago
  An honest answer, tons of tests with mocks are really frustrating to find. Anything that simply “must” be mocked is better served by returning a “command pattern” response. eg: return new DoGood() || new DoBad(), and separately: TestableIsGoodOrBad() Specifically for your “error from a dependency” situation, I’m of the mind that a function consisting of: function foo() { xyz = PrepFoo(...) try { response = DoFoo(xyz) result = Process( response ) } except { HandleError() } } ...that’s the three lines of code that I’m spending my “not 100% code coverage” on. Each sub-function is 1000% testable without mocks, and the combination of functions is a “secure, logic-less composition”, with little test value. Basically, if you write your code that way you _never_ need mocks, and generally “mocks are dumb” because saying “Mock( DoRequest( FakeRequest() ), FakeResponse() )” is just wishful thinking compared to a properly factored codebase with _real_ non-mocked, significant, functional-only components.
  - scubbo 3y ago
    Oh interesting. So in a sense you are passing the "dependency's response" as a parameter _to_ `Process`, and thus removing the need to mock a dependency in order to induce an erroring dependency-response - you can instead _create_ a payload that resembles an erroring dependency-response, and then directly test "when `Process` is called with `BadPayload`, it should respond as follows". Neat - thanks!
  - scubbo 3y ago
    Hmm - but hang on, though. Doesn't that just bring the problem up a level? This setup assumes that you have "shallow" logic: that `Process` will not call any other methods, and especially that it won't call any dependency services. While it's possible to unit test `DoFoo` and `Process` independently, you've now given up the ability to unit test `foo` itself - more specifically, you've given up the ability to determine the behaviour of `foo` in the presence of arbitrary behaviour from code which is calls (since specifying `DoFoo`'s behaviour would be mocking). I didn't initially follow what you meant by "that’s the three lines of code that I’m spending my “not 100% code coverage” on", but in the context of this realization, I'm guessing you're saying something like "when my code has to call external dependencies, I wrap the call in this ~3-line pattern, which allows me to directly test the behaviour of each of the components of the call, and I'm comfortable not testing their integration together because it's so straightforward"? Or are you saying that this Command Pattern should be applied _every_ time one calls another function? If so - I guess that works, but I'm back to wondering what that gains? After all, both mocking and this Command Pattern serve to create tests which assert "when my code encounters situation X, it responds in fashion Y" - the difference is just on the "direction of encountering" (mocking controls "when my code calls other code and receives X", Command Pattern controls "when my code is passed Command X"). Don't they both have the same strengths (ability to induce arbitrary situations for the code-under-test to deal with) and weaknesses (if the developer's beliefs about the situations in which X arises are wrong, the tests are asserting on incorrect behaviour; if the business case changes, especially if the type of X changes, then the tests need to be rewritten to match the new situation)?

by TOGoS 3y ago

PaulHoule 3y ago

Sometimes it is an improvement, sometimes it is https://dhh.dk/2014/test-induced-design-damage.html https://dhh.dk/2014/test-induced-design-damage.html

TOGoS 3y ago
> This presentation shows the fundamentals of the "decoupling from Rails" I 100% understand why someone would want to do that, though, for unit testing or otherwise. Rails (and every framework inspired by it) seems almost intentionally designed to force you into coding a big ball of mud where everything is held together with magic framework duct tape. I guess some people's brains are wired to enjoy working in big mud balls, but for those of us who don't, adding a decouple-from-Rails-or-Laravel-or-Spring-Boot-or-what-have-you layer is the only hope we have of continuing to receive a paycheck while keeping our sanity intact.

scubbo 3y ago

> If you need to mess around with a mocking framework, that means you did it wrong This is a debate that's been going around my professional circle for a while, I'd love to hear a bit more about your take on it. My position is fairly opposed to this - that mocks are a valuable tool allowing you to they test situations that are not reachable based purely on input (if you're not mocking, "how does my code behave in the presence of error responses from a dependency?" can only be tested by hard-coding the dependency with "when you encounter this particular input, short-circuit your logic and return an error"), and let you test otherwise-slow logic quickly. I can see, and agree with, the argument that tests which rely on mocking have their confidence bounded-above by your confidence in the correctness of your mocking (that is - if you test against incorrect expectations, your tests will pass for code that will fail in production. GIGO), but to my mind that doesn't mean that tests based on mocks are inherently bad - simply that their restrictions must be acknowledged. They have strengths and weaknesses, and are most valuable as part of a suite of testing strategies. Mocks themselves are not inherently bad, even though an over-reliance on them is.

ramses0 3y ago
An honest answer, tons of tests with mocks are really frustrating to find. Anything that simply “must” be mocked is better served by returning a “command pattern” response. eg: return new DoGood() || new DoBad(), and separately: TestableIsGoodOrBad() Specifically for your “error from a dependency” situation, I’m of the mind that a function consisting of: function foo() { xyz = PrepFoo(...) try { response = DoFoo(xyz) result = Process( response ) } except { HandleError() } } ...that’s the three lines of code that I’m spending my “not 100% code coverage” on. Each sub-function is 1000% testable without mocks, and the combination of functions is a “secure, logic-less composition”, with little test value. Basically, if you write your code that way you _never_ need mocks, and generally “mocks are dumb” because saying “Mock( DoRequest( FakeRequest() ), FakeResponse() )” is just wishful thinking compared to a properly factored codebase with _real_ non-mocked, significant, functional-only components.
- scubbo 3y ago
  Oh interesting. So in a sense you are passing the "dependency's response" as a parameter _to_ `Process`, and thus removing the need to mock a dependency in order to induce an erroring dependency-response - you can instead _create_ a payload that resembles an erroring dependency-response, and then directly test "when `Process` is called with `BadPayload`, it should respond as follows". Neat - thanks!
- scubbo 3y ago
  Hmm - but hang on, though. Doesn't that just bring the problem up a level? This setup assumes that you have "shallow" logic: that `Process` will not call any other methods, and especially that it won't call any dependency services. While it's possible to unit test `DoFoo` and `Process` independently, you've now given up the ability to unit test `foo` itself - more specifically, you've given up the ability to determine the behaviour of `foo` in the presence of arbitrary behaviour from code which is calls (since specifying `DoFoo`'s behaviour would be mocking). I didn't initially follow what you meant by "that’s the three lines of code that I’m spending my “not 100% code coverage” on", but in the context of this realization, I'm guessing you're saying something like "when my code has to call external dependencies, I wrap the call in this ~3-line pattern, which allows me to directly test the behaviour of each of the components of the call, and I'm comfortable not testing their integration together because it's so straightforward"? Or are you saying that this Command Pattern should be applied _every_ time one calls another function? If so - I guess that works, but I'm back to wondering what that gains? After all, both mocking and this Command Pattern serve to create tests which assert "when my code encounters situation X, it responds in fashion Y" - the difference is just on the "direction of encountering" (mocking controls "when my code calls other code and receives X", Command Pattern controls "when my code is passed Command X"). Don't they both have the same strengths (ability to induce arbitrary situations for the code-under-test to deal with) and weaknesses (if the developer's beliefs about the situations in which X arises are wrong, the tests are asserting on incorrect behaviour; if the business case changes, especially if the type of X changes, then the tests need to be rewritten to match the new situation)?