4 ms·

Does anybody else get nervous over architectural patterns that exist only/mainly to make unit testing easier? My preference for these type of situations is to

Does anybody else get nervous over architectural patterns that exist only/mainly to make unit testing easier? My preference for these type of situations is to try to encapsulate as much logic as I can into pure functions. In addition to be being easy to test, they also have additional architectural benefits that go beyond testing.

TOGoS 3y ago
From my own experience: Occasionally it feels awkward, but more often than not, the same patterns that make testing easier also lead to an architecture that is easier for me to understand and reason about, because 'having a minimal set of well-defined functions' happens to be good for both, and also leads to better composability, etc etc. And even in cases where they are not the same, I find that the confidence that automated tests give me is worth the little bit of gymnastics[1] I have to do to make the things testable. YMMV. :-P [1] And maybe I should note that I stay clear of all the fancy gizmos that 'modern' testing frameworks provide. If you need to mess around with a mocking framework, that means you did it wrong. It's hard to improve on `assertEquals( expectedOutput, doTheThing(input) )`.
- PaulHoule 3y ago
  Sometimes it is an improvement, sometimes it is https://dhh.dk/2014/test-induced-design-damage.html https://dhh.dk/2014/test-induced-design-damage.html
  - TOGoS 3y ago
    > This presentation shows the fundamentals of the "decoupling from Rails" I 100% understand why someone would want to do that, though, for unit testing or otherwise. Rails (and every framework inspired by it) seems almost intentionally designed to force you into coding a big ball of mud where everything is held together with magic framework duct tape. I guess some people's brains are wired to enjoy working in big mud balls, but for those of us who don't, adding a decouple-from-Rails-or-Laravel-or-Spring-Boot-or-what-have-you layer is the only hope we have of continuing to receive a paycheck while keeping our sanity intact.
- scubbo 3y ago
  > If you need to mess around with a mocking framework, that means you did it wrong This is a debate that's been going around my professional circle for a while, I'd love to hear a bit more about your take on it. My position is fairly opposed to this - that mocks are a valuable tool allowing you to they test situations that are not reachable based purely on input (if you're not mocking, "how does my code behave in the presence of error responses from a dependency?" can only be tested by hard-coding the dependency with "when you encounter this particular input, short-circuit your logic and return an error"), and let you test otherwise-slow logic quickly. I can see, and agree with, the argument that tests which rely on mocking have their confidence bounded-above by your confidence in the correctness of your mocking (that is - if you test against incorrect expectations, your tests will pass for code that will fail in production. GIGO), but to my mind that doesn't mean that tests based on mocks are inherently bad - simply that their restrictions must be acknowledged. They have strengths and weaknesses, and are most valuable as part of a suite of testing strategies. Mocks themselves are not inherently bad, even though an over-reliance on them is.
  - ramses0 3y ago
    An honest answer, tons of tests with mocks are really frustrating to find. Anything that simply “must” be mocked is better served by returning a “command pattern” response. eg: return new DoGood() || new DoBad(), and separately: TestableIsGoodOrBad() Specifically for your “error from a dependency” situation, I’m of the mind that a function consisting of: function foo() { xyz = PrepFoo(...) try { response = DoFoo(xyz) result = Process( response ) } except { HandleError() } } ...that’s the three lines of code that I’m spending my “not 100% code coverage” on. Each sub-function is 1000% testable without mocks, and the combination of functions is a “secure, logic-less composition”, with little test value. Basically, if you write your code that way you _never_ need mocks, and generally “mocks are dumb” because saying “Mock( DoRequest( FakeRequest() ), FakeResponse() )” is just wishful thinking compared to a properly factored codebase with _real_ non-mocked, significant, functional-only components.
    - scubbo 3y ago
PaulHoule 3y ago
Testing is at its best when you are testing functions or small subsystems that can be created and destroyed independently and be initialized with a known state. For instance the kind of things you write in an introductory programming class (somehow it goes over the heads of most students that the teachers are using automated testing.) It's certifiably insane to not test parsing code, whether or not it is "functional" (input text return a data structure) or not (function gets called to consume text, callbacks get called) UI code is tricky because usually the framework is calling into your code and your code is getting called in a shambolic way. Of course it problematizes testing but it problematic anyway. I have been looking at testing of a React application again and boy is it crazy, where you have to deal with asynchronous updates and don't really know how many times the app will update itself before you know it has "settled down" and it is OK to test the "output". Testing is bad, with tests that should take 0.04 sec taking more like 4 sec, but the user is being subjected to the same thing with lots of waiting, layout shifts, flashes of incorrect information, etc. And don't get me started with the typical "uncontrolled" component which is completely out of control since some application state updating updates the state managed by React, other ones call freaking jQuery (talk about spooky action at a distance!)
blueprint 3y ago
that's virtually precisely what the article is suggesting
KnobbleMcKnees 3y ago
Depends on the definition of easier. If easier is "it won't work with my particular flavour of framework or mocking" then yes, it's a bad idea. If it's inherently untestable for any setup, that's already a smell and needs some sort of attention. If the smell is accompanied by potential to cause catastrophic error or such, I'd rather lean in to changing the code to be more testable even if it feels like an anti-pattern.
mrkeen 3y ago
> architectural patterns that exist only/mainly to make unit testing easier? > try to encapsulate as much logic as I can into pure functions These are one and the same. The suggestion may as well have been "the IO monad is hard to test, so move as much code as you can out into pure functions"
regularfry 3y ago
Not really. It might start out being for the benefit of the tests but knock-on benefits turn up, usually sooner rather than later.
scubbo 3y ago
When I was a young nerd, I was a rabid fan of logic, objectivity, science. Anything which smacked of spirituality (or, worse, religion) was immediately dismissed as useless sentimental nonsense. I recognized that some people might _enjoy_ thinking about these fripperies, but I was convinced that any phenomenon which claimed an effect but did not explain the science-based interactions that mediated it (think astrology or religion) was hogwash. One day I was deathly bored on a school trip, and the only entertainment available was a book on Feng Shui. During the introduction, it advised the practice of imagining a dragon entering a living space through a door, coiling their way through and around the space, and then exiting. The following sentence stated that this was often a good heuristic for helping the human mind judge a space for sight lines, common travel paths, and avoiding clutter. My mind was blown. The dragon wasn't the point - the dragon was a way to hack the human brain (itself a messy, complex, poorly-understood mechanism) to a better outcome than it would get by a straightforward series of explicable evaluations. If the process demonstrably leads to better outcomes - if it is "useful" - then it can still provide value even if you can't explain _why_ it works[0] All of which is a roundabout way of getting to my point, which is: no, I don't get nervous about architectural patterns which _appear_ to only exist to make unit testing easier, because overwhelmingly I have found that "is this architecture easy to test?" is a good heuristic for "does this architecture have well-divided areas and layers of responsibility?". No matter your opinion on the value of tests _themselves_ (and there are very cogent arguments that an over-reliance on unit testing alone can lead to brittleness and false confidence), ceteris paribus making your architecture "more testable" will usually be a good thing in itself, even before any tests are written. [0] this attitude needs some nuance, of course, when it is actually important to explain how and why the process reaches conclusions. See: ML/LLMs/AGI/AI/whatever-the-current-initialism-is, and recourse in the face of disputed judgements or unforeseen effects.

TOGoS 3y ago

From my own experience: Occasionally it feels awkward, but more often than not, the same patterns that make testing easier also lead to an architecture that is easier for me to understand and reason about, because 'having a minimal set of well-defined functions' happens to be good for both, and also leads to better composability, etc etc. And even in cases where they are not the same, I find that the confidence that automated tests give me is worth the little bit of gymnastics[1] I have to do to make the things testable. YMMV. :-P [1] And maybe I should note that I stay clear of all the fancy gizmos that 'modern' testing frameworks provide. If you need to mess around with a mocking framework, that means you did it wrong. It's hard to improve on `assertEquals( expectedOutput, doTheThing(input) )`.

PaulHoule 3y ago
Sometimes it is an improvement, sometimes it is https://dhh.dk/2014/test-induced-design-damage.html https://dhh.dk/2014/test-induced-design-damage.html
- TOGoS 3y ago
  > This presentation shows the fundamentals of the "decoupling from Rails" I 100% understand why someone would want to do that, though, for unit testing or otherwise. Rails (and every framework inspired by it) seems almost intentionally designed to force you into coding a big ball of mud where everything is held together with magic framework duct tape. I guess some people's brains are wired to enjoy working in big mud balls, but for those of us who don't, adding a decouple-from-Rails-or-Laravel-or-Spring-Boot-or-what-have-you layer is the only hope we have of continuing to receive a paycheck while keeping our sanity intact.
scubbo 3y ago
> If you need to mess around with a mocking framework, that means you did it wrong This is a debate that's been going around my professional circle for a while, I'd love to hear a bit more about your take on it. My position is fairly opposed to this - that mocks are a valuable tool allowing you to they test situations that are not reachable based purely on input (if you're not mocking, "how does my code behave in the presence of error responses from a dependency?" can only be tested by hard-coding the dependency with "when you encounter this particular input, short-circuit your logic and return an error"), and let you test otherwise-slow logic quickly. I can see, and agree with, the argument that tests which rely on mocking have their confidence bounded-above by your confidence in the correctness of your mocking (that is - if you test against incorrect expectations, your tests will pass for code that will fail in production. GIGO), but to my mind that doesn't mean that tests based on mocks are inherently bad - simply that their restrictions must be acknowledged. They have strengths and weaknesses, and are most valuable as part of a suite of testing strategies. Mocks themselves are not inherently bad, even though an over-reliance on them is.
- ramses0 3y ago
  An honest answer, tons of tests with mocks are really frustrating to find. Anything that simply “must” be mocked is better served by returning a “command pattern” response. eg: return new DoGood() || new DoBad(), and separately: TestableIsGoodOrBad() Specifically for your “error from a dependency” situation, I’m of the mind that a function consisting of: function foo() { xyz = PrepFoo(...) try { response = DoFoo(xyz) result = Process( response ) } except { HandleError() } } ...that’s the three lines of code that I’m spending my “not 100% code coverage” on. Each sub-function is 1000% testable without mocks, and the combination of functions is a “secure, logic-less composition”, with little test value. Basically, if you write your code that way you _never_ need mocks, and generally “mocks are dumb” because saying “Mock( DoRequest( FakeRequest() ), FakeResponse() )” is just wishful thinking compared to a properly factored codebase with _real_ non-mocked, significant, functional-only components.
  - scubbo 3y ago

PaulHoule 3y ago

Testing is at its best when you are testing functions or small subsystems that can be created and destroyed independently and be initialized with a known state. For instance the kind of things you write in an introductory programming class (somehow it goes over the heads of most students that the teachers are using automated testing.) It's certifiably insane to not test parsing code, whether or not it is "functional" (input text return a data structure) or not (function gets called to consume text, callbacks get called) UI code is tricky because usually the framework is calling into your code and your code is getting called in a shambolic way. Of course it problematizes testing but it problematic anyway. I have been looking at testing of a React application again and boy is it crazy, where you have to deal with asynchronous updates and don't really know how many times the app will update itself before you know it has "settled down" and it is OK to test the "output". Testing is bad, with tests that should take 0.04 sec taking more like 4 sec, but the user is being subjected to the same thing with lots of waiting, layout shifts, flashes of incorrect information, etc. And don't get me started with the typical "uncontrolled" component which is completely out of control since some application state updating updates the state managed by React, other ones call freaking jQuery (talk about spooky action at a distance!)

blueprint 3y ago

that's virtually precisely what the article is suggesting

KnobbleMcKnees 3y ago

Depends on the definition of easier. If easier is "it won't work with my particular flavour of framework or mocking" then yes, it's a bad idea. If it's inherently untestable for any setup, that's already a smell and needs some sort of attention. If the smell is accompanied by potential to cause catastrophic error or such, I'd rather lean in to changing the code to be more testable even if it feels like an anti-pattern.

mrkeen 3y ago

> architectural patterns that exist only/mainly to make unit testing easier? > try to encapsulate as much logic as I can into pure functions These are one and the same. The suggestion may as well have been "the IO monad is hard to test, so move as much code as you can out into pure functions"

regularfry 3y ago

Not really. It might start out being for the benefit of the tests but knock-on benefits turn up, usually sooner rather than later.

scubbo 3y ago

When I was a young nerd, I was a rabid fan of logic, objectivity, science. Anything which smacked of spirituality (or, worse, religion) was immediately dismissed as useless sentimental nonsense. I recognized that some people might _enjoy_ thinking about these fripperies, but I was convinced that any phenomenon which claimed an effect but did not explain the science-based interactions that mediated it (think astrology or religion) was hogwash. One day I was deathly bored on a school trip, and the only entertainment available was a book on Feng Shui. During the introduction, it advised the practice of imagining a dragon entering a living space through a door, coiling their way through and around the space, and then exiting. The following sentence stated that this was often a good heuristic for helping the human mind judge a space for sight lines, common travel paths, and avoiding clutter. My mind was blown. The dragon wasn't the point - the dragon was a way to hack the human brain (itself a messy, complex, poorly-understood mechanism) to a better outcome than it would get by a straightforward series of explicable evaluations. If the process demonstrably leads to better outcomes - if it is "useful" - then it can still provide value even if you can't explain _why_ it works[0] All of which is a roundabout way of getting to my point, which is: no, I don't get nervous about architectural patterns which _appear_ to only exist to make unit testing easier, because overwhelmingly I have found that "is this architecture easy to test?" is a good heuristic for "does this architecture have well-divided areas and layers of responsibility?". No matter your opinion on the value of tests _themselves_ (and there are very cogent arguments that an over-reliance on unit testing alone can lead to brittleness and false confidence), ceteris paribus making your architecture "more testable" will usually be a good thing in itself, even before any tests are written. [0] this attitude needs some nuance, of course, when it is actually important to explain how and why the process reaches conclusions. See: ML/LLMs/AGI/AI/whatever-the-current-initialism-is, and recourse in the face of disputed judgements or unforeseen effects.