3 ms·

I think the original ideas of mocks (if you go and read Growing Object-Oriented Software, Guided by Tests) had some merit: In that style of TDD, mocks are used

I think the original ideas of mocks (if you go and read Growing Object-Oriented Software, Guided by Tests) had some merit: In that style of TDD, mocks are used to discover (hopefully somewhat stable) interfaces between components, and in theory, it fits with the idea that OOP is about "objects sending messages to each other". I can believe that it's possible to write good systems with this kind of approach. Unfortunately, in practice, mocks are rarely used like that and most "OOP" designs have horrible boundaries and are really not much about message passing anymore. That leads to brittle mocks where you constantly have to change tests when you change implementation details. I have also gravitated away from classic OOP and much more towards the "functional core, imperative shell" concept as outlined in the article (although it's difficult to keep this pattern throughout a codebase, especially if you have team members). In such a system you really rarely need mocks. Agreed that fakes, when you have them, are nicer than mocks, especially when the system to be faked has a large API (i.e. use a redis fake, instead of checking the exact commands you send to redis). However, for some outside systems, writing a fake can be a lot of effort. In such a case, I think it's totally valid to write a "gateway class" that isn't unit tested (you can cover it with integration tests instead) which exposes a nice API (e.g. "storeFile(...)") and then to use mocks of that class in other tests.

arcticbull 5y ago
In test cases where you extensively involve mocks, more often than not in my experience, you end up testing that your mocks do the thing you told them to.
- mikepurvis 5y ago
  Yeah, particularly the case with a lot of "glue" type code that's really just passing stuff back and forth and not really making any decisions with it. I've always struggled to feel that mock-based testing in this scenario was anything other than busy-work.
- pstuart 5y ago
  Yes, but the coverage numbers look great!
  - arcticbull 5y ago
    Don’t even get me started on that as a metric lol. I will say going from 0-20% coverage is a lot more valuable than going from 70-90%
    - smaudet 5y ago
      The proper metric is tests/feature - most test code bases are so bad you can't even pick out what feature is being tested, due to a blind adherence to 'code coverage' It is the equivalent of spot checking a road 100 times for cracks but forgetting to make sure you put up stop signs at the intersections - I honestly believe if you are a tech lead or boss advocating this you should be fired on the spot for encouraging dangerous and malignant practices...
rictic 5y ago
most of the bugs are in the joints of the system, not in the components it's much easier to write modules that are internally consistent, much harder to be globally consistent across modules. mocks ensure you only test for internal consistency
- barrkel 5y ago
  Mocks don't test the joints, which you have just suggested are where the bugs are. Fakes get you a lot further without needing more comprehensive integration testing. Fakes simulate dependencies - so you can test the joints - but can be tested themselves, and most importantly can have conformance tests that validate that the fake acts like what it is faking.
  - smaudet 5y ago
    Fakes also make integration testing easier too - if you want to focus on say, making sure your payments always make it into the database correctly, integration testing across the medley becomes (relatively) trivial, even automatable.
  - dustymcp 5y ago
    I could mock all day but it would not be any help since the api's we use are brittle its much more efficient to fail in a graceful way where you can recover than plastering the codebase with mocks of happy paths
  - rictic 5y ago
    since both comments in response read it the same way, I suppose I was unclear I was arguing against mocks, and for only using fakes when you really have to, for hermeticity and performance
- delusional 5y ago
  Wouldn't the integration (or global consistency as you call it) of objects be exactly what was worth testing in that's the hard part? Generally objects are simple enough that I can reason about them in my head. That's the whole point of encapsulating the state after all. That means tests are really less critical, since a thorough inspection should do. Between components it's MUCH harder to get any sort of coverage in your internal model, so tests that can be repeated become more useful. I want the tests to fail if the software doesn't work. Not if some object doesn't do what it says it will in a way that doesn't matter to the system. The hard part is exactly what I want tests to cover.

arcticbull 5y ago

In test cases where you extensively involve mocks, more often than not in my experience, you end up testing that your mocks do the thing you told them to.

mikepurvis 5y ago
Yeah, particularly the case with a lot of "glue" type code that's really just passing stuff back and forth and not really making any decisions with it. I've always struggled to feel that mock-based testing in this scenario was anything other than busy-work.
pstuart 5y ago
Yes, but the coverage numbers look great!
- arcticbull 5y ago
  Don’t even get me started on that as a metric lol. I will say going from 0-20% coverage is a lot more valuable than going from 70-90%
  - smaudet 5y ago
    The proper metric is tests/feature - most test code bases are so bad you can't even pick out what feature is being tested, due to a blind adherence to 'code coverage' It is the equivalent of spot checking a road 100 times for cracks but forgetting to make sure you put up stop signs at the intersections - I honestly believe if you are a tech lead or boss advocating this you should be fired on the spot for encouraging dangerous and malignant practices...

rictic 5y ago

most of the bugs are in the joints of the system, not in the components it's much easier to write modules that are internally consistent, much harder to be globally consistent across modules. mocks ensure you only test for internal consistency

barrkel 5y ago
Mocks don't test the joints, which you have just suggested are where the bugs are. Fakes get you a lot further without needing more comprehensive integration testing. Fakes simulate dependencies - so you can test the joints - but can be tested themselves, and most importantly can have conformance tests that validate that the fake acts like what it is faking.
- smaudet 5y ago
  Fakes also make integration testing easier too - if you want to focus on say, making sure your payments always make it into the database correctly, integration testing across the medley becomes (relatively) trivial, even automatable.
- dustymcp 5y ago
  I could mock all day but it would not be any help since the api's we use are brittle its much more efficient to fail in a graceful way where you can recover than plastering the codebase with mocks of happy paths
- rictic 5y ago
  since both comments in response read it the same way, I suppose I was unclear I was arguing against mocks, and for only using fakes when you really have to, for hermeticity and performance
delusional 5y ago
Wouldn't the integration (or global consistency as you call it) of objects be exactly what was worth testing in that's the hard part? Generally objects are simple enough that I can reason about them in my head. That's the whole point of encapsulating the state after all. That means tests are really less critical, since a thorough inspection should do. Between components it's MUCH harder to get any sort of coverage in your internal model, so tests that can be repeated become more useful. I want the tests to fail if the software doesn't work. Not if some object doesn't do what it says it will in a way that doesn't matter to the system. The hard part is exactly what I want tests to cover.