7 ms·

Even more important, IMO, regardless of whether you're mocking or faking, is to not mock or fake objects that you own. Only mock/fake truly external dependencie

Even more important, IMO, regardless of whether you're mocking or faking, is to not mock or fake objects that you own. Only mock/fake truly external dependencies. The nice property you get with this is true tests of integration and interoperability between your modules, that are actually calling into each other. Only at the leaf nodes is anything mocked or faked, and that's when it leaves your ownership.

viraptor 6y ago
You're choosing some place in between the integration and unit testing where your test will live. But this choice will not make sense for everyone. If your project includes its own persistence functions, you'll likely want a test to get a "CouldntWriteException" immediately from a mock rather than go through all the layers (that you own) to get an IO error (you don't own the fs dependency) and all layers back. It's a tiny difference for a single test, but once you get to hundreds, it will be a difference between devs getting immediate feedback and "going for coffee, my code is testing" which really impacts productivity.
ratww 6y ago
Yes! Mocking internal dependencies also has the side effect of ossifying the internal interface between the two objects, which is never as well-designed as the external ones IME. Also, in dynamic languages, you might even have tests that don’t break when you change (or sometimes even delete) the dependency because the mock was correct. That leads to a false sense of confidence.
paranoidrobot 6y ago
Perhaps I misunderstand you. Say I'm writing tests for lets say ShoppingCart. ShoppingCart depends on an implementation of ICartDiscountCalculator (CartDiscountCalculator) CartDiscountCalculator depends on a number of implementations of ICartDiscountStrategy (ChristmasCartDiscountStrategy, VIPCustomerDiscountStrategy, BulkPurchaseDiscountStrategy, etc) Each of those, in turn has their own dependencies, perhaps webservice calls, db calls, accessing caches, etc. The way I would test ShoppingCart would be to mock out ICartDiscountCalculator, and provide known responses. Are you suggesting instead that you build up the dependency graph - i.e the actual implementation of the discount calculator, the strategies, etc - and only mock when it goes to external code (eg the DB calls)? This would seem to make testing very very difficult - since each test needs to know implementation details of everything it depends on.
- virgilp 6y ago
  Um, no. The test needs to know the expected system behaviour, given that dependencies are properly wired & behave as expected. You can argue that "it no longer tests just the ShoppingCart in isolation" and that would be true; but whether that's a downside or an upside is up for debate IMO (and more often than not it really is an upside). [edit] In other words, you don't test that "if discount is 10%, adding item to shopping cart increases cart total by 0.9×<price>"; you do test that "if user X adds item Y to shopping cart, and X has a 10% discount code, then shopping cart price increase by 0.9×<price>". This does seem like a headache, but it has benefits - in that it tests actual product scenarios, and you can now easily add variations like "if discount code only applies for books, and item I added isn't a book, then shopping cart total increases by <price>".
  - paranoidrobot 6y ago
    I would argue then that this isn't Unit Testing, it's some other kind of testing. One of the problems I'm dealing with at the moment is exactly this problem - someone's gone and written a whole ton of tests which involve spinning up pretty much everything short of the DB, and now writing a small test that just tests the behaviour of one class is exceedingly difficult.
    - virgilp 6y ago
      If what you "spin up" is internal classes/functionality, not external system, I don't understand the problem? The article doesn't argue "this is how you do unit testing", it argues "this is how you do better testing". It's highly debatable whether unit tests are preferable to larger, "component tests".
      - paranoidrobot 6y ago
        The problem is that it makes testing every upstream component more difficult to write for, and more fragile. To take the shopping cart example I gave, if I write it the way that I understand you're suggesting, I would have to consider the behavior of all of the ICartDiscountStrategy implementations. For instance, I've written tests for CartDiscountCalculator and ShoppingCart that have 10 different items in the cart from Acme Corp Manufacturing. Some time later, I implement a new AcmeCorpDiscountStrategy which results in a different discount being returned. Now my tests for CartDiscountCalculator and ShoppingCart need to be modified. I also need to potentially modify everything else up the dependency graph from CartDiscountCalculator - so my internal API that is used by customer service, an API used by mobiles, and the front end website. Using Mocks/Fakes to isolate the ShoppingCart, it doesn't matter what the strategies are, we only care about how it interacts with ICartDiscountCalculator. That interaction might change, maybe ShoppingCart uses the CartDiscountCalculator in a different way yeah, that'll break ShoppingCart Unit Tests - but it's that method only, and you're already in there modifying ShoppingCart anyway.
hinkley 6y ago
Mocks for your own code, fakes for external dependencies but in both cases only where you can’t rework your most of your code to consume the outputs of dependencies instead of soliciting them. Fixtures trump mocks and fakes. Having multiple tests fail for the same reason is a waste of effort, either now or in the future (when new features are added or the constraints otherwise change). Sometimes you have to depend on transitive properties to keep your test sizes down. If a function received the data, and the function has been tested thoroughly, why test it again while testing the callers?
- srtjstjsj 6y ago
  Why would you mock your own code, when you have your own real code right there?
  - hinkley 6y ago
    Because that’s an integration test and I’m writing unit tests, which scale much higher.
  - mypalmike 6y ago
    As parent explained, if you use real objects, you end up testing the dependencies of those objects, and transitively the dependencies thereof, etc. For unit tests, the goal is to test the unit, not its dependencies. In practice, using real objects has many downsides including complex setup functions and a need to make changes to many layers of complex tests in order to implement even small code changes.
  - SAI_Peregrinus 6y ago
    In my case, I'm an embedded dev. So some of my own code is doing memory-mapped I/O to hardware, inline microcontroller ASM, or other stuff that can't physically happen on the system running the unit tests. So mocks (or fakes, it doesn't matter much here) are necessary to avoid running hardware-specific code on the wrong hardware. Of course I try to make those fakes at as low a level as possible, but there's still a substantial part of my code that can't be unit tested, and needs test programs in the actual firmware to check it.
mabbo 6y ago
While I 100% agree with you on this point, you should take a few minutes and read this: https://medium.com/@adrianbooth/test-driven-development-wars-detroit-vs-london-classicist-vs-mockist-9956c78ae95f https://medium.com/@adrianbooth/test-driven-development-wars... It's a fantastic comparison of the Detroit school of testing (what you and I believe) vs the London school, which is simply a different mental model about code and testing and mocks especially. Framing the differences this way make it clearer why some people might disagree with us, and it's not because they're stupid or wrong- they just have a different school of thought on the matter. This doesn't excuse, however, the people who are doing a little bit of both and they don't know why, and they're getting the benefits of neither side of that coin.
- mcqueenjordan 6y ago
  Nice framework for thinking about it, thanks. Totally agree. For me, the biggest benefit is that I want to be able to refactor the internals of my code and have the tests (which map to business requirements, not implementation details) still pass. It's also because a lot of the code I work with is inherently concurrent, and in my estimation, you're getting a lot more test coverage with the detroit model because a lot of your threading cases are simply covered if you use real objects and test it black-box style.
- srtjstjsj 6y ago
  What if I like outside-in, black-box testing Because I like to test everything, not just little parts I isolate?
  - mabbo 6y ago
    Then you follow the classicist, Detroit-school of testing. Me too. But if you read the article, you can see that this isn't the only school of thought.
Griffon26 6y ago
This only works on small code bases. With anything large you're going to need to mock or fake your own code if you don't want to spend countless unnecessary hours running your tests and finding root causes of failed tests. Yes, integration tests are important too, but you'll want to minimize the number of scenarios you still need to test at that level.

viraptor 6y ago

You're choosing some place in between the integration and unit testing where your test will live. But this choice will not make sense for everyone. If your project includes its own persistence functions, you'll likely want a test to get a "CouldntWriteException" immediately from a mock rather than go through all the layers (that you own) to get an IO error (you don't own the fs dependency) and all layers back. It's a tiny difference for a single test, but once you get to hundreds, it will be a difference between devs getting immediate feedback and "going for coffee, my code is testing" which really impacts productivity.

ratww 6y ago

Yes! Mocking internal dependencies also has the side effect of ossifying the internal interface between the two objects, which is never as well-designed as the external ones IME. Also, in dynamic languages, you might even have tests that don’t break when you change (or sometimes even delete) the dependency because the mock was correct. That leads to a false sense of confidence.

paranoidrobot 6y ago

Perhaps I misunderstand you. Say I'm writing tests for lets say ShoppingCart. ShoppingCart depends on an implementation of ICartDiscountCalculator (CartDiscountCalculator) CartDiscountCalculator depends on a number of implementations of ICartDiscountStrategy (ChristmasCartDiscountStrategy, VIPCustomerDiscountStrategy, BulkPurchaseDiscountStrategy, etc) Each of those, in turn has their own dependencies, perhaps webservice calls, db calls, accessing caches, etc. The way I would test ShoppingCart would be to mock out ICartDiscountCalculator, and provide known responses. Are you suggesting instead that you build up the dependency graph - i.e the actual implementation of the discount calculator, the strategies, etc - and only mock when it goes to external code (eg the DB calls)? This would seem to make testing very very difficult - since each test needs to know implementation details of everything it depends on.

virgilp 6y ago
Um, no. The test needs to know the expected system behaviour, given that dependencies are properly wired & behave as expected. You can argue that "it no longer tests just the ShoppingCart in isolation" and that would be true; but whether that's a downside or an upside is up for debate IMO (and more often than not it really is an upside). [edit] In other words, you don't test that "if discount is 10%, adding item to shopping cart increases cart total by 0.9×<price>"; you do test that "if user X adds item Y to shopping cart, and X has a 10% discount code, then shopping cart price increase by 0.9×<price>". This does seem like a headache, but it has benefits - in that it tests actual product scenarios, and you can now easily add variations like "if discount code only applies for books, and item I added isn't a book, then shopping cart total increases by <price>".
- paranoidrobot 6y ago
  I would argue then that this isn't Unit Testing, it's some other kind of testing. One of the problems I'm dealing with at the moment is exactly this problem - someone's gone and written a whole ton of tests which involve spinning up pretty much everything short of the DB, and now writing a small test that just tests the behaviour of one class is exceedingly difficult.
  - virgilp 6y ago
    If what you "spin up" is internal classes/functionality, not external system, I don't understand the problem? The article doesn't argue "this is how you do unit testing", it argues "this is how you do better testing". It's highly debatable whether unit tests are preferable to larger, "component tests".
    - paranoidrobot 6y ago
      The problem is that it makes testing every upstream component more difficult to write for, and more fragile. To take the shopping cart example I gave, if I write it the way that I understand you're suggesting, I would have to consider the behavior of all of the ICartDiscountStrategy implementations. For instance, I've written tests for CartDiscountCalculator and ShoppingCart that have 10 different items in the cart from Acme Corp Manufacturing. Some time later, I implement a new AcmeCorpDiscountStrategy which results in a different discount being returned. Now my tests for CartDiscountCalculator and ShoppingCart need to be modified. I also need to potentially modify everything else up the dependency graph from CartDiscountCalculator - so my internal API that is used by customer service, an API used by mobiles, and the front end website. Using Mocks/Fakes to isolate the ShoppingCart, it doesn't matter what the strategies are, we only care about how it interacts with ICartDiscountCalculator. That interaction might change, maybe ShoppingCart uses the CartDiscountCalculator in a different way yeah, that'll break ShoppingCart Unit Tests - but it's that method only, and you're already in there modifying ShoppingCart anyway.

hinkley 6y ago

Mocks for your own code, fakes for external dependencies but in both cases only where you can’t rework your most of your code to consume the outputs of dependencies instead of soliciting them. Fixtures trump mocks and fakes. Having multiple tests fail for the same reason is a waste of effort, either now or in the future (when new features are added or the constraints otherwise change). Sometimes you have to depend on transitive properties to keep your test sizes down. If a function received the data, and the function has been tested thoroughly, why test it again while testing the callers?

srtjstjsj 6y ago
Why would you mock your own code, when you have your own real code right there?
- hinkley 6y ago
  Because that’s an integration test and I’m writing unit tests, which scale much higher.
- mypalmike 6y ago
  As parent explained, if you use real objects, you end up testing the dependencies of those objects, and transitively the dependencies thereof, etc. For unit tests, the goal is to test the unit, not its dependencies. In practice, using real objects has many downsides including complex setup functions and a need to make changes to many layers of complex tests in order to implement even small code changes.
- SAI_Peregrinus 6y ago
  In my case, I'm an embedded dev. So some of my own code is doing memory-mapped I/O to hardware, inline microcontroller ASM, or other stuff that can't physically happen on the system running the unit tests. So mocks (or fakes, it doesn't matter much here) are necessary to avoid running hardware-specific code on the wrong hardware. Of course I try to make those fakes at as low a level as possible, but there's still a substantial part of my code that can't be unit tested, and needs test programs in the actual firmware to check it.

mabbo 6y ago

While I 100% agree with you on this point, you should take a few minutes and read this: https://medium.com/@adrianbooth/test-driven-development-wars-detroit-vs-london-classicist-vs-mockist-9956c78ae95f https://medium.com/@adrianbooth/test-driven-development-wars... It's a fantastic comparison of the Detroit school of testing (what you and I believe) vs the London school, which is simply a different mental model about code and testing and mocks especially. Framing the differences this way make it clearer why some people might disagree with us, and it's not because they're stupid or wrong- they just have a different school of thought on the matter. This doesn't excuse, however, the people who are doing a little bit of both and they don't know why, and they're getting the benefits of neither side of that coin.

mcqueenjordan 6y ago
Nice framework for thinking about it, thanks. Totally agree. For me, the biggest benefit is that I want to be able to refactor the internals of my code and have the tests (which map to business requirements, not implementation details) still pass. It's also because a lot of the code I work with is inherently concurrent, and in my estimation, you're getting a lot more test coverage with the detroit model because a lot of your threading cases are simply covered if you use real objects and test it black-box style.
srtjstjsj 6y ago
What if I like outside-in, black-box testing Because I like to test everything, not just little parts I isolate?
- mabbo 6y ago
  Then you follow the classicist, Detroit-school of testing. Me too. But if you read the article, you can see that this isn't the only school of thought.

Griffon26 6y ago

This only works on small code bases. With anything large you're going to need to mock or fake your own code if you don't want to spend countless unnecessary hours running your tests and finding root causes of failed tests. Yes, integration tests are important too, but you'll want to minimize the number of scenarios you still need to test at that level.