5 ms·

Could someone elaborate on why “mocking is a smell”? Is it opposed to running the whole system with real interactions? I find it a nice shortcut, but true as w

Could someone elaborate on why “mocking is a smell”? Is it opposed to running the whole system with real interactions? I find it a nice shortcut, but true as well than the mocking behavior can be non existent in the real code path. Still, I reach out to mocking libraries often. Is that a bad practices and why ?

deleted 3y ago
[deleted]
frankdejonge 3y ago
Mocking, when it is a large part of your test setup, makes your test focus on the blueprint of the implementation rather than the outcome of a unit of behaviour. This causes the tests to break relatively soon when the implementation changes. When a test does not use mocking and rather treats the code as a sealed box, the tests are more stable. Stable tests yield better guarantees for stable outcomes even when the underlying code is (radically) changed/refactored. Using mocking frameworks usually makes refactoring harder, as changing the underlying code immediately invalidates the test. Changing the tests and code in unison makes changes less safe.
- frankdejonge 3y ago
  I must add that it's mostly mocking libraries that auto-generate mocks that are the culprit here. They're quick to introduce, but once introduced make it harder to change the implementation as the implementation logic is now replicated in N instances of tests. More surface area, more friction for change. Many mocking libraries also allow you to fixate behaviour that is incompatible with the actual implementation, especially in dynamically types languages.
geewee 3y ago
I think it depends heavily on what you're mocking and what the code you're testing looks like. I think Martin Fowler has a nice dichotomy where he splits tests up into solitary (generally using mocks for deps) and sociable (generally using real code for dependencies): https://martinfowler.com/bliki/UnitTest.html https://martinfowler.com/bliki/UnitTest.html
geokon 3y ago
I don't do TDD - so this is my outsider's understanding The issue is that if you need to mock then your design is not really properly decoupled. The mock bakes in assumptions about your program state - which may or may not hold up in the real world. This ends up creating a sort of invisible coupling to whatever will be creating/updating the state in the real world The terminology is a bit fuzzy b/c the author describes his unit tests as using mocks - but to me that's not a unit test - that's an integration test.. At the end of the day you do need some coupling and some integration tests are inevitable - but the idea is that TDD pushes you to minimize that Would love to hear anyone's corrections :)
ryandv 3y ago
One issue with mocks is that you have to ensure that the way they are mocked accurately represents the actual behaviour of the thing being mocked, in production. Otherwise your mocks may end up driving the system to a state that is never actually encountered in production, while the unit test still passes. When you run an integration test or a real build of the application, the actual system itself will fail to behave as expected. The mock implementations can diverge from the actual production behaviour.
lambda_dn 3y ago
Does your user run your software with stuff mocked out? The reason for testing is getting ahead of issues before the user finds them.
- drewcoo 3y ago
  > The reason for testing is getting ahead of issues before the user finds them. The customer always uses software in some unexpected way, but we don't defer all "testing" to just watching the customer interact. Testing is to give developers the fastest, most meaningful, most actionable feedback about their code changes so that they can make better decisions. Customer issues are not the only things we're worried about. And software seems to be usable and make plenty of money even when users see something screwy once in a while.
- dopidopHN 3y ago
  I think I had my response : I use mock mostly for integration tests. And those are not in the scope of this article
Tainnor 3y ago
Mocks have their uses, but overzealous overmocking really means the death of a test suite. At this point, tests become more of a nuisance than anything helpful (especially in a statically typed language where you know that if your code compiles the pieces at least sort of have to fit together). Mocks (or related techniques such as stubs, fakes) are useful when you have to interact with an outside system (e.g. send emails), for testing boundary conditions that are hard to replicate (what happens if an error is thrown), for highly generic code or, potentially, for really major architectural boundaries in your code base (but I feel that most people overestimate the confidence they have that they've determined the correct boundaries). For anything else, I'd prefer to isolate pure code from code with side effects as much as possible so that the former can be unit tested without any dependencies (even if one class/module/whatever calls another, it doesn't have to be mocked out), and the latter is mostly just wiring that can be tested through a combination of integration testing and maybe some occasional use of a mock here and there. Unfortunately not many code bases are structured that way, so you'll have to pick your battles and maybe live with more (brittle) mocks than you'd like - or with slower and more flakey (integration) tests that you'd like.
- devoutsalsa 3y ago
  I have an extremely hard time reading & benefitting from tests that make use of excessive mocks. Some apps I've worked on recently have mocked out so many things, large parts of the code base don't actually get hit by the actual tests. It's super annoying. To be honest I can't even make sense of it most of the time.
  - dopidopHN 3y ago
    I felt that. But also : how do you deal with testing interaction with an external API. Or code that you control but require a large effort to instanciate? ( you want to test f but need to standup a,b,c,d and e. I find it tempting to mock e and call it a day.
    - devoutsalsa 3y ago
      I think it's fine to mock out a call to an external service (e.g. your app making calls to AWS S3). But when you mock out too many calls to code in your own app, that's brittle in my opinion. For example, say your testing the "Foo.new.().foo()" method, and you mock out calls to "Bar.new().bar()". You could make a breaking change in "Bar.new().bar()", say you go from returning "bar" to "BAR". The test in "Foo.new.().foo()" would keep passing while testing the wrong value. Now if you do that all over your application, you can have a nicely factored test suite that is completely detached from reality.
drewcoo 3y ago
There's a smell in the code but it's not the mocks. I've seen people have problems with mocks when they have poorly designed code that demands Byzantine test setups. The "bad mocks" smell usually means the production code is due for a refactor. (So let's hope you have behavioral tests and not implementation-only tests to help everyone refactor.)

deleted 3y ago

[deleted]

frankdejonge 3y ago

Mocking, when it is a large part of your test setup, makes your test focus on the blueprint of the implementation rather than the outcome of a unit of behaviour. This causes the tests to break relatively soon when the implementation changes. When a test does not use mocking and rather treats the code as a sealed box, the tests are more stable. Stable tests yield better guarantees for stable outcomes even when the underlying code is (radically) changed/refactored. Using mocking frameworks usually makes refactoring harder, as changing the underlying code immediately invalidates the test. Changing the tests and code in unison makes changes less safe.

frankdejonge 3y ago
I must add that it's mostly mocking libraries that auto-generate mocks that are the culprit here. They're quick to introduce, but once introduced make it harder to change the implementation as the implementation logic is now replicated in N instances of tests. More surface area, more friction for change. Many mocking libraries also allow you to fixate behaviour that is incompatible with the actual implementation, especially in dynamically types languages.

geewee 3y ago

I think it depends heavily on what you're mocking and what the code you're testing looks like. I think Martin Fowler has a nice dichotomy where he splits tests up into solitary (generally using mocks for deps) and sociable (generally using real code for dependencies): https://martinfowler.com/bliki/UnitTest.html https://martinfowler.com/bliki/UnitTest.html

geokon 3y ago

I don't do TDD - so this is my outsider's understanding The issue is that if you need to mock then your design is not really properly decoupled. The mock bakes in assumptions about your program state - which may or may not hold up in the real world. This ends up creating a sort of invisible coupling to whatever will be creating/updating the state in the real world The terminology is a bit fuzzy b/c the author describes his unit tests as using mocks - but to me that's not a unit test - that's an integration test.. At the end of the day you do need some coupling and some integration tests are inevitable - but the idea is that TDD pushes you to minimize that Would love to hear anyone's corrections :)

ryandv 3y ago

One issue with mocks is that you have to ensure that the way they are mocked accurately represents the actual behaviour of the thing being mocked, in production. Otherwise your mocks may end up driving the system to a state that is never actually encountered in production, while the unit test still passes. When you run an integration test or a real build of the application, the actual system itself will fail to behave as expected. The mock implementations can diverge from the actual production behaviour.

lambda_dn 3y ago

Does your user run your software with stuff mocked out? The reason for testing is getting ahead of issues before the user finds them.

drewcoo 3y ago
> The reason for testing is getting ahead of issues before the user finds them. The customer always uses software in some unexpected way, but we don't defer all "testing" to just watching the customer interact. Testing is to give developers the fastest, most meaningful, most actionable feedback about their code changes so that they can make better decisions. Customer issues are not the only things we're worried about. And software seems to be usable and make plenty of money even when users see something screwy once in a while.
dopidopHN 3y ago
I think I had my response : I use mock mostly for integration tests. And those are not in the scope of this article

Tainnor 3y ago

Mocks have their uses, but overzealous overmocking really means the death of a test suite. At this point, tests become more of a nuisance than anything helpful (especially in a statically typed language where you know that if your code compiles the pieces at least sort of have to fit together). Mocks (or related techniques such as stubs, fakes) are useful when you have to interact with an outside system (e.g. send emails), for testing boundary conditions that are hard to replicate (what happens if an error is thrown), for highly generic code or, potentially, for really major architectural boundaries in your code base (but I feel that most people overestimate the confidence they have that they've determined the correct boundaries). For anything else, I'd prefer to isolate pure code from code with side effects as much as possible so that the former can be unit tested without any dependencies (even if one class/module/whatever calls another, it doesn't have to be mocked out), and the latter is mostly just wiring that can be tested through a combination of integration testing and maybe some occasional use of a mock here and there. Unfortunately not many code bases are structured that way, so you'll have to pick your battles and maybe live with more (brittle) mocks than you'd like - or with slower and more flakey (integration) tests that you'd like.

devoutsalsa 3y ago
I have an extremely hard time reading & benefitting from tests that make use of excessive mocks. Some apps I've worked on recently have mocked out so many things, large parts of the code base don't actually get hit by the actual tests. It's super annoying. To be honest I can't even make sense of it most of the time.
- dopidopHN 3y ago
  I felt that. But also : how do you deal with testing interaction with an external API. Or code that you control but require a large effort to instanciate? ( you want to test f but need to standup a,b,c,d and e. I find it tempting to mock e and call it a day.
  - devoutsalsa 3y ago
    I think it's fine to mock out a call to an external service (e.g. your app making calls to AWS S3). But when you mock out too many calls to code in your own app, that's brittle in my opinion. For example, say your testing the "Foo.new.().foo()" method, and you mock out calls to "Bar.new().bar()". You could make a breaking change in "Bar.new().bar()", say you go from returning "bar" to "BAR". The test in "Foo.new.().foo()" would keep passing while testing the wrong value. Now if you do that all over your application, you can have a nicely factored test suite that is completely detached from reality.

drewcoo 3y ago

There's a smell in the code but it's not the mocks. I've seen people have problems with mocks when they have poorly designed code that demands Byzantine test setups. The "bad mocks" smell usually means the production code is due for a refactor. (So let's hope you have behavioral tests and not implementation-only tests to help everyone refactor.)