6 ms·

My second project at Google basically killed mocking for me and I've basically never done it since. Two things happened. The first was that I worked on a rewri

by cletus 9mo ago

My second project at Google basically killed mocking for me and I've basically never done it since. Two things happened. The first was that I worked on a rewrite of something (using GWT no less; it was more than a decade ago) and they decided to have a lot of test coverage and test requirements. That's fine but they way it was mandated and implemented, everybody just testing their service and DIed a bunch of mocks in. The results were entirely predictable. The entire system was incredibly brittle and a service that existed for only 8 weeks behaved like legacy code. You could spend half a day fixing mocks in tests for a 30 minute change just because you switched backend services, changed the order of calls or just ended up calling a given service more times than expected. It was horrible and a complete waste of time. Even the DI aspect of this was horrible because everything used Guice andd there wer emodules that installed modules that installed modules and modifying those to return mocks in a test environment was a massive effort that typically resulted in having a different environment (and injector) for test code vs production code so what are you actually testing? The second was that about this time the Java engineers at the company went on a massive boondoggle to decide on whether to use (and mandate) EasyMock vs Mockito. This was additionally a waste of time. Regardless of the relative merits of either, there's really not that much difference. At no point is it worth completely changing your mocking framework in existing code. Who knows how many engineering man-yars were wasted on this. Mocking encourages bad habits and a false sense of security. The solution is to have dummy versions of services and interfaces that have minimal correct behavior. So you might have a dummy Identity service that does simple lookups on an ID for permissions or metadata. If that's not what you're testing and you just need it to run a test, doing that with a mock is just wrong on so many levels. I've basically never used mocks since, so much so that I find anyone who is strongly in favor of mocks or has strong opinions on mocking frameworks to be a huge red flag.

LgWoodenBadger 9mo ago
“The solution is to have dummy versions of services and interfaces that have minimal correct behavior” If you aren’t doing this with mocks then you’re doing mocks wrong.
- saghm 9mo ago
  I think the argument they're making is that once you have this, you already have an easy way to test things that doesn't require bringing in an entire framework.
- cowsandmilk 9mo ago
  In part, you’re right, but there’s a practical difference between mocking and a good dummy version of a service. Take DynamoDB local as an example: you can insert items and they persist, delete items, delete tables, etc. Or in the Ruby on Rails world, one often would use SQLite as a local database for tests even if using a different DB in production. Going further, there’s the whole test containers movement of having a real version of your dependency present for your tests. Of course, in a microservices world, bringing up the whole network of dependencies is extremely complicated and likely not warranted.
  - sfn42 9mo ago
    I use test containers and similar methods to test against a "real" db, but I also use mocks. For example to mock the response of a third party api, can't very well spin that up in a test container. Nother example is simply time stamps. Can't really test time related stuff without mocking a timestamp provider. It is a hassle a lot of the time, but I see it as a necessary evil.
    - supriyo-biswas 9mo ago
      You can use a library like [1] to mock out a real HTTP server with responses. [1] https://www.mock-server.com/ https://www.mock-server.com/
- pdpi 9mo ago
  I'd go a bit farther — "mock" is basically the name for those dummy versions. That said, there is a massive difference between writing mocks and using a mocking library like Mockito — just like there is a difference between using dependency injection and building your application around a DI framework.
ebiester 9mo ago
Mockito, in every case I had to use it, was a last resort because a third party library didnt lend itself to mocking, or you were bringing legacy code under test and using it long enough to refactor it out. It should never be the first tool. But when you need it, it’s very useful.
throwaway7783 9mo ago
I'm not sure I understand. "The solution is to have dummy versions of services and interfaces that have minimal correct behavior". That's mocks in a nutshell. What other way would you use mocks?
- phanimahesh 9mo ago
  There are different kinds of mocks. Check function XYZ is called, return abc when XYZ is called etc are the bad kind that people were bit badly by. The good kind are a minimally correct fake implementation that doesn't really need any mocking library to build. Tests should not be brittle and rigidly restate the order of function calls and expected responses. That's a whole lot of ceremony that doesn't really add confidence in the code because it does not catch many classes of errors, and requires pointless updates to match the implementation 1-1 everytime it is updated. It's effectively just writing the implementation twice, if you squint at it a bit.
  - throwaway7783 9mo ago
    Why is check if XYZ is called with return value ABC bad, as long as XYZ is an interface method? Why is a minimally correct fake any better than a mock in this context? Mocks are not really about order of calls unless you are talking about different return values on different invocations. A fake simply moves the cheese to setting up data correctly, as your tests and logic change. Not a huge difference either way.
    - rcxdude 9mo ago
      The point is to test against a model of the dependency, not just the expected behavour of the code under test. If you just write a mock that exactly corresponds to the test that you're running, you're not testing the interface with the underlying system, you're just running the (probably already perfectly understandable) unit through a rote set of steps, and that's both harder to maintain and less useful than testing against a model of the underlying system. (And IMO this should only be done for heavyweight or difficult to precisely control components of the system where necessary to improve test runtime or expand the range of testable conditions. Always prefer testing as close to the real system as reasonably practical)
zem 9mo ago
my time at google likewise led me to the conclusion that fakes were better than mocks in pretty much every case (though I was working in c++ and python, not java). edit: of course google was an unusual case because you had access to all the source code. I daresay there are cases where only a mock will work because you can't satisfy type signatures with a fake.
yearolinuxdsktp 9mo ago
Heavy mocks usage comes from dogmatically following the flawed “most tests should be unit tests” prescription of the “testing pyramid,” as well as a strict adherence to not testing more than one class at a time. This necessitates heavy mocking, which is fragile, terrible to refactor, leads to lots of low-value tests. Sadly, AI these days will generate tons of those unit tests in the hands of those who don’t know better. All in all leading to the same false sense of security and killing development speed.
- fatso83 9mo ago
  I get what you are saying, but you can have your cake and eat it too. Fast, comprehensive tests that cover most of your codebase. Test through the domain, employ Fakes at the boundaries. https://asgaut.com/use-of-fakes-for-domain-driven-design-and-fast-feedback-loop/ https://asgaut.com/use-of-fakes-for-domain-driven-design-and...
rurban 9mo ago
I dont use dummy services and I dont use mocking. I'm writing simulators to test things for HW or big services which are not available for testing. Simulators need to be complete for their use cases or they cannot be used for testing.

by cletus 9mo ago

LgWoodenBadger 9mo ago

“The solution is to have dummy versions of services and interfaces that have minimal correct behavior” If you aren’t doing this with mocks then you’re doing mocks wrong.

saghm 9mo ago
I think the argument they're making is that once you have this, you already have an easy way to test things that doesn't require bringing in an entire framework.
cowsandmilk 9mo ago
In part, you’re right, but there’s a practical difference between mocking and a good dummy version of a service. Take DynamoDB local as an example: you can insert items and they persist, delete items, delete tables, etc. Or in the Ruby on Rails world, one often would use SQLite as a local database for tests even if using a different DB in production. Going further, there’s the whole test containers movement of having a real version of your dependency present for your tests. Of course, in a microservices world, bringing up the whole network of dependencies is extremely complicated and likely not warranted.
- sfn42 9mo ago
  I use test containers and similar methods to test against a "real" db, but I also use mocks. For example to mock the response of a third party api, can't very well spin that up in a test container. Nother example is simply time stamps. Can't really test time related stuff without mocking a timestamp provider. It is a hassle a lot of the time, but I see it as a necessary evil.
  - supriyo-biswas 9mo ago
    You can use a library like [1] to mock out a real HTTP server with responses. [1] https://www.mock-server.com/ https://www.mock-server.com/
pdpi 9mo ago
I'd go a bit farther — "mock" is basically the name for those dummy versions. That said, there is a massive difference between writing mocks and using a mocking library like Mockito — just like there is a difference between using dependency injection and building your application around a DI framework.

ebiester 9mo ago

Mockito, in every case I had to use it, was a last resort because a third party library didnt lend itself to mocking, or you were bringing legacy code under test and using it long enough to refactor it out. It should never be the first tool. But when you need it, it’s very useful.

throwaway7783 9mo ago

I'm not sure I understand. "The solution is to have dummy versions of services and interfaces that have minimal correct behavior". That's mocks in a nutshell. What other way would you use mocks?

phanimahesh 9mo ago
There are different kinds of mocks. Check function XYZ is called, return abc when XYZ is called etc are the bad kind that people were bit badly by. The good kind are a minimally correct fake implementation that doesn't really need any mocking library to build. Tests should not be brittle and rigidly restate the order of function calls and expected responses. That's a whole lot of ceremony that doesn't really add confidence in the code because it does not catch many classes of errors, and requires pointless updates to match the implementation 1-1 everytime it is updated. It's effectively just writing the implementation twice, if you squint at it a bit.
- throwaway7783 9mo ago
  Why is check if XYZ is called with return value ABC bad, as long as XYZ is an interface method? Why is a minimally correct fake any better than a mock in this context? Mocks are not really about order of calls unless you are talking about different return values on different invocations. A fake simply moves the cheese to setting up data correctly, as your tests and logic change. Not a huge difference either way.
  - rcxdude 9mo ago
    The point is to test against a model of the dependency, not just the expected behavour of the code under test. If you just write a mock that exactly corresponds to the test that you're running, you're not testing the interface with the underlying system, you're just running the (probably already perfectly understandable) unit through a rote set of steps, and that's both harder to maintain and less useful than testing against a model of the underlying system. (And IMO this should only be done for heavyweight or difficult to precisely control components of the system where necessary to improve test runtime or expand the range of testable conditions. Always prefer testing as close to the real system as reasonably practical)

zem 9mo ago

my time at google likewise led me to the conclusion that fakes were better than mocks in pretty much every case (though I was working in c++ and python, not java). edit: of course google was an unusual case because you had access to all the source code. I daresay there are cases where only a mock will work because you can't satisfy type signatures with a fake.

yearolinuxdsktp 9mo ago

Heavy mocks usage comes from dogmatically following the flawed “most tests should be unit tests” prescription of the “testing pyramid,” as well as a strict adherence to not testing more than one class at a time. This necessitates heavy mocking, which is fragile, terrible to refactor, leads to lots of low-value tests. Sadly, AI these days will generate tons of those unit tests in the hands of those who don’t know better. All in all leading to the same false sense of security and killing development speed.

fatso83 9mo ago
I get what you are saying, but you can have your cake and eat it too. Fast, comprehensive tests that cover most of your codebase. Test through the domain, employ Fakes at the boundaries. https://asgaut.com/use-of-fakes-for-domain-driven-design-and-fast-feedback-loop/ https://asgaut.com/use-of-fakes-for-domain-driven-design-and...

rurban 9mo ago

I dont use dummy services and I dont use mocking. I'm writing simulators to test things for HW or big services which are not available for testing. Simulators need to be complete for their use cases or they cannot be used for testing.