6 ms·

Mockito 2.1.0

67 points by raphw 10y ago

geraldtrue 10y ago
Thank you Rafael, Tim, Marcin #1 & Marcin #2! Great team and great project!
brianwawok 10y ago
And in most cases you will regret the web of mocks you have made :)
- lohankin 10y ago
  Fragile web of mocks. You will find yourself spending 90% of time fixing this web.
  - lmm 10y ago
    Mockito makes it a lot worse by not having strict mocks be first-class. EasyMock is much more usable in this regard - you're usually going to have to mock every called method anyway, so it's much better to have the framework tell you when an unmocked method is called than have it silently return null and leave you guessing.
    - uw_rob 10y ago
      I believe Mockito can verify for you that unmocked stubs haven't been called. [0] For people looking for an alternative take on why it is good practice to let mocked objects have default stubs for unmocked methods, see this article [1]. This article is referenced in the mocktio documentation. [0] http://site.mockito.org/mockito/docs/current/org/mockito/Mockito.html#25 http://site.mockito.org/mockito/docs/current/org/mockito/Moc... [1] https://monkeyisland.pl/2008/07/12/should-i-worry-about-the-unexpected/ https://monkeyisland.pl/2008/07/12/should-i-worry-about-the-...
      - david-given 10y ago
        In my experience, Mockito never actually gets to that point, because my program has called an unmocked stub, tried to use the return result, and NPEd in an obscure and hard-to-debug manner before verification call.
        uw_rob 10y ago
        In my experience with Mockito and unit tests I haven't ever ran into this. However it seems like a very valid concern, and I could definitely see it causing problems. I wonder if it is possible to use reflection and have mockito throw an exception if a stub method is called.
        lmm 10y ago
        Not really. You can set a custom default answer and you can write a method that creates mocks with that answer, but there are enough rough edges that it ends up amounting to writing your own test framework. Much easier to just use EasyMock.
      - lmm 10y ago
        > I believe Mockito can verify for you that unmocked stubs haven't been called. [0] As the other reply says, that doesn't help because it never gets that far. To my mind the article you link has the wrong premise, because I do make my data tests that way - I assert that the data is exactly what I expected. Surprise changes to data would be bad, as would surprise changes to state.
- uw_rob 10y ago
  Do you have any examples of where mocking has let you down? I found creating unit tests with mocks and DI extremely easy and satisfying. At least for me, they also provide a pretty convincing argument that the tested code is correct. Also, they are basically required if you want to do any testing with external services. I'm not sure how else you could create unit tests for DBs etc.
  - brianwawok 10y ago
    Yes. Every project I have ever used with a web of mocks, especially around final things made refactoring near impossible. At some point the codebase got so brittle, it fell apart. I find it funny that in most discussions around mocking, a database is the given example. 100% of the time when I need a database for "unit tests", I use a database. Not a mock. I don't want to know that when I do select blar from flar I get back the value 7, and hardcode it. I want to actually load the data and see it happen. Because if I don't, how do I know that my code will actually work in production? Sometimes that involves sqllite (easy but not preferred because I don't use it in production). Sometimes it just involves running a local database that gets built up for a unit test run. Sometimes something like docker, sometimes not. Sometimes a shared "dev" database. Lots of options. But literally every project I have been involved with that used a REAL database instead of mocking it out, has worked better and been more stable and easier to change. Now for actual external services.. say you call some Netflix API. Yes don't call that for real in a unit test. But you can do that with either: a) Basic mocking, because you created a very simple API. You don't just call some random driver directly with your code, right? You have an interface in front of it. Then you can mock it with no final or static override crap going on. b) If your API is clean enough, you can just code up a very simple dummy implementation. Lets you set the return value for methods, then the next call it gives them back. I don't feel that strongly between A and B above. But I feel very very very strongly about people mocking databases or, even worse, every random call between classes in their own codebase, or someone elses codebase.
    - wlamartin 10y ago
      > I want to actually load the data and see it happen. Because if I don't, how do I know that my code will actually work in production? This would fall under the purview of integration/acceptance testing?
- mabbo 10y ago
  What do you propose instead of mocks? Custom test-versions of every class you need to control the behaviour of? I've had lots of challenges with some frameworks (EasyMock makes it easy to make a mess) but Mockito is so nice to use, I can't imagine writing Java code without it for my basic unit tests.
  - ndr 10y ago
    You don't need a "Custom test-version" of every class. If you're using dependency injection you can create a "Custom test-version" only for the things that sit at the border of your domain and use real code for everything but those. The classic example is again storage, suppose you use Redis for storing key-value strings. Create an interface that has the store and get method and have two implementations of that, one that actually injects a Redis client and one that injects nothing and saves everything in a map. Now you can test the real implementation with mocks and a real Redis if you want, but all the other tests can run on the in-memory implementation just fine and super fast. No need to mock anything. I've found mocks useful to exercise interactions (and Mockito is awesome), but once you have a fake for the types at the boundary of your dependency graph you're pretty much sorted.
- oldmanjay 10y ago
  A web of mocks is a sign that your design is not clean enough to be easily unit testable. The frustration it causes is a whisper that you need to refactor things, and that refactoring has always ended up making my software far more robust and understandable. In my experience, most people just throw up their hands and integration test instead of aggressively working the design into shape.
carrja99 10y ago
After journeying through many languages since my long time with Java that ended in 2011 I have to say I have yet found a mock object framework in other languages that match the expressiveness that Mockito did. Python's mock has the least memorable API I have ever worked with. :-(
- dom0 10y ago
  This is because using Mocks in a language like Python is a bit awkward and unnatural in itself... if you look at the way tests are usually written, there is just little need for it. That's also the reason why only recently unittest.mock was added in the first place.
  - samcal 10y ago
    Can you elaborate on this? What is the correct way to test external services that may have side effects in Python?
    - bebop 10y ago
      I think maybe because there is no type checking in Python. This makes it more trivial to have objects that quack like a duck, without the need to explicitly create a class that conforms to an interface.
      - LgWoodenBadger 10y ago
        FWIW, you don't do that with Mockito either, in Java land. Mockito will happily use the existing type definition of your dependency, provided that the class declaration is not final (i.e. can't be extended).
        raphw 10y ago
        This is actually not a requirement anymore. Mockito 2 can mock final types and methods.
    - 10y ago
ThomasRooney 10y ago
I spent my undergraduate thesis on automating the production of mock objects and unit tests using program tracing via the JDK's debug APIs [0]. I'd thought it might be useful to have a tool which, requiring no user input, would output when expected contracts of behaviour (in the context of two objects interactions with each other at production) had changed. I wrote my tool to generate JUnit/Mockito tests. It was almost bulletproof in producing working tests for arbitrary Java bytecode (see the link for examples). At the end of it I was utterly disillusioned that the vast majority of tests with mocks were actually useful to an end developer. It felt too fragile and made refactoring work much more of a chore. Since then I've tested via constructing independent slices of implementation and pushing sample input/output in. In my experience, it has being vastly easier to maintain such a test suite. I've meant to go back and open source my tool for a while, but found other parts of life getting in the way. If anyone's interested, would love to chat about it (might help me find motivation to continue the work). [0] http://www.doc.ic.ac.uk/teaching/distinguished-projects/2015/t.rooney.pdf http://www.doc.ic.ac.uk/teaching/distinguished-projects/2015...
- deleted 10y ago
  [deleted]
- larsthorup 10y ago
  I have recently ventured into auto-mocking, see http://zealake.com/2015/01/05/unit-test-your-service-integration-layer/ http://zealake.com/2015/01/05/unit-test-your-service-integra..., which allows me to run super fast unit-tests, that are really doing end-to-end testing, giving me 100's of end-to-end tests per second. Is that the same line of thought as your "independent slices of implementation and pushing sample input/output in"?
  - ThomasRooney 10y ago
    That's a really interesting idea. It looks like you're capturing and proxying the latest backend request/response patterns used in its test suite into a frontend test requirement so that you can run them both independently, and thus much faster, whilst still keeping most of the assurances of an end-to-end test. I'm going to have to think about applying that approach to my own projects. What I actually meant is slightly more literal. The backend of my latest web project is 100% web client invocable functions in AWS Lambda. The frontend is a static site built in redux with sagas. The view is a dumb layer mapping state to html and dispatching redux actions which are picked up by the sagas. Each saga only invokes one lambda function, though does send/receive signals with other sagas. Responses break down into state changes which map to a single react view. By taking this path all implementation can be visualised as discrete slices of logic. Moving signals `up` and `down` the slice requires request/response patterns between the components. To test, I run a script against the developer staging area which sends API calls to each function and diffs the result with expected response back. It is fragile to any change which affects API responses, but is dumb enough that it gives me confidence to flip the switch turning staging environment into production. Its super easy to maintain and update, though I suspect it'd be difficult to apply the approach to more complex application domains (this is a virtual telephone number service).
compay 10y ago
This project is the standard case I use when telling people not to choose project names in a language you do not speak. It means "booger" in Spanish.
- euyyn 10y ago
  A little one, though.
- DanitaBaires 10y ago
  "Moquito"
jaypaulynice 10y ago
There's been a lot of discussions on Mockists vs Classicists. See Martin Fowler's article: http://martinfowler.com/articles/mocksArentStubs.html http://martinfowler.com/articles/mocksArentStubs.html. In my experience, projects that use mocks extensively usually have the most bugs. If you also have integration tests then mocks are ok. But then it's more code to maintain. Given a choice I would always do "integration test". Fact is you can test a single class/method, reality is that no class/method exists alone.
- lightbendover 10y ago
  We recently implemented an extensive test suite that can be run with either mocks for service dependencies or in a live configuration, depending on config settings. Before, we simply had integration tests that relied on very specific data across disparate services and running it in CI was a nightmare (to the point that we would typically only run the full suite in a QA environment shortly before a production deploy). Whereas our integration testing has been responsible for catching hundreds of obscure bugs over the past year, most of the bugs found by our mocks have been bugs in the mocks themselves and I expect that issue to compound due to the brittle nature of mocks in general. Mocks make every change some multiple more expensive in both initial cost and maintenance.

Mockito 2.1.0

67 points by raphw 10y ago

geraldtrue 10y ago

Thank you Rafael, Tim, Marcin #1 & Marcin #2! Great team and great project!

brianwawok 10y ago

And in most cases you will regret the web of mocks you have made :)

lohankin 10y ago
Fragile web of mocks. You will find yourself spending 90% of time fixing this web.
- lmm 10y ago
  Mockito makes it a lot worse by not having strict mocks be first-class. EasyMock is much more usable in this regard - you're usually going to have to mock every called method anyway, so it's much better to have the framework tell you when an unmocked method is called than have it silently return null and leave you guessing.
  - uw_rob 10y ago
    I believe Mockito can verify for you that unmocked stubs haven't been called. [0] For people looking for an alternative take on why it is good practice to let mocked objects have default stubs for unmocked methods, see this article [1]. This article is referenced in the mocktio documentation. [0] http://site.mockito.org/mockito/docs/current/org/mockito/Mockito.html#25 http://site.mockito.org/mockito/docs/current/org/mockito/Moc... [1] https://monkeyisland.pl/2008/07/12/should-i-worry-about-the-unexpected/ https://monkeyisland.pl/2008/07/12/should-i-worry-about-the-...
    - david-given 10y ago
      In my experience, Mockito never actually gets to that point, because my program has called an unmocked stub, tried to use the return result, and NPEd in an obscure and hard-to-debug manner before verification call.
      - uw_rob 10y ago
        In my experience with Mockito and unit tests I haven't ever ran into this. However it seems like a very valid concern, and I could definitely see it causing problems. I wonder if it is possible to use reflection and have mockito throw an exception if a stub method is called.
        lmm 10y ago
        Not really. You can set a custom default answer and you can write a method that creates mocks with that answer, but there are enough rough edges that it ends up amounting to writing your own test framework. Much easier to just use EasyMock.
    - lmm 10y ago
      > I believe Mockito can verify for you that unmocked stubs haven't been called. [0] As the other reply says, that doesn't help because it never gets that far. To my mind the article you link has the wrong premise, because I do make my data tests that way - I assert that the data is exactly what I expected. Surprise changes to data would be bad, as would surprise changes to state.
uw_rob 10y ago
Do you have any examples of where mocking has let you down? I found creating unit tests with mocks and DI extremely easy and satisfying. At least for me, they also provide a pretty convincing argument that the tested code is correct. Also, they are basically required if you want to do any testing with external services. I'm not sure how else you could create unit tests for DBs etc.
- brianwawok 10y ago
  Yes. Every project I have ever used with a web of mocks, especially around final things made refactoring near impossible. At some point the codebase got so brittle, it fell apart. I find it funny that in most discussions around mocking, a database is the given example. 100% of the time when I need a database for "unit tests", I use a database. Not a mock. I don't want to know that when I do select blar from flar I get back the value 7, and hardcode it. I want to actually load the data and see it happen. Because if I don't, how do I know that my code will actually work in production? Sometimes that involves sqllite (easy but not preferred because I don't use it in production). Sometimes it just involves running a local database that gets built up for a unit test run. Sometimes something like docker, sometimes not. Sometimes a shared "dev" database. Lots of options. But literally every project I have been involved with that used a REAL database instead of mocking it out, has worked better and been more stable and easier to change. Now for actual external services.. say you call some Netflix API. Yes don't call that for real in a unit test. But you can do that with either: a) Basic mocking, because you created a very simple API. You don't just call some random driver directly with your code, right? You have an interface in front of it. Then you can mock it with no final or static override crap going on. b) If your API is clean enough, you can just code up a very simple dummy implementation. Lets you set the return value for methods, then the next call it gives them back. I don't feel that strongly between A and B above. But I feel very very very strongly about people mocking databases or, even worse, every random call between classes in their own codebase, or someone elses codebase.
  - wlamartin 10y ago
    > I want to actually load the data and see it happen. Because if I don't, how do I know that my code will actually work in production? This would fall under the purview of integration/acceptance testing?
mabbo 10y ago
What do you propose instead of mocks? Custom test-versions of every class you need to control the behaviour of? I've had lots of challenges with some frameworks (EasyMock makes it easy to make a mess) but Mockito is so nice to use, I can't imagine writing Java code without it for my basic unit tests.
- ndr 10y ago
  You don't need a "Custom test-version" of every class. If you're using dependency injection you can create a "Custom test-version" only for the things that sit at the border of your domain and use real code for everything but those. The classic example is again storage, suppose you use Redis for storing key-value strings. Create an interface that has the store and get method and have two implementations of that, one that actually injects a Redis client and one that injects nothing and saves everything in a map. Now you can test the real implementation with mocks and a real Redis if you want, but all the other tests can run on the in-memory implementation just fine and super fast. No need to mock anything. I've found mocks useful to exercise interactions (and Mockito is awesome), but once you have a fake for the types at the boundary of your dependency graph you're pretty much sorted.
oldmanjay 10y ago
A web of mocks is a sign that your design is not clean enough to be easily unit testable. The frustration it causes is a whisper that you need to refactor things, and that refactoring has always ended up making my software far more robust and understandable. In my experience, most people just throw up their hands and integration test instead of aggressively working the design into shape.

carrja99 10y ago

After journeying through many languages since my long time with Java that ended in 2011 I have to say I have yet found a mock object framework in other languages that match the expressiveness that Mockito did. Python's mock has the least memorable API I have ever worked with. :-(

dom0 10y ago
This is because using Mocks in a language like Python is a bit awkward and unnatural in itself... if you look at the way tests are usually written, there is just little need for it. That's also the reason why only recently unittest.mock was added in the first place.
- samcal 10y ago
  Can you elaborate on this? What is the correct way to test external services that may have side effects in Python?
  - bebop 10y ago
    I think maybe because there is no type checking in Python. This makes it more trivial to have objects that quack like a duck, without the need to explicitly create a class that conforms to an interface.
    - LgWoodenBadger 10y ago
      FWIW, you don't do that with Mockito either, in Java land. Mockito will happily use the existing type definition of your dependency, provided that the class declaration is not final (i.e. can't be extended).
      - raphw 10y ago
        This is actually not a requirement anymore. Mockito 2 can mock final types and methods.
  - 10y ago

ThomasRooney 10y ago

I spent my undergraduate thesis on automating the production of mock objects and unit tests using program tracing via the JDK's debug APIs [0]. I'd thought it might be useful to have a tool which, requiring no user input, would output when expected contracts of behaviour (in the context of two objects interactions with each other at production) had changed. I wrote my tool to generate JUnit/Mockito tests. It was almost bulletproof in producing working tests for arbitrary Java bytecode (see the link for examples). At the end of it I was utterly disillusioned that the vast majority of tests with mocks were actually useful to an end developer. It felt too fragile and made refactoring work much more of a chore. Since then I've tested via constructing independent slices of implementation and pushing sample input/output in. In my experience, it has being vastly easier to maintain such a test suite. I've meant to go back and open source my tool for a while, but found other parts of life getting in the way. If anyone's interested, would love to chat about it (might help me find motivation to continue the work). [0] http://www.doc.ic.ac.uk/teaching/distinguished-projects/2015/t.rooney.pdf http://www.doc.ic.ac.uk/teaching/distinguished-projects/2015...

deleted 10y ago
[deleted]
larsthorup 10y ago
I have recently ventured into auto-mocking, see http://zealake.com/2015/01/05/unit-test-your-service-integration-layer/ http://zealake.com/2015/01/05/unit-test-your-service-integra..., which allows me to run super fast unit-tests, that are really doing end-to-end testing, giving me 100's of end-to-end tests per second. Is that the same line of thought as your "independent slices of implementation and pushing sample input/output in"?
- ThomasRooney 10y ago
  That's a really interesting idea. It looks like you're capturing and proxying the latest backend request/response patterns used in its test suite into a frontend test requirement so that you can run them both independently, and thus much faster, whilst still keeping most of the assurances of an end-to-end test. I'm going to have to think about applying that approach to my own projects. What I actually meant is slightly more literal. The backend of my latest web project is 100% web client invocable functions in AWS Lambda. The frontend is a static site built in redux with sagas. The view is a dumb layer mapping state to html and dispatching redux actions which are picked up by the sagas. Each saga only invokes one lambda function, though does send/receive signals with other sagas. Responses break down into state changes which map to a single react view. By taking this path all implementation can be visualised as discrete slices of logic. Moving signals `up` and `down` the slice requires request/response patterns between the components. To test, I run a script against the developer staging area which sends API calls to each function and diffs the result with expected response back. It is fragile to any change which affects API responses, but is dumb enough that it gives me confidence to flip the switch turning staging environment into production. Its super easy to maintain and update, though I suspect it'd be difficult to apply the approach to more complex application domains (this is a virtual telephone number service).

compay 10y ago

This project is the standard case I use when telling people not to choose project names in a language you do not speak. It means "booger" in Spanish.

euyyn 10y ago
A little one, though.
DanitaBaires 10y ago
"Moquito"

jaypaulynice 10y ago

There's been a lot of discussions on Mockists vs Classicists. See Martin Fowler's article: http://martinfowler.com/articles/mocksArentStubs.html http://martinfowler.com/articles/mocksArentStubs.html. In my experience, projects that use mocks extensively usually have the most bugs. If you also have integration tests then mocks are ok. But then it's more code to maintain. Given a choice I would always do "integration test". Fact is you can test a single class/method, reality is that no class/method exists alone.

lightbendover 10y ago
We recently implemented an extensive test suite that can be run with either mocks for service dependencies or in a live configuration, depending on config settings. Before, we simply had integration tests that relied on very specific data across disparate services and running it in CI was a nightmare (to the point that we would typically only run the full suite in a QA environment shortly before a production deploy). Whereas our integration testing has been responsible for catching hundreds of obscure bugs over the past year, most of the bugs found by our mocks have been bugs in the mocks themselves and I expect that issue to compound due to the brittle nature of mocks in general. Mocks make every change some multiple more expensive in both initial cost and maintenance.