4 ms·

I'm new to mocking (to serious unit testing, really), is mocking functions of 3rd-party libraries and/or web API's recommended? It sounds like a good idea, espe

by john2x 12y ago

I'm new to mocking (to serious unit testing, really), is mocking functions of 3rd-party libraries and/or web API's recommended? It sounds like a good idea, especially for functions that need to hit the network for a web API, but maybe I'm missing something?

michaelmior 12y ago
I would say it's generally a bad idea to mock 3rd party libraries. It's code that's not under your control and testing with a mock which may not match the actual behaviour could cause you trouble. In the case of network requests, something like VCR[1] helps a lot. [1] https://github.com/vcr/vcr https://github.com/vcr/vcr
- SideburnsOfDoom 12y ago
  I would disagree. If you want to unit-test your code that depends on those 3rd party libraries, then your choices are to: 1) Use the real thing. This can be slower and a lot less reliable. 2) Capture some typical output of the 3rd party api, and make a mock that emits it. Which technique is useful? Both are. But the second one gives you fast-running fine-grained repeatable unit tests so therefore is your first line of testing. So you discover in an integration test (#1) that the live api has a behaviour that the mock doesn't, maybe an occasional malformed response due to an error at the other end. Great, capture it and add it to the mock. Now you can reproduce it at will and write fast tests on how you handle it.
  - derefr 12y ago
    3) Define a Gateway interface (e.g. DataStoreGateway) that presents exactly the API you want to consume; write one implementation of it that speaks to your third-party API (e.g. ODBCDataStoreGateway), and another that doesn't do much (e.g. MemoryDataStoreGateway). Note that a trivial gateway-implementation, like MemoryDataStoreGateway, isn't a mock! It's a fully-functional component that works perfectly well from its consumers' perspectives. But, unlike the version that consumes a third-party component, it doesn't have nearly any failure modes that would confuse your tests. It just does what it does, simply, and lets your tests test what you're trying to test. And note that when you stop mocking, and limit yourself to switching out fully-fledged gateway implementations, "dependency injection" stops being this huge pain with Factories and heavily-parameterized initializers. Instead, you can just make your objects discover their collaborators through a service registry. Your unit tests just register the trivial implementations in the service registry on setup(). --- † Don't get a bad taste in your mouth thinking about Spring here. A simpler, healthier example is Erlang's "global" module (http://erlang.org/doc/man/global.html http://erlang.org/doc/man/global.html).
    - SideburnsOfDoom 12y ago
      > But, unlike the version that consumes a third-party component, it doesn't have nearly any failure modes that would confuse your tests. That's all fine until you need to test handling the third-party component's failure modes. > Note that a trivial gateway-implementation, like MemoryDataStoreGateway, isn't a mock! Probably not. Though it depends at which point you insert the "trivial" components. If you're using a trivial component to unit test a collaborating class that isn't at the edges of the system, then that's a mock and it's helpful. It's not the only helpful technique or even the usual one, but "mocks are never helpful, always avoid them" is silly thing to say. A mock store for one unit test is far more lightweight than a MemoryDataStoreGateway, and allows an easy way to simulate server errors, so sometimes it is a better thing to code. > This is what "dependency injection" is really about, actually: not inserting mock-objects ... but rather allowing you to plug in ... versions of your collaborators. It can be about both. They are not exclusive. > Instead, you can just make your objects discover their collaborators through a service registry You may be happy with every object reaching out to a global singleton. But I'm going to stay well away. Thanks but no thanks.
      - derefr 12y ago
        When you're testing the third-party component's failure modes, the tests you're writing are functional tests belonging to the nontrivial gateway implementation (e.g. ODBCDataStoreGateway). The important thing about the gateway interface is that it's an error boundary: you don't need to think about "server errors" when dealing with a DataStoreGateway; DataStoreGateways don't have server errors. The ODBC object the ODBCDataStoreGateway holds might hand it a server error, but it won't propagate that back to you. It'll just put out a plain† DataStoreGateway::InsertError or somesuch. † The exception should probably hold a copy of the implementation-exception that triggered it, but this is just for the sake of debugging the implementation. The client should never unwrap the internal exception.
        SideburnsOfDoom 12y ago
        > you don't need to think about "server errors" when dealing with a DataStoreGateway; DataStoreGateways don't have server errors. I'm not sure why you even make that distinction. More than once I've dealt with servers that occasionally time out and fail; (say once a week or so). I wish to test what I show on my UI under those conditions, and what part of the text and status codes returned from the remote server that represent (or whatever) its failure I will choose to display. Doing so without using a mock seems pointlessly obtuse and roundabout when I can make a mock to insert whatever error response I want at any interesting point in my code.
  - michaelmior 12y ago
    I was thinking more of the case where the 3rd party library does no network communication. If it still takes a long time to execute, that may be a concern. But if the output is complex, then mocking it may be more trouble than it's worth. For network requests to an API, your option #2 is exactly what I was suggesting.
    - SideburnsOfDoom 12y ago
      I'm not familiar with VCR, but it looks like it's a little similar to https://docs.angularjs.org/api/ngMock/service/$httpBackend https://docs.angularjs.org/api/ngMock/service/$httpBackend There's a lot of value in coarse-grained mocks like this for testing the system; I also find value in finer grained mocks for testing smaller components in isolation. But neither of these techniques is the only valid one. And neither of them forces you to write things upfront that you don't need later; which is what the original article seems to imply.
      - michaelmior 12y ago
        VCR seems similar to $httpBackend. The main difference is that it automatically records the HTTP response from the target server the first time tests are executed. Then it allows you to reuse that response in future test runs to save having to manually mock responses. Anyway, sounds like we're more or less agreeing :)
lmm 12y ago
For something like a web API I think most people would support mocking. If it's just a library it's a lot less clear-cut; my approach would be to use the real thing until it gets to be a problem (because it's making your tests slow, or because it doesn't expose the right interface, or because it's too hard to get it into the state that you want to test). Certainly use mocks when you need to, but the rest of the time there are arguments for both sides and you probably need to figure out what you prefer yourself.

by john2x 12y ago

michaelmior 12y ago

I would say it's generally a bad idea to mock 3rd party libraries. It's code that's not under your control and testing with a mock which may not match the actual behaviour could cause you trouble. In the case of network requests, something like VCR[1] helps a lot. [1] https://github.com/vcr/vcr https://github.com/vcr/vcr

SideburnsOfDoom 12y ago
I would disagree. If you want to unit-test your code that depends on those 3rd party libraries, then your choices are to: 1) Use the real thing. This can be slower and a lot less reliable. 2) Capture some typical output of the 3rd party api, and make a mock that emits it. Which technique is useful? Both are. But the second one gives you fast-running fine-grained repeatable unit tests so therefore is your first line of testing. So you discover in an integration test (#1) that the live api has a behaviour that the mock doesn't, maybe an occasional malformed response due to an error at the other end. Great, capture it and add it to the mock. Now you can reproduce it at will and write fast tests on how you handle it.
- derefr 12y ago
  3) Define a Gateway interface (e.g. DataStoreGateway) that presents exactly the API you want to consume; write one implementation of it that speaks to your third-party API (e.g. ODBCDataStoreGateway), and another that doesn't do much (e.g. MemoryDataStoreGateway). Note that a trivial gateway-implementation, like MemoryDataStoreGateway, isn't a mock! It's a fully-functional component that works perfectly well from its consumers' perspectives. But, unlike the version that consumes a third-party component, it doesn't have nearly any failure modes that would confuse your tests. It just does what it does, simply, and lets your tests test what you're trying to test. And note that when you stop mocking, and limit yourself to switching out fully-fledged gateway implementations, "dependency injection" stops being this huge pain with Factories and heavily-parameterized initializers. Instead, you can just make your objects discover their collaborators through a service registry. Your unit tests just register the trivial implementations in the service registry on setup(). --- † Don't get a bad taste in your mouth thinking about Spring here. A simpler, healthier example is Erlang's "global" module (http://erlang.org/doc/man/global.html http://erlang.org/doc/man/global.html).
  - SideburnsOfDoom 12y ago
    > But, unlike the version that consumes a third-party component, it doesn't have nearly any failure modes that would confuse your tests. That's all fine until you need to test handling the third-party component's failure modes. > Note that a trivial gateway-implementation, like MemoryDataStoreGateway, isn't a mock! Probably not. Though it depends at which point you insert the "trivial" components. If you're using a trivial component to unit test a collaborating class that isn't at the edges of the system, then that's a mock and it's helpful. It's not the only helpful technique or even the usual one, but "mocks are never helpful, always avoid them" is silly thing to say. A mock store for one unit test is far more lightweight than a MemoryDataStoreGateway, and allows an easy way to simulate server errors, so sometimes it is a better thing to code. > This is what "dependency injection" is really about, actually: not inserting mock-objects ... but rather allowing you to plug in ... versions of your collaborators. It can be about both. They are not exclusive. > Instead, you can just make your objects discover their collaborators through a service registry You may be happy with every object reaching out to a global singleton. But I'm going to stay well away. Thanks but no thanks.
    - derefr 12y ago
      When you're testing the third-party component's failure modes, the tests you're writing are functional tests belonging to the nontrivial gateway implementation (e.g. ODBCDataStoreGateway). The important thing about the gateway interface is that it's an error boundary: you don't need to think about "server errors" when dealing with a DataStoreGateway; DataStoreGateways don't have server errors. The ODBC object the ODBCDataStoreGateway holds might hand it a server error, but it won't propagate that back to you. It'll just put out a plain† DataStoreGateway::InsertError or somesuch. † The exception should probably hold a copy of the implementation-exception that triggered it, but this is just for the sake of debugging the implementation. The client should never unwrap the internal exception.
      - SideburnsOfDoom 12y ago
        > you don't need to think about "server errors" when dealing with a DataStoreGateway; DataStoreGateways don't have server errors. I'm not sure why you even make that distinction. More than once I've dealt with servers that occasionally time out and fail; (say once a week or so). I wish to test what I show on my UI under those conditions, and what part of the text and status codes returned from the remote server that represent (or whatever) its failure I will choose to display. Doing so without using a mock seems pointlessly obtuse and roundabout when I can make a mock to insert whatever error response I want at any interesting point in my code.
- michaelmior 12y ago
  I was thinking more of the case where the 3rd party library does no network communication. If it still takes a long time to execute, that may be a concern. But if the output is complex, then mocking it may be more trouble than it's worth. For network requests to an API, your option #2 is exactly what I was suggesting.
  - SideburnsOfDoom 12y ago
    I'm not familiar with VCR, but it looks like it's a little similar to https://docs.angularjs.org/api/ngMock/service/$httpBackend https://docs.angularjs.org/api/ngMock/service/$httpBackend There's a lot of value in coarse-grained mocks like this for testing the system; I also find value in finer grained mocks for testing smaller components in isolation. But neither of these techniques is the only valid one. And neither of them forces you to write things upfront that you don't need later; which is what the original article seems to imply.
    - michaelmior 12y ago
      VCR seems similar to $httpBackend. The main difference is that it automatically records the HTTP response from the target server the first time tests are executed. Then it allows you to reuse that response in future test runs to save having to manually mock responses. Anyway, sounds like we're more or less agreeing :)

lmm 12y ago

For something like a web API I think most people would support mocking. If it's just a library it's a lot less clear-cut; my approach would be to use the real thing until it gets to be a problem (because it's making your tests slow, or because it doesn't expose the right interface, or because it's too hard to get it into the state that you want to test). Certainly use mocks when you need to, but the rest of the time there are arguments for both sides and you probably need to figure out what you prefer yourself.