4 ms·

> Do whatever that will increase confidence in code and testability. Maybe for some strict definition of "testability", but a lot of people in the Python world

> Do whatever that will increase confidence in code and testability. Maybe for some strict definition of "testability", but a lot of people in the Python world use patching to circumvent anything that might reasonably be called "testability" in the pursuit of "confidence" and some puritanical desire to avoid changing the code base to make it testable. This makes everything a complete mess for anyone who comes along later to extend the code or increase test coverage, as the tests now depend on the private internals of the code and the patching logic is magical and unintuitive. If your "testability" remark precludes patching, then I'm more bought in, but I do worry that some will read this and use it to argue for patching. That said, experience tells me that mocking delivers a lot less confidence than using the real deal, although there are some cases (e.g., network errors) that effectively require mocks _somewhere_.

icedchai 4y ago
Definitely this. I ran into this at a previous company (Python environment) where every test basically relied on mucking around with the internals through patches. Any sort of integration test that relied on calling an internal service was difficult or impossible, so it had to be mocked. External services were always mocked. The code base was a pain to work with. At at an earlier company, we had very little patching or mocking. We spun up all the internal services when running the tests. Internal calls were never mocked. External calls were sometimes mocked (if test accounts / API keys were not convenient.) We preferred testing the "real thing." Tests ran slower but in general I had much more confidence tests actually did something useful.
- throwaway894345 4y ago
  I like to distinguish between mocks and fakes. Mocks just return whatever you tell them to return for that test, but a fake is a simplified in-memory implementation (fakes aren't useful for testing error conditions, however, so you may need a few tests which actually do mock if you want to get more complete coverage). Most of the time for third-party services, I'll just fake the service client for unit tests. For example, if I'm writing a BookStoreApp that has a BookStore dependency (where BookStore is an interface), the concrete implementation for which is a PostgresBookStore, then I'll write my PostgresBookStore client library with unit tests that include a real postgres connection but then my BookStoreApp tests will use a FakeBookStore (which is probably a thin wrapper around a dict, but which behaves like a BookStore should modulo persistence). Finally, I'll have considerably fewer API tests which actually do stand up the complete backend service (including a PostgresBookStore). This way I get a high degree of confidence and also speed of test execution--it's also nice not to have to stand up the entire world just to run one test (in particular, it's nice not to have to debug unrelated connection/auth issues).

icedchai 4y ago

Definitely this. I ran into this at a previous company (Python environment) where every test basically relied on mucking around with the internals through patches. Any sort of integration test that relied on calling an internal service was difficult or impossible, so it had to be mocked. External services were always mocked. The code base was a pain to work with. At at an earlier company, we had very little patching or mocking. We spun up all the internal services when running the tests. Internal calls were never mocked. External calls were sometimes mocked (if test accounts / API keys were not convenient.) We preferred testing the "real thing." Tests ran slower but in general I had much more confidence tests actually did something useful.

throwaway894345 4y ago
I like to distinguish between mocks and fakes. Mocks just return whatever you tell them to return for that test, but a fake is a simplified in-memory implementation (fakes aren't useful for testing error conditions, however, so you may need a few tests which actually do mock if you want to get more complete coverage). Most of the time for third-party services, I'll just fake the service client for unit tests. For example, if I'm writing a BookStoreApp that has a BookStore dependency (where BookStore is an interface), the concrete implementation for which is a PostgresBookStore, then I'll write my PostgresBookStore client library with unit tests that include a real postgres connection but then my BookStoreApp tests will use a FakeBookStore (which is probably a thin wrapper around a dict, but which behaves like a BookStore should modulo persistence). Finally, I'll have considerably fewer API tests which actually do stand up the complete backend service (including a PostgresBookStore). This way I get a high degree of confidence and also speed of test execution--it's also nice not to have to stand up the entire world just to run one test (in particular, it's nice not to have to debug unrelated connection/auth issues).