4 ms·

> I gain nothing by mocking [database queries and transactions] them out. What about the fact that you will need to spin up, migrate and possibly seed a databa

> I gain nothing by mocking [database queries and transactions] them out. What about the fact that you will need to spin up, migrate and possibly seed a database in your CI pipeline? What about the toll this will take on the execution speed of your test suite? Additionally, consider that you also need to test the behavior of your system when the query fails, and using a mock implementation that always throws an exception is a trivial and reliable way of achieving this. > That said if the interaction between your system and it's deps is sufficiently unreliable that you need mocks for test reliability, how are you going to have any confidence in the system's production behavior? Sometimes your codebase depends on external services which are flaky for reasons beyond your control, just the way it be sometimes. Mocks are useful to ensure the system behaves a certain way when everything goes right as well as when everything goes wrong. Ultimately the article raises many good points about avoiding mocks if it can be helped, but don't forget a test that only tests the happy path of your system is not very useful. Mock an error in that dependency you expect to always work and understand what would happen, make the necessary provisions.

asdfasgasdgasdg 5y ago
> What about the fact that you will need to spin up, migrate and possibly seed a database in your CI pipeline? The database system I use can be configured to start up reasonably quickly, and can be configured to operate on memfiles to reduce io pressure on the CI system. In fact, testing against the full scale local database is the only supported methodology for this particular rdbms. > Sometimes your codebase depends on external services which are flaky for reasons beyond your control, just the way it be sometimes. No doubt no doubt. As I mentioned, mocks or fakes might be necessary in a condition like this. > Ultimately the article raises many good points about avoiding mocks if it can be helped, but don't forget a test that only tests the happy path of your system is not very useful. My team uses interception and error injection for this case. We still have the real backend, but requests can be forced to fail either before or after executing on the backend.
- lobstrosity420 5y ago
  > My team uses interception and error injection for this case. We still have the real backend, but requests can be forced to fail either before or after executing on the backend. Really cool. What technologies are you using to achieve this? We've also had to tackle stuff like this before, but I'm not sure of the optimal way of doing it.
  - asdfasgasdgasdg 5y ago
    To be honest, this part is handled manually via a test value injector. There's a global map of names to "adjusters", which are either values or callbacks that are allowed to manipulate a value. If there's a particular request we want to fail after making, we do something like this: // System under test code: auto status = DoRequest(); TestManipulate("after_request", status); if (IsError(status)) { // Handle errors. } // Test code SetTestValue("after_request", MyErrorStatus()); RunSystemUnderTest(); The module that handles test value manipulation is typically disabled by a global variable, and lives in the cold section under FDO and opt compilation, so it costs essentially nothing. It cannot be enabled in release builds due to the code that enables it being compiled out. We find in many cases this isn't really necessary, as the error handling code in most cases just bubbles up the error, and so it is not particularly interesting to test. There are a few cases where we do this where the error handling is more complex than "return same error to user" or "retry after delay." This type of thing can't always solve the problem, but it's often good enough to get things done.

asdfasgasdgasdg 5y ago

> What about the fact that you will need to spin up, migrate and possibly seed a database in your CI pipeline? The database system I use can be configured to start up reasonably quickly, and can be configured to operate on memfiles to reduce io pressure on the CI system. In fact, testing against the full scale local database is the only supported methodology for this particular rdbms. > Sometimes your codebase depends on external services which are flaky for reasons beyond your control, just the way it be sometimes. No doubt no doubt. As I mentioned, mocks or fakes might be necessary in a condition like this. > Ultimately the article raises many good points about avoiding mocks if it can be helped, but don't forget a test that only tests the happy path of your system is not very useful. My team uses interception and error injection for this case. We still have the real backend, but requests can be forced to fail either before or after executing on the backend.

lobstrosity420 5y ago
> My team uses interception and error injection for this case. We still have the real backend, but requests can be forced to fail either before or after executing on the backend. Really cool. What technologies are you using to achieve this? We've also had to tackle stuff like this before, but I'm not sure of the optimal way of doing it.
- asdfasgasdgasdg 5y ago
  To be honest, this part is handled manually via a test value injector. There's a global map of names to "adjusters", which are either values or callbacks that are allowed to manipulate a value. If there's a particular request we want to fail after making, we do something like this: // System under test code: auto status = DoRequest(); TestManipulate("after_request", status); if (IsError(status)) { // Handle errors. } // Test code SetTestValue("after_request", MyErrorStatus()); RunSystemUnderTest(); The module that handles test value manipulation is typically disabled by a global variable, and lives in the cold section under FDO and opt compilation, so it costs essentially nothing. It cannot be enabled in release builds due to the code that enables it being compiled out. We find in many cases this isn't really necessary, as the error handling code in most cases just bubbles up the error, and so it is not particularly interesting to test. There are a few cases where we do this where the error handling is more complex than "return same error to user" or "retry after delay." This type of thing can't always solve the problem, but it's often good enough to get things done.