4 ms·

>Yes, integration tests do tend to cover integration with third party libraries and even entire products (such as databases). But even in integration tests, the

>Yes, integration tests do tend to cover integration with third party libraries and even entire products (such as databases). But even in integration tests, the third party code is incidental. It definitely shouldnt be. At least, not if you want your tests to tell you when upgrading a 3rd party dependency will break something. >For instance, when created integration tests for databases, it's very common to use an embedded or in-memory DB I worked on a project that did this once and they very quickly got blocked writing a test by the in memory database not supporting a feature postgres had. Result? "Oh well, I guess we dont write an automated test for that." Manual QA's problem now. That's slow. Realism matters. Sacrificing realism for speed often means you will get neither. >you're not supposed use mocks (like a mocked clock) I do do this and it usually works well. Why am I wrong?

ivan_gammel 3y ago
>>I worked on a project that did this once and they very quickly got blocked writing a test by the in memory database not supporting a feature postgres had. >Realism matters It is often just a small subset of tests which have to be very expensive to build, maintain and run because of an external dependency that cannot be mocked. Mocks work for 95% of use cases and should be used even if it is not 100%.
- MoreQARespect 3y ago
  I never found that to be true and moreover, these scenarios with nonstandard features tended to be the scenarios I was most worried about breaking. There is also the problem of the in memory database behaving differently when given the same SQL so your test might make it look like everything works while a bug crops up in production. Realism matters.
  - randomdata 3y ago
    > There is also the problem of the in memory database behaving differently when given the same SQL so your test might make it look like everything works while a bug crops up in production. You've horribly messed up the architecture of your application if that is a problem. (Or you've misunderstood previous comments)
    - MoreQARespect 3y ago
      Postgres and sqlite/in memory dbs just behave differently to each other sometimes. Knowing this fact doesn't mean you've messed up your architecture it means that you have some understanding of how these databases work. Realism matters.
      - anarazel 3y ago
        I'd say that in-memory/not-in-memory isn't the big difference - it's whether your database is in-process or not. Even with just a database running on the same node, but in a different process, connected to via unix socket, the context switches alone lead to very different performance characteristics. Actually going over the network obviously changes more. It's very easy to miss antipatterns like N+1 queries when you test on sqlite but run on a shared database in prod.
        ivan_gammel 3y ago
        This is irrelevant for unit tests. Performance testing does not make any sense in build environment, you need to do it in an environment close to production and that’s completely different test automation scope.
        anarazel 3y ago
        You can see this stuff often even in test workloads. But even if you disregard that kind of issue, you still have stuff like needing to integrate networked database connections into e.g. event loops, which you don't really need to do for things like sqlite.

ivan_gammel 3y ago

>>I worked on a project that did this once and they very quickly got blocked writing a test by the in memory database not supporting a feature postgres had. >Realism matters It is often just a small subset of tests which have to be very expensive to build, maintain and run because of an external dependency that cannot be mocked. Mocks work for 95% of use cases and should be used even if it is not 100%.

MoreQARespect 3y ago
I never found that to be true and moreover, these scenarios with nonstandard features tended to be the scenarios I was most worried about breaking. There is also the problem of the in memory database behaving differently when given the same SQL so your test might make it look like everything works while a bug crops up in production. Realism matters.
- randomdata 3y ago
  > There is also the problem of the in memory database behaving differently when given the same SQL so your test might make it look like everything works while a bug crops up in production. You've horribly messed up the architecture of your application if that is a problem. (Or you've misunderstood previous comments)
  - MoreQARespect 3y ago
    Postgres and sqlite/in memory dbs just behave differently to each other sometimes. Knowing this fact doesn't mean you've messed up your architecture it means that you have some understanding of how these databases work. Realism matters.
    - anarazel 3y ago
      I'd say that in-memory/not-in-memory isn't the big difference - it's whether your database is in-process or not. Even with just a database running on the same node, but in a different process, connected to via unix socket, the context switches alone lead to very different performance characteristics. Actually going over the network obviously changes more. It's very easy to miss antipatterns like N+1 queries when you test on sqlite but run on a shared database in prod.
      - ivan_gammel 3y ago
        This is irrelevant for unit tests. Performance testing does not make any sense in build environment, you need to do it in an environment close to production and that’s completely different test automation scope.
        anarazel 3y ago
        You can see this stuff often even in test workloads. But even if you disregard that kind of issue, you still have stuff like needing to integrate networked database connections into e.g. event loops, which you don't really need to do for things like sqlite.