4 ms·

> Don't forget the part where the queries are impossible to test, because you can't spin up real instances of all 15 APIs in a test environment, Why? What's st

by rualca 6y ago

> Don't forget the part where the queries are impossible to test, because you can't spin up real instances of all 15 APIs in a test environment, Why? What's stopping you? > so all the HTTP calls are mocked and the responses are meaningless! Why are you talking about mocks in the context of an integration tests?

bpodgursky 6y ago
You're the one who said "integration test". A single DB query is normally something that _can_ be trivially unit-tested. But if the DB query is split across half a dozen complex services, suddenly it can _only_ be integration tested. Which usually means slow iteration and no local testing.
- rualca 6y ago
  > You're the one who said "integration test". Yes, I was. And that's the problem, because it seems you didn't realize that tests that involve checking how 15 APIs interact are indeed tests that check how different services are integrated. Do you disagree with anything? > A single DB query is normally something that _can_ be trivially unit-tested. Well, not really. You unit test the call to the repository, not the call to the DB. Calls to the DB are tested in integration tests. Think about it. > But if the DB query is split across half a dozen complex services, Aren't you trying to tests how half a dozen complex services integrate? I mean, take a step back and read your own description of the problem. As soon as you talk about interacting "complex services" is already a clear tell that you're way beyond unit testing territory. > Which usually means slow iteration and no local testing. No, not really. It still means you have to add unit tests that cover how your individual clients and repositories and business logic works under the expected scenarios of a distributed system, and after you pass all of those you check how your service interacts with other services in integration tests.
  - bpodgursky 6y ago
    > Calls to the DB are tested in integration tests This might be our disagreement -- this is IMO outdated terminology, and in my experience not modern practice. It's super easy now-days to swap in a local lightweight database like sqlite which is 99.9% compatible with your production database, and use this for even tiny unit tests. Validating any single database call would definitely be a unit test in any of the codebases I've worked with. "Integration" tests in my mind cover cross-service interfaces and infrastructure which is a PITA to access locally. For example, interacting AWS infrastructure whose behavior you can't reliably mock locally.
    - DougBTX 6y ago
      There's a fair amount of truth to this. Many applications are glorified SQL generators, that effectively JIT compile the application logic to SQL. So if you want to test the application logic, then you need to run it on the runtime that it has been compiled for, that is: the database. Testing the application code in isolation would be like trying to test that your compiled binary contains the machine code instructions you expect, rather than testing that it actually runs correctly. There are niche times when that's useful, but most of the time, not really.
    - wpietri 6y ago
      If you're spinning up a database, however lightweight, I'd still call it an integration test. I think part of the terminology problem is that a lot of apps and services are basically code for turning an HTTP request into an SQL request, and then some more code for turning a SQL response into an HTTP response. That means there's very little in the way of discrete logic to unit test without a bunch of mocking. In a system like that, I think the only meaningful tests are integration tests. For people who only work on that kind of glue code, it's easy to forget that testable units really do occur in other kinds of coding. But I promise they do.
    - GordonS 6y ago
      > this is IMO outdated terminology, and in my experience not modern practice. It's super easy now-days to swap in a local lightweight database like sqlite which is 99.9% compatible with your production database, and use this for even tiny unit tests. I actually see it as more "modern" to use an actual database. I've been around long enough to have seen trends going from database -> mocks -> fake -> back to databases again! Mocking is fine for some cases, as are fakes (e.g. lightweight SQLite wrapper you mention), but for anything non-trivial, such solutions often have "blind spots" - such as foreign keys, or unsupported data types. At the same time Docker was becoming a thing, I (and others) were growing tired of seeing issues related to these limitations. At some point, it pays to have integration tests that just hit a real database. And with Docker, you can do that trivially - it takes only a few seconds to spin up and populate a fresh Postgres instance. I haven't looked back since.

by rualca 6y ago

bpodgursky 6y ago

You're the one who said "integration test". A single DB query is normally something that _can_ be trivially unit-tested. But if the DB query is split across half a dozen complex services, suddenly it can _only_ be integration tested. Which usually means slow iteration and no local testing.

rualca 6y ago
> You're the one who said "integration test". Yes, I was. And that's the problem, because it seems you didn't realize that tests that involve checking how 15 APIs interact are indeed tests that check how different services are integrated. Do you disagree with anything? > A single DB query is normally something that _can_ be trivially unit-tested. Well, not really. You unit test the call to the repository, not the call to the DB. Calls to the DB are tested in integration tests. Think about it. > But if the DB query is split across half a dozen complex services, Aren't you trying to tests how half a dozen complex services integrate? I mean, take a step back and read your own description of the problem. As soon as you talk about interacting "complex services" is already a clear tell that you're way beyond unit testing territory. > Which usually means slow iteration and no local testing. No, not really. It still means you have to add unit tests that cover how your individual clients and repositories and business logic works under the expected scenarios of a distributed system, and after you pass all of those you check how your service interacts with other services in integration tests.
- bpodgursky 6y ago
  > Calls to the DB are tested in integration tests This might be our disagreement -- this is IMO outdated terminology, and in my experience not modern practice. It's super easy now-days to swap in a local lightweight database like sqlite which is 99.9% compatible with your production database, and use this for even tiny unit tests. Validating any single database call would definitely be a unit test in any of the codebases I've worked with. "Integration" tests in my mind cover cross-service interfaces and infrastructure which is a PITA to access locally. For example, interacting AWS infrastructure whose behavior you can't reliably mock locally.
  - DougBTX 6y ago
    There's a fair amount of truth to this. Many applications are glorified SQL generators, that effectively JIT compile the application logic to SQL. So if you want to test the application logic, then you need to run it on the runtime that it has been compiled for, that is: the database. Testing the application code in isolation would be like trying to test that your compiled binary contains the machine code instructions you expect, rather than testing that it actually runs correctly. There are niche times when that's useful, but most of the time, not really.
  - wpietri 6y ago
    If you're spinning up a database, however lightweight, I'd still call it an integration test. I think part of the terminology problem is that a lot of apps and services are basically code for turning an HTTP request into an SQL request, and then some more code for turning a SQL response into an HTTP response. That means there's very little in the way of discrete logic to unit test without a bunch of mocking. In a system like that, I think the only meaningful tests are integration tests. For people who only work on that kind of glue code, it's easy to forget that testable units really do occur in other kinds of coding. But I promise they do.
  - GordonS 6y ago
    > this is IMO outdated terminology, and in my experience not modern practice. It's super easy now-days to swap in a local lightweight database like sqlite which is 99.9% compatible with your production database, and use this for even tiny unit tests. I actually see it as more "modern" to use an actual database. I've been around long enough to have seen trends going from database -> mocks -> fake -> back to databases again! Mocking is fine for some cases, as are fakes (e.g. lightweight SQLite wrapper you mention), but for anything non-trivial, such solutions often have "blind spots" - such as foreign keys, or unsupported data types. At the same time Docker was becoming a thing, I (and others) were growing tired of seeing issues related to these limitations. At some point, it pays to have integration tests that just hit a real database. And with Docker, you can do that trivially - it takes only a few seconds to spin up and populate a fresh Postgres instance. I haven't looked back since.