4 ms·

> IO can’t be unit tested hence why you mock it. Say, I have this module that uses a private MongoDB as a cache. Its unit tests spin up a standard MongoDB cont

> IO can’t be unit tested hence why you mock it. Say, I have this module that uses a private MongoDB as a cache. Its unit tests spin up a standard MongoDB container and use it (and then tear it down). Are they still unit tests or should I start calling them "integration tests"?

ninetyninenine 2y ago
Integration. Unit tests should just test code units. All the external stuff should be mocked or not tested. The example in the post is a unit test. It’s good to keep it separate as unit tests are really easy to run and less complicated and much faster. Integration tests are much more complicated and often sort of freeze your code as it locks in the implementation.
- joshfee 2y ago
  How you define "external stuff" matters though. As soon as your function is calling another function, your test can be argued to be an "integration test", as you're now implicitly also testing the logic of the other function. Alternatively you mock _everything_ and then your "unit test" ends up just being a tautological test asserting that "the code I wrote executes in the way I wrote it". (Not to mention that every time you mock something you are also implicitly asserting what the expected behavior of that thing is). The only truly reliable tests are E2E tests, but they are too expensive to cover all possible permutations as there are just too many. This is the catch 22 with testing, and we're always forced to make pragmatic choices about where to draw our boundaries to maximize the value (i.e. actual bugs caught) of them.
  - ninetyninenine 2y ago
    By external I mean anything outside the compiled executable. Or anything that you have to use ipc like sockets or shared memory.
  - meltyness 2y ago
    Functions, in the sense that there are global variables, is strongly discouraged by the language. It's usually much easier to define a type so there is a clear delineation. That is, from Mongo, you use Serde and wind up with only valid records operated upon, of a table of such values.
tialaramex 2y ago
I think that's still unit tests if that "standard MongoDB container" is actually hermetic. It's pretty easy to make testing stuff like "We'll conjure into existence a loopback network server" hermetic and likewise for the Entity Framework trick where it runs tests against a SQLite db even though that's not how your real DB works, it's often good enough. Containers are something which could be hermetic, but I am dubious.

ninetyninenine 2y ago

Integration. Unit tests should just test code units. All the external stuff should be mocked or not tested. The example in the post is a unit test. It’s good to keep it separate as unit tests are really easy to run and less complicated and much faster. Integration tests are much more complicated and often sort of freeze your code as it locks in the implementation.

joshfee 2y ago
How you define "external stuff" matters though. As soon as your function is calling another function, your test can be argued to be an "integration test", as you're now implicitly also testing the logic of the other function. Alternatively you mock _everything_ and then your "unit test" ends up just being a tautological test asserting that "the code I wrote executes in the way I wrote it". (Not to mention that every time you mock something you are also implicitly asserting what the expected behavior of that thing is). The only truly reliable tests are E2E tests, but they are too expensive to cover all possible permutations as there are just too many. This is the catch 22 with testing, and we're always forced to make pragmatic choices about where to draw our boundaries to maximize the value (i.e. actual bugs caught) of them.
- ninetyninenine 2y ago
  By external I mean anything outside the compiled executable. Or anything that you have to use ipc like sockets or shared memory.
- meltyness 2y ago
  Functions, in the sense that there are global variables, is strongly discouraged by the language. It's usually much easier to define a type so there is a clear delineation. That is, from Mongo, you use Serde and wind up with only valid records operated upon, of a table of such values.

tialaramex 2y ago

I think that's still unit tests if that "standard MongoDB container" is actually hermetic. It's pretty easy to make testing stuff like "We'll conjure into existence a loopback network server" hermetic and likewise for the Entity Framework trick where it runs tests against a SQLite db even though that's not how your real DB works, it's often good enough. Containers are something which could be hermetic, but I am dubious.