4 ms·

Who thinks that "unit test" refers to the unit under test? I've not once in my life heard anyone using that definition.

paxys 5y ago
The author is talking about what the "unit" in unit test means. The conventional definition is that it refers to the granular unit of your application being tested. He instead argues it means the test itself is an independent unit within the rest of your test suite.
tsimionescu 5y ago
A "unit test" is a test that covers a single unit of code, as opposed to an "integration test" that that covers several integrated units. This is the common definition. This other definition of "a test that doesn't depend on other tests" seems of little use, since no tests, even in something like a UI regression, should depend on other tests.
- layer8 5y ago
  What is a “unit of code” though. Most units of code call (“integrate”?) other units of code in their implementation. Libraries, language runtime, runtime environment (OS), other functional modules of the application, utility functions, … What is the defining property where it becomes an integration test?
  - tsimionescu 5y ago
    That is more subjective and depends on the team, but for any particular definition, there will typically be a clear distinction (for that particular team/code base) between the two. A typical distinction is that a Unit Test only covers a single function/class method, or tightly coupled group of classes, and almost always no IO (network calls, disk writes). Conversely, integration tests typically cover the public API of an entire publically released library, network service, or CLI executable, usually run in the targeted deployment. For example, for a network service, an integration test may do a PUT and then a GET and check that the resource is returned. It would be running against a full copy of the service which is connected to a real database, but probably one populated with mock data. The same service would typically have unit tests checking something like "the handler for PUT generates an SQL query string that contains the fields defined in the PUT body" (assuming this is a code base without an ORM). Another unit test would check that the same method returns an error if a field is not supported. Another one may check that it doesn't create unsanitized SQL, and so on.
    - layer8 5y ago
      My question is, what are the distinguishing properties of unit tests vs. integration tests that motivate treating them differently, and in what ways do they have to be treated differently? In broader terms, why does that distinction exist and how is it relevant?
      - tsimionescu 5y ago
        The biggest difference is typically that unit tests don't require a test fixture of any kind, and are expected to run wherever the compiler runs. You typically expect a huge battery of unit tests to run in seconds, and they are normally expected to run before every commit, even for every build on a dev machine. A new contributor is expected to be able to do something close to "apt get langauge-toolchain && git clone $repo && make build && make unit" and see that all unit tests pass. However, they are also expected to sometimes miss obvious bugs, like "the queries we generate are not valid with the new MySQL flags we set". They often need to be changed even for refactorings, as they typically cover implementation details, not just the public API. In contrast, integration tests typically require a test fixture of some kind and are thus expected to require more complex setup instructions, perhaps even access to shared resources (like a DB server with a large batch of test data that can't realistically be replicated, or a testing instance of a real microservice you integrate with). By virtue of often requiring IO (network operations, DB operations etc), they are often considerably slower, so that they become impractical to run at every build. The upside is that they catch errors that were not obvious from the code, and they usually give you a high degree of assurance that a user of your service will at least not encounter silly bugs. A break in the integration tests after a redactor is also often a potential sign you've broken backwards compatibility of your API.
ribs 5y ago
I have always thought that.

The author is talking about what the "unit" in unit test means. The conventional definition is that it refers to the granular unit of your application being tested. He instead argues it means the test itself is an independent unit within the rest of your test suite.

tsimionescu 5y ago

A "unit test" is a test that covers a single unit of code, as opposed to an "integration test" that that covers several integrated units. This is the common definition. This other definition of "a test that doesn't depend on other tests" seems of little use, since no tests, even in something like a UI regression, should depend on other tests.

layer8 5y ago
What is a “unit of code” though. Most units of code call (“integrate”?) other units of code in their implementation. Libraries, language runtime, runtime environment (OS), other functional modules of the application, utility functions, … What is the defining property where it becomes an integration test?
- tsimionescu 5y ago
  That is more subjective and depends on the team, but for any particular definition, there will typically be a clear distinction (for that particular team/code base) between the two. A typical distinction is that a Unit Test only covers a single function/class method, or tightly coupled group of classes, and almost always no IO (network calls, disk writes). Conversely, integration tests typically cover the public API of an entire publically released library, network service, or CLI executable, usually run in the targeted deployment. For example, for a network service, an integration test may do a PUT and then a GET and check that the resource is returned. It would be running against a full copy of the service which is connected to a real database, but probably one populated with mock data. The same service would typically have unit tests checking something like "the handler for PUT generates an SQL query string that contains the fields defined in the PUT body" (assuming this is a code base without an ORM). Another unit test would check that the same method returns an error if a field is not supported. Another one may check that it doesn't create unsanitized SQL, and so on.
  - layer8 5y ago
    My question is, what are the distinguishing properties of unit tests vs. integration tests that motivate treating them differently, and in what ways do they have to be treated differently? In broader terms, why does that distinction exist and how is it relevant?
    - tsimionescu 5y ago
      The biggest difference is typically that unit tests don't require a test fixture of any kind, and are expected to run wherever the compiler runs. You typically expect a huge battery of unit tests to run in seconds, and they are normally expected to run before every commit, even for every build on a dev machine. A new contributor is expected to be able to do something close to "apt get langauge-toolchain && git clone $repo && make build && make unit" and see that all unit tests pass. However, they are also expected to sometimes miss obvious bugs, like "the queries we generate are not valid with the new MySQL flags we set". They often need to be changed even for refactorings, as they typically cover implementation details, not just the public API. In contrast, integration tests typically require a test fixture of some kind and are thus expected to require more complex setup instructions, perhaps even access to shared resources (like a DB server with a large batch of test data that can't realistically be replicated, or a testing instance of a real microservice you integrate with). By virtue of often requiring IO (network operations, DB operations etc), they are often considerably slower, so that they become impractical to run at every build. The upside is that they catch errors that were not obvious from the code, and they usually give you a high degree of assurance that a user of your service will at least not encounter silly bugs. A break in the integration tests after a redactor is also often a potential sign you've broken backwards compatibility of your API.

ribs 5y ago

I have always thought that.