5 ms·

> Modelling the happy path is great for refactoring - even a necessity, but doesn’t help with finding bugs. This is a common misconception (one that I also ini

> Modelling the happy path is great for refactoring - even a necessity, but doesn’t help with finding bugs. This is a common misconception (one that I also initially held). Unit tests aren't meant to find bugs, they're meant to protect against regressions, and in doing so, act as a documentation of how a component is supposed to behave in response to different input.

UK-AL 2y ago
They can do both.
postalrat 2y ago
I think writing tests as a form of documentation is a waste of time. If I'm using a component I don't want to read unit tests to figure out what it should do. Unit tests are most often used to cover a few more lines that need coverage. That's the value they provide.
- Izkata 2y ago
  Documentation for other developers of that code, not users of that code.
  - deleted 2y ago
    [deleted]
  - curun1r 2y ago
    Also worth noting that "other developers" includes the original developer when they return to the code several months later.
- randomdata 2y ago
  A well designed API will generally allow users to understand usage without any additional documentation, sure. However, those who modify the API in the future will want to know every last detail that you knew when you were writing it originally. That must be documented to ensure that they don't get something wrong and break things – and for their general sanity. That is, unless you hate future developers for some reason. You could do it in Word instead, I suppose, but if you write it in code then a computer can validate that the documentation you wrote is true. That brings tremendous value.
  - postalrat 2y ago
    Undocumented and undefined are the same thing. If you are worried about changing undefined features then you must be in some sort of legacy hell.
    - randomdata 2y ago
      Nothing is left undefined. Absent of documentation, most likely something will end up defined by inference. Which is not a good place to be as a developer as you have lost the nuance that went into it originally.
      - postalrat 2y ago
        Then how do you separate what is defined and can't change to what can be changed? If everything is defined then nothing can change.
        randomdata 2y ago
        You don’t change what is already defined (even if only by inference). Change is only by amendment. Will you successfully amend the changes without also changing what was previously defined if all you have is inference to go on? Probably not. That’s assuming change is even necessary. Oftentimes you only need to modify the implementation, which doesn’t change what is defined. A change in implementation has no impact on the outside, at least as long as you have properly covered your bases, which should you should be able to do as long as you have proper documentation. Without documentation, good luck to you.
- nerder92 2y ago
  Well tests are usually in English and easier to read then code IMH(uman)O
mehagar 2y ago
On that topic, static type checking can effectively be seen as a "unit test" that tests as well as documents the expected types for an interface.
- layer8 2y ago
  No, static typing proves correctness (with respect to the types), which unit testing doesn’t do.
  - mehagar 2y ago
    Unit testing proves correctness in regard to the test written (not necessarily the correctness of the application itself). They're similar in that they are both typically fast to run, and that they check an aspect of the program for correctness.
    - layer8 2y ago
      They typically can only prove correctness for specific input data, and then there’s often still some runtime or environment-dependent chance involved which may cause some fraction of the invocations to fail. Is it correct or not if a single invocation succeeds? How can you be sure?
      - mehagar 2y ago
        Unit tests should be independent of the environment they are run in, and pass or fail consistently. Otherwise, it is not a unit test.
chuckadams 2y ago
> Unit tests aren't meant to find bugs, they're meant to protect against regressions That hasn't been the general consensus on unit tests for at least 30 years now. Regression tests are a small subset of tests, typically named for an ID in some bug tracker, and are about validating a fix. The majority of unit tests catch issues before a bug is even opened, and pretty much any random developer you talk to will consider that to be the point.
- senordevnyc 2y ago
  The majority of unit tests catch issues before a bug is even opened The "issue" that is being caught is the bug the parent is talking about, not a "bug" in JIRA or something.
- whoisthemachine 2y ago
  They do not "find" bugs in the way that exploratory testing or user operation might (or even in the way that broader integration tests might), that is they don't find bugs that are not in the known problem space. But they are very good at proving a method works correctly and covers the known execution permutations.
- Izkata 2y ago
  > Regression tests are a small subset of tests, typically named for an ID in some bug tracker, and are about validating a fix. This is how I also tend to think of them, but it's not how the phrase is generally used. The general meaning of regression tests it to ensure known correct functionality doesn't break with a future change. There's no actual requirement it be tied to a known bug.
andoando 2y ago
There's a few issues with this IMO: 1. Changes often require changing the functionality of a component, which means many of the current unit tests are bunk and need to be updated. Changes that are simply refactoring but should retain the same behavior, need to update/rewrite the tests, in which case again often requires significant refactoring of the existing tests. 2. Small isolated changes usually require testing everything which in a big org is very time consuming and slows down builds and deploys unnecessarily. 3. A lot of false confidence is instilled by passing unit tests. The tests passed, were good! Most of the production bugs I've seen are things you'd never catch in a unit test. I really can't imagine a large refactor where we wouldn't end up rewriting all the tests. Integration tests are much better for that imo, "units" should be flexible.
- danielovichdk 2y ago
  Yes changing contracts implies updating tests. They should. Refactoring under the same contract should not lead to refactoring of tests. Unless of course you introduce a new dependency you have to mock ? That's just one example. If your code changes a lot it has nothing to do with tests being hard to change. It has to do with the code it tests changes too often. Poor contracts perhaps. And just like the parent comment. Tests are not about finding or solving bugs, they are about regressions and making sure your contracts are correctly implemented.
  - andoando 2y ago
    If your refactoring includes changes to interfaces, different abstractions, logical changes, business logic, then most of your tests need to be effectively rewritten. The only part where I see unit tests being useful for refactoring is making changes to the internals of a single unit. Its always been more trouble than its worth for me. In some cases it makes sense, like testing small units that heavy in logic (function that calculates order prices for example, scientific computing, etc). But unit testing every single piece of code has always seemed dogmatic to me (unit tests are good engineering, write unit tests always everywhere). Everything has tradeoffs and as engineers I think our job is to understand the pros and cons and apply them effectively.
layer8 2y ago
Regressions are bugs.

UK-AL 2y ago

They can do both.

postalrat 2y ago

I think writing tests as a form of documentation is a waste of time. If I'm using a component I don't want to read unit tests to figure out what it should do. Unit tests are most often used to cover a few more lines that need coverage. That's the value they provide.

Izkata 2y ago
Documentation for other developers of that code, not users of that code.
- deleted 2y ago
  [deleted]
- curun1r 2y ago
  Also worth noting that "other developers" includes the original developer when they return to the code several months later.
randomdata 2y ago
A well designed API will generally allow users to understand usage without any additional documentation, sure. However, those who modify the API in the future will want to know every last detail that you knew when you were writing it originally. That must be documented to ensure that they don't get something wrong and break things – and for their general sanity. That is, unless you hate future developers for some reason. You could do it in Word instead, I suppose, but if you write it in code then a computer can validate that the documentation you wrote is true. That brings tremendous value.
- postalrat 2y ago
  Undocumented and undefined are the same thing. If you are worried about changing undefined features then you must be in some sort of legacy hell.
  - randomdata 2y ago
    Nothing is left undefined. Absent of documentation, most likely something will end up defined by inference. Which is not a good place to be as a developer as you have lost the nuance that went into it originally.
    - postalrat 2y ago
      Then how do you separate what is defined and can't change to what can be changed? If everything is defined then nothing can change.
      - randomdata 2y ago
        You don’t change what is already defined (even if only by inference). Change is only by amendment. Will you successfully amend the changes without also changing what was previously defined if all you have is inference to go on? Probably not. That’s assuming change is even necessary. Oftentimes you only need to modify the implementation, which doesn’t change what is defined. A change in implementation has no impact on the outside, at least as long as you have properly covered your bases, which should you should be able to do as long as you have proper documentation. Without documentation, good luck to you.
nerder92 2y ago
Well tests are usually in English and easier to read then code IMH(uman)O

mehagar 2y ago

On that topic, static type checking can effectively be seen as a "unit test" that tests as well as documents the expected types for an interface.

layer8 2y ago
No, static typing proves correctness (with respect to the types), which unit testing doesn’t do.
- mehagar 2y ago
  Unit testing proves correctness in regard to the test written (not necessarily the correctness of the application itself). They're similar in that they are both typically fast to run, and that they check an aspect of the program for correctness.
  - layer8 2y ago
    They typically can only prove correctness for specific input data, and then there’s often still some runtime or environment-dependent chance involved which may cause some fraction of the invocations to fail. Is it correct or not if a single invocation succeeds? How can you be sure?
    - mehagar 2y ago
      Unit tests should be independent of the environment they are run in, and pass or fail consistently. Otherwise, it is not a unit test.

chuckadams 2y ago

> Unit tests aren't meant to find bugs, they're meant to protect against regressions That hasn't been the general consensus on unit tests for at least 30 years now. Regression tests are a small subset of tests, typically named for an ID in some bug tracker, and are about validating a fix. The majority of unit tests catch issues before a bug is even opened, and pretty much any random developer you talk to will consider that to be the point.

senordevnyc 2y ago
The majority of unit tests catch issues before a bug is even opened The "issue" that is being caught is the bug the parent is talking about, not a "bug" in JIRA or something.
whoisthemachine 2y ago
They do not "find" bugs in the way that exploratory testing or user operation might (or even in the way that broader integration tests might), that is they don't find bugs that are not in the known problem space. But they are very good at proving a method works correctly and covers the known execution permutations.
Izkata 2y ago
> Regression tests are a small subset of tests, typically named for an ID in some bug tracker, and are about validating a fix. This is how I also tend to think of them, but it's not how the phrase is generally used. The general meaning of regression tests it to ensure known correct functionality doesn't break with a future change. There's no actual requirement it be tied to a known bug.

andoando 2y ago

There's a few issues with this IMO: 1. Changes often require changing the functionality of a component, which means many of the current unit tests are bunk and need to be updated. Changes that are simply refactoring but should retain the same behavior, need to update/rewrite the tests, in which case again often requires significant refactoring of the existing tests. 2. Small isolated changes usually require testing everything which in a big org is very time consuming and slows down builds and deploys unnecessarily. 3. A lot of false confidence is instilled by passing unit tests. The tests passed, were good! Most of the production bugs I've seen are things you'd never catch in a unit test. I really can't imagine a large refactor where we wouldn't end up rewriting all the tests. Integration tests are much better for that imo, "units" should be flexible.

danielovichdk 2y ago
Yes changing contracts implies updating tests. They should. Refactoring under the same contract should not lead to refactoring of tests. Unless of course you introduce a new dependency you have to mock ? That's just one example. If your code changes a lot it has nothing to do with tests being hard to change. It has to do with the code it tests changes too often. Poor contracts perhaps. And just like the parent comment. Tests are not about finding or solving bugs, they are about regressions and making sure your contracts are correctly implemented.
- andoando 2y ago
  If your refactoring includes changes to interfaces, different abstractions, logical changes, business logic, then most of your tests need to be effectively rewritten. The only part where I see unit tests being useful for refactoring is making changes to the internals of a single unit. Its always been more trouble than its worth for me. In some cases it makes sense, like testing small units that heavy in logic (function that calculates order prices for example, scientific computing, etc). But unit testing every single piece of code has always seemed dogmatic to me (unit tests are good engineering, write unit tests always everywhere). Everything has tradeoffs and as engineers I think our job is to understand the pros and cons and apply them effectively.

layer8 2y ago

Regressions are bugs.