4 ms·

If your product is used by humans, then it needs to be tested by humans - this cannot be automated. Those humans can be your QA people, or your customers. Perha

If your product is used by humans, then it needs to be tested by humans - this cannot be automated. Those humans can be your QA people, or your customers. Perhaps your customers are happy to be testers, perhaps not. Unit tests are very expensive and return little value. Conversely, a (manual?) 'smoke test' is very cheap and returns great value - the first thing you do when updating a server for example is to check it still responds (and nothing has gone wrong in the deployment process), takes 2 seconds to do, prevents highly embarrassing downtime due to a misconfigured docker pull or whatever.

baal80spam 6mo ago
> Unit tests are very expensive and return little value Why are unit tests very expensive? This goes against everything I know.
- somewhereoutth 6mo ago
  Unit tests very roughly double the amount of effort required to make any meaningful change to your codebase. They are also require maintenance same as ordinary code - but the customer does not care in the slightest whether or not they pass. On the other side, they can only really tell you about low level bugs that you already expected, they cannot surface system level bugs - the actual hard bugs that cause problems for you and your customers. Then there is the danger of thinking that green=all good, an example of 'automation bias' where we learn to trust the automation even as things go wrong. As makers, it is also tempting to believe that [all] problems can be solved by making something (i.e. code), but actually many problems are not of that nature, and cannot be solved in that way.
  - baal80spam 6mo ago
    Thank you, that makes sense. What I meant was that today all unit tests are basically written by an AI so the "cost" is almost zero. Am I wrong?
    - somewhereoutth 6mo ago
      Sorry yes. If an LLM written unit test fails, then it has to be determined whether the test was wrong or the code was wrong. This is an expense in human oversight, unless of course we believe that LLMs will get it right at a high enough rate that they can be left to code everything themselves completely automatically.
ivan_gammel 6mo ago
Automated tests are no longer expensive to write and nowadays less expensive to execute.
hrmtst93837 6mo ago
[flagged]
Pannoniae 6mo ago
Sad to see you being downvoted, but you're exactly right. Well, almost - if you can afford to invest in a good integration test suite, that can catch many errors without requiring a human to regression-test every time. At the same time, many quality attributes can't really be automatically tested, so automation shouldn't try to replace manual testing, it should be used to augment it.

baal80spam 6mo ago

> Unit tests are very expensive and return little value Why are unit tests very expensive? This goes against everything I know.

somewhereoutth 6mo ago
Unit tests very roughly double the amount of effort required to make any meaningful change to your codebase. They are also require maintenance same as ordinary code - but the customer does not care in the slightest whether or not they pass. On the other side, they can only really tell you about low level bugs that you already expected, they cannot surface system level bugs - the actual hard bugs that cause problems for you and your customers. Then there is the danger of thinking that green=all good, an example of 'automation bias' where we learn to trust the automation even as things go wrong. As makers, it is also tempting to believe that [all] problems can be solved by making something (i.e. code), but actually many problems are not of that nature, and cannot be solved in that way.
- baal80spam 6mo ago
  Thank you, that makes sense. What I meant was that today all unit tests are basically written by an AI so the "cost" is almost zero. Am I wrong?
  - somewhereoutth 6mo ago
    Sorry yes. If an LLM written unit test fails, then it has to be determined whether the test was wrong or the code was wrong. This is an expense in human oversight, unless of course we believe that LLMs will get it right at a high enough rate that they can be left to code everything themselves completely automatically.

ivan_gammel 6mo ago

Automated tests are no longer expensive to write and nowadays less expensive to execute.

hrmtst93837 6mo ago

[flagged]

Pannoniae 6mo ago

Sad to see you being downvoted, but you're exactly right. Well, almost - if you can afford to invest in a good integration test suite, that can catch many errors without requiring a human to regression-test every time. At the same time, many quality attributes can't really be automatically tested, so automation shouldn't try to replace manual testing, it should be used to augment it.