4 ms·

A test will only catch an edge case you already thought of. If you thought of it anyway why just not fix the bug instead? Tests have burned out software engine

A test will only catch an edge case you already thought of. If you thought of it anyway why just not fix the bug instead? Tests have burned out software engineers who waste the majority of their time deriving tests that will pass anyway. And then a significant code change will render them useless, at which point they have to be rewritten from scratch. No your program will not be more correct with more tests. Deal with it.

7bit 2y ago
Do you think the test is written and the bug left in? What a weird take. And then, you write the test so that future changes (small or big) that causes regressions get noticed before the regression is put into production again. Especially in complex systems, you can define the end result and test if all your cases are covered. You do this anyway manually, so why not just write a test instead?
- whatever1 2y ago
  The time that you spent to write a test is the time you would have spent in finding another bug in the code. Your time is finite. Bugs are not.
becquerel 2y ago
You write the test to prevent the bug from being accidentally reintroduced in the future. I have seen showstopper bugs reintroduced into production multiple times after they were fixed.
bubblebeard 2y ago
For me at least, designing a test will usually let me discover problems with my code which may otherwise gone unnoticed. Leaving the tests there once written to help us in future refactoring costs nothing. Granted, in some languages tests are more complicated to write compared to others. In PHP it’s a nightmare, in Rust it’s so easy it’s hard to avoid doing. I hear what you are saying though, sometimes writing tests consume more time then is necessary.
- yard2010 2y ago
  I completely agree with what you're saying - tests help me ensure nothing breaks and change stuff fast. But leaving EVERY code behind is a liability. In the best case, it's free, otherwise, it's another point of failure and other engineers might spend time understanding it. Code is a liability. It has to have a good reason to be there in the first place - in the case of tests, it's worth it because it saves more time on bugs, but this can easily turn into a premature optimization.
  - bubblebeard 2y ago
    Very well put! Couldn’t have said it better myself
wesselbindt 2y ago
> A test will only catch an edge case you already thought of. If you thought of it anyway why just not fix the bug instead? The reason I do this is to prevent the bug from re-occurring with future changes. The alternative is to just remember for every part of the system I work on all edge cases and past bugs, but sadly I simply do not have the mental capacity to do this, and honestly doubt if anyone does.
- whatever1 2y ago
  If a future change is relevant to an existing piece of code then the logic needs to be rethought from scratch. Your past tests have no guarantee that will be still relevant or comprehensive. So skip the tests and work more on the code instead.
  - quectophoton 2y ago
    If a requirement changes, the test for that requirement obviously has to change. These tests breaking is normal (you had a requirement that "this is red", and a test ensuring "this is red", but now suddenly higher ups decide that "this is not red", so it's obvious why this test breaking is normal). If a requirement doesn't change, the test for those requirements should not change, no matter what you change. If these tests break, it likely means they are at the wrong abstraction level or just plainly wrong. Those are the things I look at. I don't even care if people call stuff "unit tests", "integration tests". I don't care about what should be mocked/faked/stubbed. I don't care about whatever other bikeshedding people want to go on. E.g. if your app is an HTTP API, then you should be able to change your database engine without breaking tests like "user shouldn't be able to change the email of another user". And you should also be able to change your programming language without breaking any tests for user-facing behavior (e.g. "`GET /preferences` returns the preferences for the authenticated user"). E.g. if your code is a compiler, you should be able to add and remove optimizations without changing any tests, other than those specific to those optimizations (e.g. the test for "code with optimizations should behave the same as code without optimizations" shouldn't change, except for specific cases like compiling only with that optimization enabled or with some specific set of optimizations that includes this optimization).
quectophoton 2y ago
Will all your team members also think about those edge cases when changing that part of the code? Will they ensure the behavior is the same when a library dependency is updated? So, tests catch edge cases that someone else thought of but not everyone might have. This "not everyone" includes yourself, either yourself from the future (e.g. because some parts of the product are not so fresh in your mind), or yourself from now (e.g. because you didn't even know there was a requirement that must be met and your change here broke a requirement over there). To put an easy to understand example, vulnerability checkers are still tests (and so are linters and similar tools, but let's focus on vulnerabilities). Your post implies you don't need them because you can perfectly prevent a vulnerability from ever happening again once you know about it, both because you write code that doesn't have that vulnerability and because you check that your dependencies don't have that vulnerability. So, think of tests more like assertions or checksums.
- kagevf 2y ago
  > think of tests more like assertions or checksums. That's a good way to summarize how tests can catch regressions. And, I think I'm stealing that!
creesch 2y ago
I assume you are talking about unit tests here. Thinking of edge cases is exactly what unit tests are for. They are, when used properly, a way to think about various edge cases *before* you write your code. And then, once you have written your code, validate that it indeed does what you expected to do so beforehand. The issue I am seeing, more often than not, is that people try to write unit tests after the fact. Which means that a lot of the value of them will be lost. In addition to that, if you rewrite your code so often that it renders many of your tests invalid I'd argue that there is a fundamental issue elsewhere. In more stable environments, unit tests help document the behavior of your code, which in turn helps when rewriting your code. Basically, if you are just writing tests because people told you to write tests, it is no surprise you burn out over them. To be fair, this happens all too often. Certainly with the idiotic requirement added to it that you need 80% coverage without any other context. If you write tests while understanding where they fit in the process, they can actually be valuable for you.
ahartmetz 2y ago
There are things that are easier to verify than to do correctly. Almost anything that vaguely looks like a proper algorithm has that property. Sorting, balanced trees, hashtables, some kinds of data splicing, even some slightly more complicated string processing. Sometimes it's also possible to do exhaustive testing. I once did that with a state machine-like piece of code, test transitions from all states to all other states.
codr7 2y ago
Writing a test is often the best way to reproduce and make sure you fixed a bug. Keeping them for a while lets you make sure it doesn't pop up again. 10 years later, they probably don't add much value. Tests are tools, that's like saying 'No, your food won't taste better with more salt.', it depends.
drewcoo 2y ago
> A test will only catch an edge case you already thought of. Property-based tests and model-based tests can catch edge cases I never thought of. > Tests have burned out software engineers who waste the majority of their time deriving tests that will pass anyway. Burn, baby, burn! We don't need programmers who can't handle testing.

7bit 2y ago

Do you think the test is written and the bug left in? What a weird take. And then, you write the test so that future changes (small or big) that causes regressions get noticed before the regression is put into production again. Especially in complex systems, you can define the end result and test if all your cases are covered. You do this anyway manually, so why not just write a test instead?

whatever1 2y ago
The time that you spent to write a test is the time you would have spent in finding another bug in the code. Your time is finite. Bugs are not.

becquerel 2y ago

You write the test to prevent the bug from being accidentally reintroduced in the future. I have seen showstopper bugs reintroduced into production multiple times after they were fixed.

bubblebeard 2y ago

For me at least, designing a test will usually let me discover problems with my code which may otherwise gone unnoticed. Leaving the tests there once written to help us in future refactoring costs nothing. Granted, in some languages tests are more complicated to write compared to others. In PHP it’s a nightmare, in Rust it’s so easy it’s hard to avoid doing. I hear what you are saying though, sometimes writing tests consume more time then is necessary.

yard2010 2y ago
I completely agree with what you're saying - tests help me ensure nothing breaks and change stuff fast. But leaving EVERY code behind is a liability. In the best case, it's free, otherwise, it's another point of failure and other engineers might spend time understanding it. Code is a liability. It has to have a good reason to be there in the first place - in the case of tests, it's worth it because it saves more time on bugs, but this can easily turn into a premature optimization.
- bubblebeard 2y ago
  Very well put! Couldn’t have said it better myself

wesselbindt 2y ago

> A test will only catch an edge case you already thought of. If you thought of it anyway why just not fix the bug instead? The reason I do this is to prevent the bug from re-occurring with future changes. The alternative is to just remember for every part of the system I work on all edge cases and past bugs, but sadly I simply do not have the mental capacity to do this, and honestly doubt if anyone does.

whatever1 2y ago
If a future change is relevant to an existing piece of code then the logic needs to be rethought from scratch. Your past tests have no guarantee that will be still relevant or comprehensive. So skip the tests and work more on the code instead.
- quectophoton 2y ago
  If a requirement changes, the test for that requirement obviously has to change. These tests breaking is normal (you had a requirement that "this is red", and a test ensuring "this is red", but now suddenly higher ups decide that "this is not red", so it's obvious why this test breaking is normal). If a requirement doesn't change, the test for those requirements should not change, no matter what you change. If these tests break, it likely means they are at the wrong abstraction level or just plainly wrong. Those are the things I look at. I don't even care if people call stuff "unit tests", "integration tests". I don't care about what should be mocked/faked/stubbed. I don't care about whatever other bikeshedding people want to go on. E.g. if your app is an HTTP API, then you should be able to change your database engine without breaking tests like "user shouldn't be able to change the email of another user". And you should also be able to change your programming language without breaking any tests for user-facing behavior (e.g. "`GET /preferences` returns the preferences for the authenticated user"). E.g. if your code is a compiler, you should be able to add and remove optimizations without changing any tests, other than those specific to those optimizations (e.g. the test for "code with optimizations should behave the same as code without optimizations" shouldn't change, except for specific cases like compiling only with that optimization enabled or with some specific set of optimizations that includes this optimization).

quectophoton 2y ago

Will all your team members also think about those edge cases when changing that part of the code? Will they ensure the behavior is the same when a library dependency is updated? So, tests catch edge cases that someone else thought of but not everyone might have. This "not everyone" includes yourself, either yourself from the future (e.g. because some parts of the product are not so fresh in your mind), or yourself from now (e.g. because you didn't even know there was a requirement that must be met and your change here broke a requirement over there). To put an easy to understand example, vulnerability checkers are still tests (and so are linters and similar tools, but let's focus on vulnerabilities). Your post implies you don't need them because you can perfectly prevent a vulnerability from ever happening again once you know about it, both because you write code that doesn't have that vulnerability and because you check that your dependencies don't have that vulnerability. So, think of tests more like assertions or checksums.

kagevf 2y ago
> think of tests more like assertions or checksums. That's a good way to summarize how tests can catch regressions. And, I think I'm stealing that!

creesch 2y ago

I assume you are talking about unit tests here. Thinking of edge cases is exactly what unit tests are for. They are, when used properly, a way to think about various edge cases *before* you write your code. And then, once you have written your code, validate that it indeed does what you expected to do so beforehand. The issue I am seeing, more often than not, is that people try to write unit tests after the fact. Which means that a lot of the value of them will be lost. In addition to that, if you rewrite your code so often that it renders many of your tests invalid I'd argue that there is a fundamental issue elsewhere. In more stable environments, unit tests help document the behavior of your code, which in turn helps when rewriting your code. Basically, if you are just writing tests because people told you to write tests, it is no surprise you burn out over them. To be fair, this happens all too often. Certainly with the idiotic requirement added to it that you need 80% coverage without any other context. If you write tests while understanding where they fit in the process, they can actually be valuable for you.

ahartmetz 2y ago

There are things that are easier to verify than to do correctly. Almost anything that vaguely looks like a proper algorithm has that property. Sorting, balanced trees, hashtables, some kinds of data splicing, even some slightly more complicated string processing. Sometimes it's also possible to do exhaustive testing. I once did that with a state machine-like piece of code, test transitions from all states to all other states.

codr7 2y ago

Writing a test is often the best way to reproduce and make sure you fixed a bug. Keeping them for a while lets you make sure it doesn't pop up again. 10 years later, they probably don't add much value. Tests are tools, that's like saying 'No, your food won't taste better with more salt.', it depends.

drewcoo 2y ago

> A test will only catch an edge case you already thought of. Property-based tests and model-based tests can catch edge cases I never thought of. > Tests have burned out software engineers who waste the majority of their time deriving tests that will pass anyway. Burn, baby, burn! We don't need programmers who can't handle testing.