3 ms·

> I'm not going to go there much more because I still don't unit test my stuff. I'm currently not against it. I just don't know about it much. Ouch, not even

> I'm not going to go there much more because I still don't unit test my stuff. I'm currently not against it. I just don't know about it much. Ouch, not even unit tests to catch regressions? So many of the bugs that I deal with in $BIG_FOSS_PROJECT are regressions where a bug fixed in version X.Y.5 was reintroduced in X.Y+1.0 by someone's cool shiny new feature. And yes, there's a significant lack of unit tests in some of these errors, plus some hard to test code. (While I'm happy I can use Mockito to mock static methods now, the fact that I have to is a very definite code smell). Which is why I recommend never installing version A.B.0 of $BIG_FOSS_PROJECT. I always wait until everyone else finds the new bugs for you, then install A.B.1.

dvt 5y ago
> Ouch, not even unit tests to catch regressions? I also don't like unit tests and very rarely unit test. I think they provide a false sense of security and were invented by corporate software shops to better quantify "units of work" (oh, how many times I've had unit tests assigned to me in tickets!). If you can't formally prove something doesn't break (in your head or with pen & paper or via pseudocode), your code is too complex. There are a few exceptions to this, but they are highly technical (e.g. FFT, cryptographic, physics/math implementations, tricky pointer arithmetic, regular expressions, etc.) where a hard-to-see typo can actually break things in non-obvious ways. Most code is not that -- it's just written poorly (because code standards aren't enforced). Linux is a great example of a project where coding standards are annoyingly enforced (and there's no real "unit" testing) and lo and behold, the code is of exceptional quality.
- EdwardDiego 5y ago
  I'd rather have an existing suite of tests that I can use to verify that when working with other people's code. And more importantly, if JIRAISSUE-15295 is reproducible, then a unit test that a) reproduces it and b) verifies that the bug no longer occurs, is invaluable to prevent someone bringing JIRAISSUE-15295 back from the dead. Of course, if the unit test is too tightly coupled, and has too many insights into code it shouldn't, then it's worthless as JIRAISSUE-15295 will most likely reoccur via a different code path. But, poorly written unit tests aside, I have found significant value in unit tests when maintaining a rapidly changing code base.
  - bluefirebrand 5y ago
    A rapidly changing codebase is where unit tests are least useful though. Most of the changes are going to be because of new requirements which just means the test has to be updated. It's just busywork at that point. If you have discovered a way to write unit tests that can tell the difference between a regression and an enhancement, please let me know.
- jay_kyburz 5y ago
  This is probably a bad habit, but I've been writing tests so that I don't have to actually navigate though my app, put it in the correct state, and push a button to see if the feature works or not. Writing the test is just faster.
  - dvt 5y ago
    > This is probably a bad habit, but I've been writing tests so that I don't have to actually navigate though my app, put it in the correct state, and push a button to see if the feature works or not. Writing the test is just faster. This is a clever way of solving this problem, and I've ran into it before, as well. Applications where state is very deep (like a game) and you need to verify certain operations on that state is quite tricky. IMO, I'd probably call these integration tests, not unit tests.
  - Draiken 5y ago
    This is one of the major sources of bugs in my experience. People write unit tests that are perfect and pass, but only pass because you give it exactly the right input to make it pass in the first place. Then when running the actual app, the data is not exactly like on the unit test and you have bugs. To cover those scenarios, we use integration/acceptance tests. So I always come back to the same question: then why should I even bother with unit tests? Especially the unit tests that people normalized (test per class/public method). You end up with unit tests that are extremely coupled to the implementation, and without proper integration tests, you can't guarantee it will all work anyways. It only works on a bubble. I believe tests should be much more about the broader behaviors than the implementation details, but TDD and evangelists of today will have you writing tests for every small class you create. I personally still use unit tests from time to time when there are a ton of edge cases on a single behavior that I want to test, but that's the exception not the rule. By avoiding unit tests, or to put it differently making the units tested larger, you get more space to refactor, less coupling between tests and implementation as well as more meaningful tests. Somewhere along the way we lost the meaning of "unit" and it became "class/methods", when it was originally supposed to be more at a module level.
    - 5y ago
EnKopVand 5y ago
I’m not of any opinion on the matter, but with decades in the industry I’ve not seen a lot of projects where unit testing made a net profit for the company. I feel that it’s important for me to note that this is because almost all the projects I’ve been around have been build in imperfect scenarios. Sometimes projects had years worth of terrible unit tests that were terrible because they were written by people who didn’t really know how to test correctly. Sometimes because they were added waaaaaaay late in the process. Sometimes because they were sometimes skipped due to time pressure. Sometimes because the pipelines weren’t really effective or set up correctly. And so on. The issue I personally have with them is the same issue that I have with a lot of other dogmas around software development. All our theoretic tools are nice, if people actually adhere to them and know how to utilise them. But as soon as things like Test Driven Developmebt, Agile, SCRUM, Enterprise Architecture or even Unit Testing meets reality, they break in most of the cases because the theories aren’t fascist enough. By that I mean is that you can implement things in so many different ways that almost nobody manages to make them work, not really. This is anecdotal of course, and I’m sure there are much more talented people, teams and companies that derive a benefit from these things than what I’ve seen, but the only thing that’s ever, really, worked for me is too keep things as simple and single responsibility as possible. That sometimes require OOP or Unit Testing though. We wrote our general ODATA API with inheritance as an example. We do so because it lets us have a unified way of handling auditing and code-first SQL database IDs and convention and because it lets us write a single API controller and then use it in, every, other controller instead of writing the same 90 lines or code 10.000 times. You can likely do that without OOP but it’s just easy with OOP in C#. So I see a lot of these things as, do what is necessary, what works, and, what is easily maintainable. If that’s Unit Testing for you, then do it, but I can assure you that it won’t be unit testing for a lot of people out there for whatever reason. At least with single responsibility you still have a somewhat general idea of what goes wrong simply by where it happens if it isn’t caught but automated tests.
Diris 5y ago
I've seen multiple people pointing this out, but completely ignoring the next sentence. > I find it much easier to formally verify things correct than to test that they're correct. I don't have anything else to say but I wanted to put it here for context.
- rbanffy 5y ago
  Tests are also useful as a form of documentation. They explicitly exercise all plausible interfaces the developers expect to be used. I'm not sure how formally verifying things continue to be correct after a change (which could be to a dependency) can be automated and scaled.

dvt 5y ago

> Ouch, not even unit tests to catch regressions? I also don't like unit tests and very rarely unit test. I think they provide a false sense of security and were invented by corporate software shops to better quantify "units of work" (oh, how many times I've had unit tests assigned to me in tickets!). If you can't formally prove something doesn't break (in your head or with pen & paper or via pseudocode), your code is too complex. There are a few exceptions to this, but they are highly technical (e.g. FFT, cryptographic, physics/math implementations, tricky pointer arithmetic, regular expressions, etc.) where a hard-to-see typo can actually break things in non-obvious ways. Most code is not that -- it's just written poorly (because code standards aren't enforced). Linux is a great example of a project where coding standards are annoyingly enforced (and there's no real "unit" testing) and lo and behold, the code is of exceptional quality.

EdwardDiego 5y ago
I'd rather have an existing suite of tests that I can use to verify that when working with other people's code. And more importantly, if JIRAISSUE-15295 is reproducible, then a unit test that a) reproduces it and b) verifies that the bug no longer occurs, is invaluable to prevent someone bringing JIRAISSUE-15295 back from the dead. Of course, if the unit test is too tightly coupled, and has too many insights into code it shouldn't, then it's worthless as JIRAISSUE-15295 will most likely reoccur via a different code path. But, poorly written unit tests aside, I have found significant value in unit tests when maintaining a rapidly changing code base.
- bluefirebrand 5y ago
  A rapidly changing codebase is where unit tests are least useful though. Most of the changes are going to be because of new requirements which just means the test has to be updated. It's just busywork at that point. If you have discovered a way to write unit tests that can tell the difference between a regression and an enhancement, please let me know.
jay_kyburz 5y ago
This is probably a bad habit, but I've been writing tests so that I don't have to actually navigate though my app, put it in the correct state, and push a button to see if the feature works or not. Writing the test is just faster.
- dvt 5y ago
  > This is probably a bad habit, but I've been writing tests so that I don't have to actually navigate though my app, put it in the correct state, and push a button to see if the feature works or not. Writing the test is just faster. This is a clever way of solving this problem, and I've ran into it before, as well. Applications where state is very deep (like a game) and you need to verify certain operations on that state is quite tricky. IMO, I'd probably call these integration tests, not unit tests.
- Draiken 5y ago
  This is one of the major sources of bugs in my experience. People write unit tests that are perfect and pass, but only pass because you give it exactly the right input to make it pass in the first place. Then when running the actual app, the data is not exactly like on the unit test and you have bugs. To cover those scenarios, we use integration/acceptance tests. So I always come back to the same question: then why should I even bother with unit tests? Especially the unit tests that people normalized (test per class/public method). You end up with unit tests that are extremely coupled to the implementation, and without proper integration tests, you can't guarantee it will all work anyways. It only works on a bubble. I believe tests should be much more about the broader behaviors than the implementation details, but TDD and evangelists of today will have you writing tests for every small class you create. I personally still use unit tests from time to time when there are a ton of edge cases on a single behavior that I want to test, but that's the exception not the rule. By avoiding unit tests, or to put it differently making the units tested larger, you get more space to refactor, less coupling between tests and implementation as well as more meaningful tests. Somewhere along the way we lost the meaning of "unit" and it became "class/methods", when it was originally supposed to be more at a module level.
  - 5y ago

EnKopVand 5y ago

I’m not of any opinion on the matter, but with decades in the industry I’ve not seen a lot of projects where unit testing made a net profit for the company. I feel that it’s important for me to note that this is because almost all the projects I’ve been around have been build in imperfect scenarios. Sometimes projects had years worth of terrible unit tests that were terrible because they were written by people who didn’t really know how to test correctly. Sometimes because they were added waaaaaaay late in the process. Sometimes because they were sometimes skipped due to time pressure. Sometimes because the pipelines weren’t really effective or set up correctly. And so on. The issue I personally have with them is the same issue that I have with a lot of other dogmas around software development. All our theoretic tools are nice, if people actually adhere to them and know how to utilise them. But as soon as things like Test Driven Developmebt, Agile, SCRUM, Enterprise Architecture or even Unit Testing meets reality, they break in most of the cases because the theories aren’t fascist enough. By that I mean is that you can implement things in so many different ways that almost nobody manages to make them work, not really. This is anecdotal of course, and I’m sure there are much more talented people, teams and companies that derive a benefit from these things than what I’ve seen, but the only thing that’s ever, really, worked for me is too keep things as simple and single responsibility as possible. That sometimes require OOP or Unit Testing though. We wrote our general ODATA API with inheritance as an example. We do so because it lets us have a unified way of handling auditing and code-first SQL database IDs and convention and because it lets us write a single API controller and then use it in, every, other controller instead of writing the same 90 lines or code 10.000 times. You can likely do that without OOP but it’s just easy with OOP in C#. So I see a lot of these things as, do what is necessary, what works, and, what is easily maintainable. If that’s Unit Testing for you, then do it, but I can assure you that it won’t be unit testing for a lot of people out there for whatever reason. At least with single responsibility you still have a somewhat general idea of what goes wrong simply by where it happens if it isn’t caught but automated tests.

Diris 5y ago

I've seen multiple people pointing this out, but completely ignoring the next sentence. > I find it much easier to formally verify things correct than to test that they're correct. I don't have anything else to say but I wanted to put it here for context.

rbanffy 5y ago
Tests are also useful as a form of documentation. They explicitly exercise all plausible interfaces the developers expect to be used. I'm not sure how formally verifying things continue to be correct after a change (which could be to a dependency) can be automated and scaled.