7 ms·

True "unit" tests are: * faster to run * give you less confidence in the correctness of the system (per time spent writing them) * when they fail, give you m

• curun1r 5y ago
  The other point about unit tests is that they allow you to test far more permutations. At a micro level, you can test more of the possible types of inputs to your function. When you move to integration tests, the permutations multiply to the point that it would require billions of tests to test the same range of inputs. This is why both are so necessary. Integration testing tests whether the whole thing is coherent when put together. But it’s terrible for testing edge cases and error scenarios. That’s where unit testing comes in. It gives the developer a chance to codify every edge case they’ve considered to automate the process of catching regressions.
• arkh 5y ago
  Don't forget a very important thing: unit test are hindering any refactoring. People will resist it because it means they have to rewrite their tests. And if you don't have enough e2e tests, you have nothing to check your refactoring efforts have not broken anything.
  • GrumpyNl 5y ago
    I have noticed that 80% of the time the unit test fails, the unit test has to be rewritten.
  • rileymat2 5y ago
    Do they hinder refactoring or rewriting? When I am doing legitimate refactoring in simple steps by the book, unit tests are helpful and speed it up. When I am rewriting, I see your issue. For example, a sprout method or sprout class refactor do not change the tests.
  • hamandcheese 5y ago
    If a test is in my way, I delete it.
    • amw-zero 5y ago
      Then what is the point of it being there in the first place?
      • booi 5y ago
        The point of what being there? There's nothing there.
        • amw-zero 5y ago
          What is the point of the initial test if you’re just going to delete it.
          • bluGill 5y ago
            Depends on why it failed. Many times it fails because a requirement changed, so delete the no longer valid test
      • hamandcheese 5y ago
        As other said, if/when it comes to this it is because the code in question has change in a significant way. So the point was to test a previous version.
        • BerislavLopac 5y ago
          Tests have nothing to do with the code. They are supposed to test the behaviour.
    • Aeolun 5y ago
      Must be fun to work with you.
  • spaetzleesser 5y ago
    Very true. A lot of unit tests, especially the ones with mocks tend to test a lot of internal behavior. So when you refactor your system, you often have to rewrite unit tests although the system actually performs correctly.
  • rualca 5y ago
    > Don't forget a very important thing: unit test are hindering any refactoring. People will resist it because it means they have to rewrite their tests. Well, unit tests verify a contract. If a developer wants to change code but is incapable of respecting a contract (i.e., preserving old invariants or adding new ones that make sense) then he should not be messing with that code at all, shouldn't he? In that sense alone, it's quite clear that unit tests pay off and add a lot of.value, namely in avoiding needless regressions from Rambo developers who want to change code without caring if it even works.
    • Normal_gaussian 5y ago
      > unit tests verify a contract This is the most important thing. Rambo can delete all the unit tests he wants in his MR; our discussion will be about whether: - the contracts have changed. - the unit tests shouldn't have been there in the first place. - he's violating contracts
  • kazinator 5y ago
    Test the heck out of the external, documented contracts. Then the test suite helps you refactor: reshuffle the internals while maintaining the contracts.
  • lowbloodsugar 5y ago
    If unit tests are hindering your refactoring then you aren't writing SOLID code. The only time I find myself, e.g. splitting a class in two, and then having to deal with its shitty tests, are when I'm dealing with legacy classes that did five things at once. Oh, and their tests were just "did it crash?" tests, i.e. call a method but only check the happy path. If you have reached the point where you feel tests hinder refactoring then you have two paths: a/ learn to write smaller, SOLID classes, or b/ meh unit tests are stoopid.
  • Aeolun 5y ago
    Lolwut? Unit tests are one of the few things that give me confidence in my refactored code. Integration tests are nice, but they don’t give me enough confidence about the specific part of the system I’m altering.
    • bluGill 5y ago
      Until you realize that your api is wrong. So long as you got your api right up front unit tests make it easy to refactor. As soon as the api needs nontrivial changes you are sunk.
      • grayclhn 5y ago
        Cool, but that’s not “refactoring.”
      • Aeolun 5y ago
        True, to some extend. But the number of times I need to do that is low compared to the number of times I need to do a much milder refactor.
      • JoshuaDavid 5y ago
        If your api is wrong, and you have unit tests for that wrong api, and you have time to rewrite it the right way, you can write a set of methods with the correct api which is a shim for the wrong api (you don't care about perf here), write your refactored version, and switch your now-tested shim to the correct api (and then probably keep it as a reference for the new way of doing things). If you find yourself doing this frequently though unit tests might not be ideal for your situation.
• rualca 5y ago
  > I think people have learned to undervalue the properties of integration tests and overvalue the properties of unit tests. Is it nice to know exactly what's broken based on the test failure? Sure. Is it _as_ nice as having confidence that the whole system is working? Probably not, in a lot of cases. This comment alone demonstrates a collosal misunderstanding regarding unit tests. Unit tests are not used to verify if the system is working. They were never used for that. At all. Unit tests only specify invariants which are a necessary but insufficient to verify that the system works as expected. These invariants are used to verify that specific realizations of your input and state will lead your implementation to produce specific results, no matter which code you change. If you expect function foo to return true if you pass it a positive integer, you want to know if a code change suddenly leads it to return false or throw an exception. That's what unit tests are for. If unit tests worked anything remotely similar to the way you misrepresented them, we would not need integration or end-to-end tests at all, would we?
  • alecbz 5y ago
    Are you saying unit tests are not meant (in part) to detect bugs?
• imiric 5y ago
  I think that a well covered codebase should include mostly unit tests, plenty of integration tests, and few E2E tests, i.e. model the test pyramid[1]. Mocking in unit tests is generally acceptable to avoid also testing other systems. E.g. if the function does OS or network calls it's a good idea to mock that and keep the test isolated. But you should also have integration tests for the feature which do make external calls without mocking (or mocking at a higher level), as well as E2E tests that test the system as a whole. This way you can generally quickly pinpoint where something breaks, which may or may not break higher level tests. [1]: https://martinfowler.com/bliki/TestPyramid.html https://martinfowler.com/bliki/TestPyramid.html
• fragbait65 5y ago
  I think it's a failure to ever believe that unit tests provide any correctness proof. The only thing you achieve with unit tests is to prove that you are still bug for bug compatible when changing your code. I think experience is needed to know when to unit test. Some code might not need any tests at all while other code might need quite a lot of tests. My personal experience, for the code bases I work on, shows that for those code bases mocking is usually not beneficial, we find more problems with integration testing and fuzzing. That holds for those code bases, it might very well be different for other code bases, and here's where experience comes in again...
  • lugged 5y ago
    > The only thing you achieve with unit tests is to prove that you are still bug for bug compatible when changing your code. People love to ignore how fucking awful code is when you aren't forced to write unit tests. Unit tests before you write code help you code, they provide a tight feedback loop, they force or at least encourage modularity to be testable. Adding tests to my code after writing it forces me to challenge assumptions and frequently requires refactoring to improve the code along with testability. It also documents my intent better than comments can and often gives consumers valuable example usage code. Yes it's a trade off and sometimes overkill but that's why 100% coverage is a stupid goal. You should always evaluate RoI of the tests you're writing. Integration and end to end testing is much the same, they're simply different grain sizes, we need all of them to be effective. The biggest mistake that happens with testing is assuming it's all about correctness, doing so is why we have 100% coverage people around, they're equating 100% coverage with 100% correct which is just a wrong conclusion.
    • ncrmro 5y ago
      Quite iteration/feedback loop is a big plus for sure
    • fragbait65 5y ago
      I used to do TDD, I found that it didn't work that well for me. I still think that the code base you work on dictates how you should be testing. It also dictates which testing strategy I use. I test for different things when writing C than I do when I write Python for example. My testing strategy is different if I write a networked C application compared to if I write a Ruby on Rails app. Also the tools I have available to me when writing the code dictates how I will test, which is tied into which language I write in. And I really disagree that code has to be awful because you are not forced to write unit tests. I work with a team of 4 other experienced and responsible programmers. We actually don't have to have rules that force us to do anything. We are responsible enough and experienced enough to know what to do and when to do it. 40 - 50 % of our code base doesn't have a single unit test, because it doesn't have to have any. Are we infallible, no. Does mistakes happen? Yes. Do we sometimes go back and add unit tests to code that we thought didn't need any. Yes, it happens. Would unit tests have saved us sometimes? Yes. If it would have, then we are responsible and go back and add that test.
• dnrjdjxixi 5y ago
  Unit tests are for TDD. They make it difficult to write untestable code. They rarely catch bugs, they are your spec.
• bluGill 5y ago
  1 and 3 is the dogma, but mostly I find it false. My integration tests are fast because they deal with small data sets. A few dozen function calls adds a few ms, who cares. I find it easy to find where an integration tests broke because I back out my last change.
• teddyh 5y ago
  I’ve seen some people make the distiction between “unit” tests and “programmer” tests. Using this terminology, “programmer” tests are what are used in TDD; they help with development and cover the desired functionality, not implementation. When code is refactored internally, no such tests should break. “Unit” tests, on the other hand, are called such because they test an isolated unit of code, and are frequently heavlily mocking all its dependencies, both internal and external. These tests, when created, exist for code quality purposes, and are almost always tightly coupled to the implementation, and must be rewritten when the implementation changes.
• sporkland 5y ago
  Having noodled on these comments for a few, I share your view, but I'm curious if the type of testing varies based on seniority/carefulness. I find people earlier in their careers (or people with less care) write code where the failures at boundary conditions are obvious to me, but not to them. I can see unit tests (or quick check) paying big dividends with that group while being lower ROI for more experienced / more careful teams.