5 ms·

It depends on what you unit test and why. If it's for 100% test coverage: forget about it. If you test private methods: you're doing something wrong. What pe

by arkh 9y ago

It depends on what you unit test and why. If it's for 100% test coverage: forget about it. If you test private methods: you're doing something wrong. What people usually see from the "unit test evangelists" are codebase for which you have tests for every method in the code. Then you do some refactoring and you have to rewrite tons of tests. And as those tests are just made to get 100% coverage you end-up with logic bugs because most unit tests have been written to go through the code, not check limits and edge-case. When you stumble upon this kind of test harness you can only think this as only cons (more to write upfront, less willingness to refactor) and no pro (the code is still brittle). Then your integration tests feel like your real harness: you can change anything in your code it'll tell you what has been broken when used. Now if you consider your unit tests as a kind of integration tests for the API your classes present then you get the benefits of unit tests. But this mean testing only public methods. And mutation testing resilience is a better metric than test coverage. Also: those tests do not replace a real documentation which can be a lot faster to read and understand than code.

josteink 9y ago
> If it's for 100% test coverage: forget about it. Im a realist and don’t see any point or value in chasing that goal for a 20+ year old company code-base. But I expect new modules to be fundamentally testable. > if you test private methods: you're doing something wrong. Agreed. > What people usually see from the "unit test evangelists" are codebase for which you have tests for every method in the code. Is that really so? It’s easy to be opposed to extremists of any form. I only “evangelize” that business-logic should be tested and thus needs a code structure to isolate the business-logic from its dependencies (databases, services, factories, etc). I find that perfectly reasonable.
growse 9y ago
> If you test private methods: you're doing something wrong. Maybe a silly question, but why? If I refactor a class to pull a common piece of functionality into a private method, why would I not want a test for that? One of the principle benefits of tests I see is allowing me to change the implementation without worrying about the behaviour, and I'm not sure why that wouldn't apply to private methods?
- josteink 9y ago
  I read it as using introspection/reflection to test what is essentially implementation details which are very likely to change. This is how you write brittle tests which fail easily and cause high maintenance costs and reduced confidence in the unit-tests as a safety net. Definitely an anti-pattern.
- arkh 9y ago
  Private methods are the internals of your classes. It may change a lot for performance or to make it easy to maintain, one method may become 3 or 4. But people who use your class don't care. They input something in your public methods and expect something in return. The details of what happen inside should not matter. Adding tests there only help to slow you down and make the dev team resist needed changes. And when you add tests you increase the chances you make them useless or wrong.
  - growse 9y ago
    Ah, I think I see. If I break the functionality by changing the implementation of a private class, that should be reflected in the public API unit tests.
    - Marazan 9y ago
      That's how I see it (in Java at least), unit tests are for guaranteeing that your classes API does what it says it does. In Python I am more loosely goosey about my unit tests and unit tests there are more for helping me write/think about tricky code.
- weavie 9y ago
  If your private method is wrong, then your public methods will also be wrong. If your public methods are right, then it doesn't really matter what your private methods do..
- chewbacha 9y ago
  One reason why is because you should be testing the public behavior of a function/class not the details. The reason for this is because the public interface is what other parts of the codebase will come to rely on. Refactoring generally shouldn’t change the public interface as it will break other pieces of code within your codebase, or other codebases if it’s a library, and other systems if it’s a network api. So, if you test the public interface, generally refactors won’t break the tests. Testing private functions also seems to be a smell that the overall setup of testing the class or function is too difficult. This can be because the class has too many branches in it, the argument list is too large, or too many other systems must be in place for it to function correctly. This, to me, indicates a public interface that is hard to use and will pass much of these issues on to the caller. Lastly, if you are testing private functions to gain coverage then arguably the behavior in the private method isn’t actually useful to the public interface. The reason I say this is that testing the behavior of the class should end up touching all branch conditions inside the class or the public interface isn’t fully tested. By only testing the public interface it then also becomes easier to locate dead/unreachable code. Hope that answers the why.
BlackFly 9y ago
People test private methods because edge cases occur in those private methods and the test for the edge cases do not belong in the unit test for the consumer of the private unit. If the consumer simply loops over a list of objects which it receives from the private unit, the consumer does not need to know that particular integer arguments are special cases in the private unit; that would be a leaky abstraction. However, it still makes sense to verify that you have correctly handled each special case via a unit test. As for the difficulty or refactoring, if you refactor the private unit, you ensure that its tests continue to pass, since its consumers depend on that behavior: you ignore the failing tests of the consumers so long as the subordinate unit's tests are failing. If you eliminate the private unit, you eliminate its tests. Modifying the behavior of the private unit may be equivalent to eliminating the unit or refactoring it. The number of tests you will have to modify is equal to the number of units you modified the behavior of: the branch count of the private unit, or that plus its consumers. If each private consumer was responsible for testing all of the edge cases of the private unit, then you will have to change its branch count multiplied by the number of consumers worth of tests instead. The distinction between private and public is wholly synthetic. It is a binary layering mechanism that does not map well onto most architectures which have many layers. From the perspective of many full architectures, everything down in the data layer is private: no customer will have direct access to the data layer. Yet you will still test the data layer. The internals of a library are not special simply because the layering is thinner and binary.
- andrewaylett 9y ago
  My general theory is that if a private method is complex enough to need separate testing, it's usually complex enough to pull out into its own class and test as a separate public interface. That's 'interface' as in 'what a class exposes to its callers', not necessarily using an actual Java interface or making it part of the public API of the library. A side-benefit is that the tests for the original class can be a lot simpler too, as I can just mock the responses from what used to be the class internals. Another benefit for libraries, is that it allows consumers to swap out your implementation. I've lost track of the times I've wanted to change something deep inside a library but it's implemented in a private method so I can't even override it. This does lead to more, smaller, class files. But unless taken to extremes I've not found it to make things less comprehensible, and it definitely makes things more composable.
mikegerwitz 9y ago
> Then you do some refactoring and you have to rewrite tons of tests. One of the core principles of TDD is that you write tests to facilitate refactoring---to make it _easy_ to refactor and have confidence in the system after doing so. I've been practicing TDD for ~8y and this is rarely a problem. TDD encourages good abstractions and architecture that lends itself well to composition and separation of concerns. But if you change an implementation, of course a test is going to fail---you broke the code. It works exactly as designed. What you want to do is change the test first to reflect the new, desired implementation. Any tests that fail that you didn't expect to fail may represent a bug in your implementation. Of course, I haven't seen the code, so I can't comment on it, and I won't try to do so.
- discreteevent 9y ago
  "What you want to do is change the test first to reflect the new, desired implementation". Not sure if you meant this but this is exactly what is wrong with most unit tests that I have come across. They test the implementation and not the interface. That's why I agree that the focus should mainly be on integration tests. Or at least functional tests. Ideally what you want is to have a system where all the state and logic is in a model (that model can include an external db). The gui should be as much as possible a function of the model i.e. model-view. Then you write the majority of your tests as integration tests against the model and include as many scenarios as you can think of. These tests should reflect the requirements/interface for the system and not the implementation. You should write some gui tests but these should be much less. They just need to verify that the ui reflects the model accurately. You shouldn't be testing scenarios as part of the gui tests. I have come across too many code bases where the unit tests test that the code is what it is, rather than the code does what it should. Where 'what it should' == 'requirements/interface' == 'something close to integration tests'

by arkh 9y ago

josteink 9y ago

> If it's for 100% test coverage: forget about it. Im a realist and don’t see any point or value in chasing that goal for a 20+ year old company code-base. But I expect new modules to be fundamentally testable. > if you test private methods: you're doing something wrong. Agreed. > What people usually see from the "unit test evangelists" are codebase for which you have tests for every method in the code. Is that really so? It’s easy to be opposed to extremists of any form. I only “evangelize” that business-logic should be tested and thus needs a code structure to isolate the business-logic from its dependencies (databases, services, factories, etc). I find that perfectly reasonable.

growse 9y ago

> If you test private methods: you're doing something wrong. Maybe a silly question, but why? If I refactor a class to pull a common piece of functionality into a private method, why would I not want a test for that? One of the principle benefits of tests I see is allowing me to change the implementation without worrying about the behaviour, and I'm not sure why that wouldn't apply to private methods?

josteink 9y ago
I read it as using introspection/reflection to test what is essentially implementation details which are very likely to change. This is how you write brittle tests which fail easily and cause high maintenance costs and reduced confidence in the unit-tests as a safety net. Definitely an anti-pattern.
arkh 9y ago
Private methods are the internals of your classes. It may change a lot for performance or to make it easy to maintain, one method may become 3 or 4. But people who use your class don't care. They input something in your public methods and expect something in return. The details of what happen inside should not matter. Adding tests there only help to slow you down and make the dev team resist needed changes. And when you add tests you increase the chances you make them useless or wrong.
- growse 9y ago
  Ah, I think I see. If I break the functionality by changing the implementation of a private class, that should be reflected in the public API unit tests.
  - Marazan 9y ago
    That's how I see it (in Java at least), unit tests are for guaranteeing that your classes API does what it says it does. In Python I am more loosely goosey about my unit tests and unit tests there are more for helping me write/think about tricky code.
weavie 9y ago
If your private method is wrong, then your public methods will also be wrong. If your public methods are right, then it doesn't really matter what your private methods do..
chewbacha 9y ago
One reason why is because you should be testing the public behavior of a function/class not the details. The reason for this is because the public interface is what other parts of the codebase will come to rely on. Refactoring generally shouldn’t change the public interface as it will break other pieces of code within your codebase, or other codebases if it’s a library, and other systems if it’s a network api. So, if you test the public interface, generally refactors won’t break the tests. Testing private functions also seems to be a smell that the overall setup of testing the class or function is too difficult. This can be because the class has too many branches in it, the argument list is too large, or too many other systems must be in place for it to function correctly. This, to me, indicates a public interface that is hard to use and will pass much of these issues on to the caller. Lastly, if you are testing private functions to gain coverage then arguably the behavior in the private method isn’t actually useful to the public interface. The reason I say this is that testing the behavior of the class should end up touching all branch conditions inside the class or the public interface isn’t fully tested. By only testing the public interface it then also becomes easier to locate dead/unreachable code. Hope that answers the why.

BlackFly 9y ago

People test private methods because edge cases occur in those private methods and the test for the edge cases do not belong in the unit test for the consumer of the private unit. If the consumer simply loops over a list of objects which it receives from the private unit, the consumer does not need to know that particular integer arguments are special cases in the private unit; that would be a leaky abstraction. However, it still makes sense to verify that you have correctly handled each special case via a unit test. As for the difficulty or refactoring, if you refactor the private unit, you ensure that its tests continue to pass, since its consumers depend on that behavior: you ignore the failing tests of the consumers so long as the subordinate unit's tests are failing. If you eliminate the private unit, you eliminate its tests. Modifying the behavior of the private unit may be equivalent to eliminating the unit or refactoring it. The number of tests you will have to modify is equal to the number of units you modified the behavior of: the branch count of the private unit, or that plus its consumers. If each private consumer was responsible for testing all of the edge cases of the private unit, then you will have to change its branch count multiplied by the number of consumers worth of tests instead. The distinction between private and public is wholly synthetic. It is a binary layering mechanism that does not map well onto most architectures which have many layers. From the perspective of many full architectures, everything down in the data layer is private: no customer will have direct access to the data layer. Yet you will still test the data layer. The internals of a library are not special simply because the layering is thinner and binary.

andrewaylett 9y ago
My general theory is that if a private method is complex enough to need separate testing, it's usually complex enough to pull out into its own class and test as a separate public interface. That's 'interface' as in 'what a class exposes to its callers', not necessarily using an actual Java interface or making it part of the public API of the library. A side-benefit is that the tests for the original class can be a lot simpler too, as I can just mock the responses from what used to be the class internals. Another benefit for libraries, is that it allows consumers to swap out your implementation. I've lost track of the times I've wanted to change something deep inside a library but it's implemented in a private method so I can't even override it. This does lead to more, smaller, class files. But unless taken to extremes I've not found it to make things less comprehensible, and it definitely makes things more composable.

mikegerwitz 9y ago

> Then you do some refactoring and you have to rewrite tons of tests. One of the core principles of TDD is that you write tests to facilitate refactoring---to make it _easy_ to refactor and have confidence in the system after doing so. I've been practicing TDD for ~8y and this is rarely a problem. TDD encourages good abstractions and architecture that lends itself well to composition and separation of concerns. But if you change an implementation, of course a test is going to fail---you broke the code. It works exactly as designed. What you want to do is change the test first to reflect the new, desired implementation. Any tests that fail that you didn't expect to fail may represent a bug in your implementation. Of course, I haven't seen the code, so I can't comment on it, and I won't try to do so.

discreteevent 9y ago
"What you want to do is change the test first to reflect the new, desired implementation". Not sure if you meant this but this is exactly what is wrong with most unit tests that I have come across. They test the implementation and not the interface. That's why I agree that the focus should mainly be on integration tests. Or at least functional tests. Ideally what you want is to have a system where all the state and logic is in a model (that model can include an external db). The gui should be as much as possible a function of the model i.e. model-view. Then you write the majority of your tests as integration tests against the model and include as many scenarios as you can think of. These tests should reflect the requirements/interface for the system and not the implementation. You should write some gui tests but these should be much less. They just need to verify that the ui reflects the model accurately. You shouldn't be testing scenarios as part of the gui tests. I have come across too many code bases where the unit tests test that the code is what it is, rather than the code does what it should. Where 'what it should' == 'requirements/interface' == 'something close to integration tests'