5 ms·

I work in a company where quite a few of the developers simply are incapable of writing anything but integration-tests. The reason? They don’t “believe” in uni

I work in a company where quite a few of the developers simply are incapable of writing anything but integration-tests. The reason? They don’t “believe” in unit-tests. They don’t think unit-testing “works in the real world”. They absolutely fail to accept that they need to write their code differently for automated testing to work well. How do you change such a mindset?

hvidgaard 9y ago
I doubt that you can. There was a study a while back, and I apologize in advance because I do not have a link, that showed projects written with unittests took significantly longer to reach the market, but with significantly less bugs. However, overall time spend on the code was less. So conclusion was that unittests are a commitment to a long term goal of minimizing developer time, and the tradeoff is that it takes longer for the first version to be done. That is, as far as I know, the only tangible evidence that unittests are good unless you need to get something out the door quickly (which sadly is most of it). I'd argue that is not the main benefit of unittesting however. That is the way code is structured, and especially how dependencies are explicit, e.g. injected with constructor arguments.
- chewbacha 9y ago
  Man, I really want to read that study!
- hinkley 9y ago
  That parallels my experiences. I got tired really early on with projects that ground to a halt because of brittleness and a lot of my focus is on building skill and confidence so that version 3 is no harder to ship than version 2 was. Every team I’ve left on good terms was more effective when I left than when I got there. The ones that fought me the whole way frustrate me and it shows.
- alien_at_work 9y ago
  Was this the microsoft study? I think it was a breakdown of their delivery of Vista?
xab9 9y ago
Trying to "change" others' mindset is good recipe for frustration.
diggernet 9y ago
Perhaps a well written, easy to follow guide on how to structure different types of code to simplify testing would be helpful. Know of any?
- davidjnelson 9y ago
  Object Oriented Software Guided By Tests is good.
- arkh 9y ago
  Working Effectively with Legacy Code is a good read: it presents what kind of code you want to attain and methods to get there from a crappy code base. The definition of legacy code for the author (which I like) is: untested code. So the book is more about getting code in a testable state than random refactoring to get to Clean Code level.
  - josteink 9y ago
    That book is indeed good, at least on a personal level. It has helped me refine what I consider good code and good effort wrt to testing.
flukus 9y ago
> How do you change such a mindset? IME you can't. If they even recognize that what they are doing is not unit testing then your doing well. > They absolutely fail to accept that they need to write their code differently for automated testing to work well. I've been thinking lately that having to code differently might be a fault of the tooling that's built up over the years and that they might be right. I've been getting back into c lately and had a look at the mock/stub options there which were very complicated and not very compelling compared to what I've been used to in the .net world. In the end I found the pre-processor was the best (for my project) option: #ifdef test #define some_method mock_some_method #endif The advantage has been that the code is written exactly (more or less) as it would have been if there were no tests. There are no interfaces to add, functions to virtualize or un-static and no dependencies to add, this all translates to no performance hit in the production code and the project being simpler all around.
NotSammyHagar 9y ago
Sounds like Tableau in Seattle.
spc476 9y ago
For some reason, unit tests vs. integration tests reminds me of this image: https://pbs.twimg.com/media/CZX0O-tWQAAeaLi.jpg https://pbs.twimg.com/media/CZX0O-tWQAAeaLi.jpg The unit tests past, but why bother with integration tests? I wrote a component at work. Sure, there are some unit tests (the SIP parser working? Check. But for that component, unit tests only go so far as I need to query another program that actually implements the business logic (it goes SIP -> my program -> custom protocol [1] -> business logic program and back again). To mock the business logic program (I need to make sure to return the proper information per the SIP request) is to reimplement the business logic program, so when testing my component, we also use the business logic unit. The business logic also requires two more programs to run. At this point, there is no difference between a unit test and an integration test as I'm running five programs ... no, ... six, I do have to mock a cell phone (basically, respond to a custom protocol and make a web request), to test the "unit" that is my program. Oh, and to make it even nicer, this is legacy C and C++ (C with classes) code. [1] Legacy code. It works. At the rate of production deployments we (our team) gets, it would be around two years [2] to remove the custom protocol. So it stays. [2] Have I mentioned the very scary SLAs?
arkh 9y ago
It depends on what you unit test and why. If it's for 100% test coverage: forget about it. If you test private methods: you're doing something wrong. What people usually see from the "unit test evangelists" are codebase for which you have tests for every method in the code. Then you do some refactoring and you have to rewrite tons of tests. And as those tests are just made to get 100% coverage you end-up with logic bugs because most unit tests have been written to go through the code, not check limits and edge-case. When you stumble upon this kind of test harness you can only think this as only cons (more to write upfront, less willingness to refactor) and no pro (the code is still brittle). Then your integration tests feel like your real harness: you can change anything in your code it'll tell you what has been broken when used. Now if you consider your unit tests as a kind of integration tests for the API your classes present then you get the benefits of unit tests. But this mean testing only public methods. And mutation testing resilience is a better metric than test coverage. Also: those tests do not replace a real documentation which can be a lot faster to read and understand than code.
- josteink 9y ago
  > If it's for 100% test coverage: forget about it. Im a realist and don’t see any point or value in chasing that goal for a 20+ year old company code-base. But I expect new modules to be fundamentally testable. > if you test private methods: you're doing something wrong. Agreed. > What people usually see from the "unit test evangelists" are codebase for which you have tests for every method in the code. Is that really so? It’s easy to be opposed to extremists of any form. I only “evangelize” that business-logic should be tested and thus needs a code structure to isolate the business-logic from its dependencies (databases, services, factories, etc). I find that perfectly reasonable.
- growse 9y ago
  > If you test private methods: you're doing something wrong. Maybe a silly question, but why? If I refactor a class to pull a common piece of functionality into a private method, why would I not want a test for that? One of the principle benefits of tests I see is allowing me to change the implementation without worrying about the behaviour, and I'm not sure why that wouldn't apply to private methods?
grey-area 9y ago
Assuming you are a developer, start writing some. Next bug you find that needs a unit/functional test (e.g. it is caused by a simple error in transformation in one function), write the test first as a table of inputs vs outputs, find it fails, fix the function, and leave the test in. Gradually, the code base will contain unit tests which are useful, people will see they are useful, and other people might start using them too where appropriate. You are unlikely to persuade them without actually doing what you say is beneficial and exposing others to its benefits.
- webignition 9y ago
  I agree. Tests for bug fixes are extremely valuable. Of such tests, unit tests are often very feasible. A test accompanying a bug fix holds value in many ways. Firstly, it demonstrates to those reviewing the change that the fix is suitable. Secondly, the presence of a test encourages reviewers to consider what a test does and doesn't cover, sometimes resulting in comments regarding improvements that had not otherwise been considered. Thirdly, and of most importance in the long term, a test for a bug fix serves to document oddities previously discovered that were for a time not known about.
  - mercutio2 9y ago
    I’m confident you know this, but just for the peanut gallery: Tests that go along with bug fixes are some of the highest value tests, but they must be previously failing tests. I can’t tell you how many times I’ve reviewed fixes with “tests” that are basically affirming the consequent; they assert something that was already true and it turns out they’re not actually fixing the reported bug.
profalseidol 9y ago
The keyword is "believe". Such behavior is not science. In science, you don't believe, you understand not faith. You check evidence, you use reason. A mindset that relies on "believe" but not on reason. It maybe because of decades of advertising. Can be a side-effect of participating in believe movements. How do you change such a mindset? For me, it was reading lots of philosophy and atheist vs theist debates. For Socrates, he died for it.

hvidgaard 9y ago

I doubt that you can. There was a study a while back, and I apologize in advance because I do not have a link, that showed projects written with unittests took significantly longer to reach the market, but with significantly less bugs. However, overall time spend on the code was less. So conclusion was that unittests are a commitment to a long term goal of minimizing developer time, and the tradeoff is that it takes longer for the first version to be done. That is, as far as I know, the only tangible evidence that unittests are good unless you need to get something out the door quickly (which sadly is most of it). I'd argue that is not the main benefit of unittesting however. That is the way code is structured, and especially how dependencies are explicit, e.g. injected with constructor arguments.

chewbacha 9y ago
Man, I really want to read that study!
hinkley 9y ago
That parallels my experiences. I got tired really early on with projects that ground to a halt because of brittleness and a lot of my focus is on building skill and confidence so that version 3 is no harder to ship than version 2 was. Every team I’ve left on good terms was more effective when I left than when I got there. The ones that fought me the whole way frustrate me and it shows.
alien_at_work 9y ago
Was this the microsoft study? I think it was a breakdown of their delivery of Vista?

xab9 9y ago

Trying to "change" others' mindset is good recipe for frustration.

diggernet 9y ago

Perhaps a well written, easy to follow guide on how to structure different types of code to simplify testing would be helpful. Know of any?

davidjnelson 9y ago
Object Oriented Software Guided By Tests is good.
arkh 9y ago
Working Effectively with Legacy Code is a good read: it presents what kind of code you want to attain and methods to get there from a crappy code base. The definition of legacy code for the author (which I like) is: untested code. So the book is more about getting code in a testable state than random refactoring to get to Clean Code level.
- josteink 9y ago
  That book is indeed good, at least on a personal level. It has helped me refine what I consider good code and good effort wrt to testing.

flukus 9y ago

> How do you change such a mindset? IME you can't. If they even recognize that what they are doing is not unit testing then your doing well. > They absolutely fail to accept that they need to write their code differently for automated testing to work well. I've been thinking lately that having to code differently might be a fault of the tooling that's built up over the years and that they might be right. I've been getting back into c lately and had a look at the mock/stub options there which were very complicated and not very compelling compared to what I've been used to in the .net world. In the end I found the pre-processor was the best (for my project) option: #ifdef test #define some_method mock_some_method #endif The advantage has been that the code is written exactly (more or less) as it would have been if there were no tests. There are no interfaces to add, functions to virtualize or un-static and no dependencies to add, this all translates to no performance hit in the production code and the project being simpler all around.

NotSammyHagar 9y ago

Sounds like Tableau in Seattle.

spc476 9y ago

For some reason, unit tests vs. integration tests reminds me of this image: https://pbs.twimg.com/media/CZX0O-tWQAAeaLi.jpg https://pbs.twimg.com/media/CZX0O-tWQAAeaLi.jpg The unit tests past, but why bother with integration tests? I wrote a component at work. Sure, there are some unit tests (the SIP parser working? Check. But for that component, unit tests only go so far as I need to query another program that actually implements the business logic (it goes SIP -> my program -> custom protocol [1] -> business logic program and back again). To mock the business logic program (I need to make sure to return the proper information per the SIP request) is to reimplement the business logic program, so when testing my component, we also use the business logic unit. The business logic also requires two more programs to run. At this point, there is no difference between a unit test and an integration test as I'm running five programs ... no, ... six, I do have to mock a cell phone (basically, respond to a custom protocol and make a web request), to test the "unit" that is my program. Oh, and to make it even nicer, this is legacy C and C++ (C with classes) code. [1] Legacy code. It works. At the rate of production deployments we (our team) gets, it would be around two years [2] to remove the custom protocol. So it stays. [2] Have I mentioned the very scary SLAs?

arkh 9y ago

It depends on what you unit test and why. If it's for 100% test coverage: forget about it. If you test private methods: you're doing something wrong. What people usually see from the "unit test evangelists" are codebase for which you have tests for every method in the code. Then you do some refactoring and you have to rewrite tons of tests. And as those tests are just made to get 100% coverage you end-up with logic bugs because most unit tests have been written to go through the code, not check limits and edge-case. When you stumble upon this kind of test harness you can only think this as only cons (more to write upfront, less willingness to refactor) and no pro (the code is still brittle). Then your integration tests feel like your real harness: you can change anything in your code it'll tell you what has been broken when used. Now if you consider your unit tests as a kind of integration tests for the API your classes present then you get the benefits of unit tests. But this mean testing only public methods. And mutation testing resilience is a better metric than test coverage. Also: those tests do not replace a real documentation which can be a lot faster to read and understand than code.

josteink 9y ago
> If it's for 100% test coverage: forget about it. Im a realist and don’t see any point or value in chasing that goal for a 20+ year old company code-base. But I expect new modules to be fundamentally testable. > if you test private methods: you're doing something wrong. Agreed. > What people usually see from the "unit test evangelists" are codebase for which you have tests for every method in the code. Is that really so? It’s easy to be opposed to extremists of any form. I only “evangelize” that business-logic should be tested and thus needs a code structure to isolate the business-logic from its dependencies (databases, services, factories, etc). I find that perfectly reasonable.
growse 9y ago
> If you test private methods: you're doing something wrong. Maybe a silly question, but why? If I refactor a class to pull a common piece of functionality into a private method, why would I not want a test for that? One of the principle benefits of tests I see is allowing me to change the implementation without worrying about the behaviour, and I'm not sure why that wouldn't apply to private methods?

grey-area 9y ago

Assuming you are a developer, start writing some. Next bug you find that needs a unit/functional test (e.g. it is caused by a simple error in transformation in one function), write the test first as a table of inputs vs outputs, find it fails, fix the function, and leave the test in. Gradually, the code base will contain unit tests which are useful, people will see they are useful, and other people might start using them too where appropriate. You are unlikely to persuade them without actually doing what you say is beneficial and exposing others to its benefits.

webignition 9y ago
I agree. Tests for bug fixes are extremely valuable. Of such tests, unit tests are often very feasible. A test accompanying a bug fix holds value in many ways. Firstly, it demonstrates to those reviewing the change that the fix is suitable. Secondly, the presence of a test encourages reviewers to consider what a test does and doesn't cover, sometimes resulting in comments regarding improvements that had not otherwise been considered. Thirdly, and of most importance in the long term, a test for a bug fix serves to document oddities previously discovered that were for a time not known about.
- mercutio2 9y ago
  I’m confident you know this, but just for the peanut gallery: Tests that go along with bug fixes are some of the highest value tests, but they must be previously failing tests. I can’t tell you how many times I’ve reviewed fixes with “tests” that are basically affirming the consequent; they assert something that was already true and it turns out they’re not actually fixing the reported bug.

profalseidol 9y ago

The keyword is "believe". Such behavior is not science. In science, you don't believe, you understand not faith. You check evidence, you use reason. A mindset that relies on "believe" but not on reason. It maybe because of decades of advertising. Can be a side-effect of participating in believe movements. How do you change such a mindset? For me, it was reading lots of philosophy and atheist vs theist debates. For Socrates, he died for it.