4 ms·

The * fire your QA team * dev team is the level 2 production support, and * get to continuous integration nirvana management fads have been sweeping t

The * fire your QA team * dev team is the level 2 production support, and * get to continuous integration nirvana management fads have been sweeping through my Scrum enterprise for the last 18 months. Teams that aren't testing constantly, well, they've got tons of escape defects on every release. And those devs are constantly in fire-fighting mode, it's miserable for them. And I see that leading to compressed schedules for them and more reckless behavior like asking to push their releases during the holidays where there could be severe financial consequences to bugs. As far as I'm concerned, in an environment like mine, where developers can no longer hide their incompetence behind bureaucracy like a QA team, it is official insanity to not spend inordinate amounts of development time writing automated tests. You should be spending 70% of your dev time writing tests and doing devops and 30% writing features. I read in these comments a lot of bellyaching about how much time it takes to write tests. First, TDD is a skill that you can get good at, and it won't take as much time as you think once you get good. Second, I just don't think you have a choice to not test comprehensively when escape defects become a mark of shame in the organization.

hirsin 10y ago
Potentially contentious opinion, but one that's been echoed through our halls - Developing software and developing tests and test infrastructure are two different skills and mindsets that are often inversely coupled. It's not that "incompetence" is revealed by firing the test team (although it sometimes is) - it's that "being bad at writing tests" is revealed. A team with 3 dedicated testers and 7 devs will probably outperform (in code output and reliability) 10 devs spending 70% of their time on testing.
- mercurial 10y ago
  Not to mention that there is a lot of stuff, especially for UI, that is either hard or impossible to write tests for...
- ajmurmann 10y ago
  I've actually found that most QA people I've worked with were good at deciding what to test but were awful at writing automated tests. Of course that's purely anecdotal and based on a very small sample size. I like to believe that I sometimes will forget to test for edge cases, but have mastered how to write tests. I don't think that how to write tests is easily acquired by any descend programmer. However, you need to learn a few patterns you otherwise won't need.
  - dasmoth 10y ago
    The real strength of QA people can be manual testing. There's a local maximum of "meets the stated requirements but horrid to actually use", and a singular focus on test automation seems to push projects in that direction.
    - ajmurmann 10y ago
      Totally agree. I think they are great for what some people refer to as "exploratory testing".
- kodfodrasz 10y ago
  I have experienced the opposite. After the dedicated QA team was disbanded and repurposed to development, where every developer had to write tests for someone else's code, the code and designs started to become more testable, the tests eventually became simpler to maintain and understand. (the project was and embedded system) The "We" and "Them" distinctions made people in different teams and different roles ignore the needs of the others. The change brought a culture change and also gave a view on the needs of the other side. This could make it possible that for the first time the development of tests and features could really be done in parallel (officially that was the methodology before, but never really worked out well, because the testability considerations were usually ignored at design time, and the docs were lagging, because of the bad bandwagoning culture. With the mixed team where everyone was treated as an equal these problems dissolved surprisingly quickly (in less than half a year)). So my point is having dedicated testers can give base for bad culture which hurts the product and the company. Having everyone do the same job with regards to development and testing is better.
  - hvidgaard 10y ago
    There is no silver bullet here. For some teams it makes sense to integrate development and testing. For some teams it makes sense to have dedicated QA people. For some teams a different constellation is optimal. I know developers that are brilliant at the big picture, finishing the implementation however was lacking. I also know developers that cannot get the major architecture right, but they can finish tasks and get it shipped. pick one of each and a devops guy that can write tests, and you have a team of 3 that produce top quality software.
    - kodfodrasz 10y ago
      You are probably right with your example for a small project. The one I referred was a medium sized safety critical piece with dev/test staff of hundreds. My example was for such larger organization. Small teams almost always worked out well for me if a single leader person was present to sort out initial problems.
falsedan 10y ago
> You should be spending 70% of your dev time writing tests and doing devops and 30% writing features. I laugh, but the tears are real.
mehwoot 10y ago
The study is about the effects of "test first" vs "test later", not "writing tests" vs "not having tests".
- EdSharkey 10y ago
  Yeah, I tried to avoid specifying a methodology. I just couldn't resist throwing in a 'TDD' because it turns better quality results for me personally than 'test later'. My point in commenting was: testing comprehensively is not optional at my workplace due to the way the devs have suddenly been held so accountable. Personally, I don't care how you test. Just get decent coverage over all the control paths and have a test suite that catches regression bugs. Ideally also have integration tests to prove your connections to your partners are good and maybe a smoke test to prove your code is wired up right once it hits an environment. I'm still on the journey myself, so I can't claim to be an expert. I know there are people in my org that are suffering because of code quality issues. I call them incompetent because they're producing shoddy work products with lots of defects. A smart way to combat bugs and kill them for good is to test, so that's why I connect not testing to incompetence. A dumb way to combat bugs is heroics and working 18 hour work days trying to debug with "GOT HERE" logging.
omouse 10y ago
>As far as I'm concerned, in an environment like mine, where developers can no longer hide their incompetence behind bureaucracy like a QA team, it is official insanity to not spend inordinate amounts of development time writing automated tests. You should be spending 70% of your dev time writing tests and doing devops and 30% writing features. Highly agreed, anytime there has been QA available, the code base starts to look more hacky and fewer tests are written and only the tests that cover the happy path are written. When things are looking bleak, that's when I start to think we should have certifications for developers, optional of course, like the PMP or CAPM, but they should be there as a rough indicator that a dev won't skimp on QAing their own code. >I read in these comments a lot of bellyaching about how much time it takes to write tests. First, TDD is a skill that you can get good at, and it won't take as much time as you think once you get good. Second, I just don't think you have a choice to not test comprehensively when escape defects become a mark of shame in the organization. After years of looking at tests written poorly, I've come to the conclusion that you have to treat test code like your regular code and do code reviews on it and refactor the heck out of it. If you don't, you'll end up in the situation I'm in, where our company has almost 4000 spec tests in ruby/rails that can take an hour to run on a local dev box and still take 20 min when parallelized across 5 boxes. Shaving off a couple of seconds here and there by removing unneeded tests or refactoring to simplify them or to share mock objects is a worthwhile endeavour but because they're "just tests", the time to do that isn't allocated and everyone on the team merrily continues to write and modify tests in a sloppy way.

hirsin 10y ago

Potentially contentious opinion, but one that's been echoed through our halls - Developing software and developing tests and test infrastructure are two different skills and mindsets that are often inversely coupled. It's not that "incompetence" is revealed by firing the test team (although it sometimes is) - it's that "being bad at writing tests" is revealed. A team with 3 dedicated testers and 7 devs will probably outperform (in code output and reliability) 10 devs spending 70% of their time on testing.

mercurial 10y ago
Not to mention that there is a lot of stuff, especially for UI, that is either hard or impossible to write tests for...
ajmurmann 10y ago
I've actually found that most QA people I've worked with were good at deciding what to test but were awful at writing automated tests. Of course that's purely anecdotal and based on a very small sample size. I like to believe that I sometimes will forget to test for edge cases, but have mastered how to write tests. I don't think that how to write tests is easily acquired by any descend programmer. However, you need to learn a few patterns you otherwise won't need.
- dasmoth 10y ago
  The real strength of QA people can be manual testing. There's a local maximum of "meets the stated requirements but horrid to actually use", and a singular focus on test automation seems to push projects in that direction.
  - ajmurmann 10y ago
    Totally agree. I think they are great for what some people refer to as "exploratory testing".
kodfodrasz 10y ago
I have experienced the opposite. After the dedicated QA team was disbanded and repurposed to development, where every developer had to write tests for someone else's code, the code and designs started to become more testable, the tests eventually became simpler to maintain and understand. (the project was and embedded system) The "We" and "Them" distinctions made people in different teams and different roles ignore the needs of the others. The change brought a culture change and also gave a view on the needs of the other side. This could make it possible that for the first time the development of tests and features could really be done in parallel (officially that was the methodology before, but never really worked out well, because the testability considerations were usually ignored at design time, and the docs were lagging, because of the bad bandwagoning culture. With the mixed team where everyone was treated as an equal these problems dissolved surprisingly quickly (in less than half a year)). So my point is having dedicated testers can give base for bad culture which hurts the product and the company. Having everyone do the same job with regards to development and testing is better.
- hvidgaard 10y ago
  There is no silver bullet here. For some teams it makes sense to integrate development and testing. For some teams it makes sense to have dedicated QA people. For some teams a different constellation is optimal. I know developers that are brilliant at the big picture, finishing the implementation however was lacking. I also know developers that cannot get the major architecture right, but they can finish tasks and get it shipped. pick one of each and a devops guy that can write tests, and you have a team of 3 that produce top quality software.
  - kodfodrasz 10y ago
    You are probably right with your example for a small project. The one I referred was a medium sized safety critical piece with dev/test staff of hundreds. My example was for such larger organization. Small teams almost always worked out well for me if a single leader person was present to sort out initial problems.

falsedan 10y ago

> You should be spending 70% of your dev time writing tests and doing devops and 30% writing features. I laugh, but the tears are real.

mehwoot 10y ago

The study is about the effects of "test first" vs "test later", not "writing tests" vs "not having tests".

EdSharkey 10y ago
Yeah, I tried to avoid specifying a methodology. I just couldn't resist throwing in a 'TDD' because it turns better quality results for me personally than 'test later'. My point in commenting was: testing comprehensively is not optional at my workplace due to the way the devs have suddenly been held so accountable. Personally, I don't care how you test. Just get decent coverage over all the control paths and have a test suite that catches regression bugs. Ideally also have integration tests to prove your connections to your partners are good and maybe a smoke test to prove your code is wired up right once it hits an environment. I'm still on the journey myself, so I can't claim to be an expert. I know there are people in my org that are suffering because of code quality issues. I call them incompetent because they're producing shoddy work products with lots of defects. A smart way to combat bugs and kill them for good is to test, so that's why I connect not testing to incompetence. A dumb way to combat bugs is heroics and working 18 hour work days trying to debug with "GOT HERE" logging.

omouse 10y ago

>As far as I'm concerned, in an environment like mine, where developers can no longer hide their incompetence behind bureaucracy like a QA team, it is official insanity to not spend inordinate amounts of development time writing automated tests. You should be spending 70% of your dev time writing tests and doing devops and 30% writing features. Highly agreed, anytime there has been QA available, the code base starts to look more hacky and fewer tests are written and only the tests that cover the happy path are written. When things are looking bleak, that's when I start to think we should have certifications for developers, optional of course, like the PMP or CAPM, but they should be there as a rough indicator that a dev won't skimp on QAing their own code. >I read in these comments a lot of bellyaching about how much time it takes to write tests. First, TDD is a skill that you can get good at, and it won't take as much time as you think once you get good. Second, I just don't think you have a choice to not test comprehensively when escape defects become a mark of shame in the organization. After years of looking at tests written poorly, I've come to the conclusion that you have to treat test code like your regular code and do code reviews on it and refactor the heck out of it. If you don't, you'll end up in the situation I'm in, where our company has almost 4000 spec tests in ruby/rails that can take an hour to run on a local dev box and still take 20 min when parallelized across 5 boxes. Shaving off a couple of seconds here and there by removing unneeded tests or refactoring to simplify them or to share mock objects is a worthwhile endeavour but because they're "just tests", the time to do that isn't allocated and everyone on the team merrily continues to write and modify tests in a sloppy way.