8 ms·

I'm in full agreement. In practise, a manual QA team encourages Devs to throw shit over the wall and expect someone else to do some basic sanity checks they sh

by jboy55 11y ago

I'm in full agreement. In practise, a manual QA team encourages Devs to throw shit over the wall and expect someone else to do some basic sanity checks they should have already done. By the time those are done, what the QA team theoretically could find gets shipped. Then, when its discovered in Prod, the QA team will get in the way of a speedy fix.

DougWebb 11y ago
That's a management problem, not a problem with having a QA team. The top-level QA manager and the top-level Dev manager should both report to the CTO, and the Dev manager should be judged by how many issues the QA team finds. To keep things fair, the QA manager should not be rewarded nor penalized based on the number of issues found pre-release, and everyone should be rewarded or penalized based on the number of issues found post-release. This keeps the devs incentivized to make sure everything works before the code goes to QA, and it keeps everyone incentivized to eliminate as many bugs as possible before release.
- fein 11y ago
  Have you ever worked for a fortune 100 company? The dev manager has probably never met the CTO. Im in this right now with my currrent job. I still keep my builds and test them longer than i should. Less broken shit gets through, but i regularly delay my qa ba's because of it.
  - DougWebb 11y ago
    So, not the CTO, but some shared higher-level manager. The point is that the QA lead cannot report into Development, and Development cannot report into QA. Both of those arrangements can lead to a bad actor stopping the other group from doing the right thing. (Eg: if QA reports to Dev, then Dev manager can force bad code to ship, and if Dev reports to QA than QA manager can prevent shipping and force Dev to waste time on excessive/pointless test code development that's not cost-justified.)
    - zhte415 11y ago
      I've met people that call something similar 'fake agile'. Surely everyone should report into the business (end-user on the business side) lead responsible for making this functionality happen. This includes Dev, QA, business. QA should be business? This is no longer 1985, there is no reason business (product owner) shouldn't be a capable PM and BA for their own product.
      - DougWebb 11y ago
        In the company I used to work for, the CTO and VP of Product Management were equals, both reporting to the CEO. The CTO's reports included the QA director, Dev director, and Project Management director. On any given project, we had a Product Manager who set the business requirements, a Project Manager who ran the project, a Dev lead in charge of the development team, and a QA lead in charge of QA. None of these people could override the others because of the reporting arrangement, so we had to collaborate instead to make sure we all met our goals. Early on this arrangement worked really well. But it started falling apart when ownership of the company changed and politics became a factor as people who cared more about moving up through the organization than about the success of the company and quality of the products were hired and promoted. My take is that bad management will destroy any approach you might take to producing good software. That's why I eventually left.
  - rikkus 11y ago
    I work for a FTSE 100 company and the dev manager and CTO in the division where I work are very well connected. I don't think the size of company is necessarily the only factor here.
- dragonwriter 11y ago
  > The top-level QA manager and the top-level Dev manager should both report to the CTO For an organization with a single product (and, particularly, one small enough that the "top-level" QA and Dev manager are also first level managers), this makes some sense. Otherwise, this means that each product has no common level of technical management below the CTO, meaning it has no meaningful technical ownership.
  - DougWebb 11y ago
    It doesn't have to be the CTO; as I mentioned in another reply my intent was that QA and Dev need to be equals in the reporting chain, rather than one reporting to the other. That can work at any scale; even on an agile team with one QA and one Dev, they both report to the same boss who's responsible for both sides.
    - knughit 11y ago
      Of you are fine tuning the reporting chain, your org is already hopelessly broken. That is the central planning Communist model.
- vkou 11y ago
  What will happen is that devs will ask their QA counterparts to report issues through an undocumented side-channel. If you give an engineer a career incentive to optimize something, you'd be surprised how seriously some will take it.
  - jboy55 11y ago
    Incentives often cause really nasty political wars within orgs. When a site gives a 500 because a database went down and the web app couldn't connect to it... is that a bug, a missed test case or should ops take a hit on down time? Furthermore, if you argue that the dev team should have reasonable failsafes in code to connect to a db, in a 10 year old organization, should the current Dev team pay for something that could have been in the code for years? If you set up a system where devs hand off to QA then they hand off to ops to deploy. Furthermore every step is incentivised somewhat against each other. Even if all teams are equal, it ends up in my opinion, to the path to CYA and Waterfall. Everyone is more concerned with problems not being 'their' fault than shipping good code.
    - jrumbut 11y ago
      You're absolutely right, assuming we're talking about business/consumer apps and not pacemakers or rocket ships. An incentive/penalty system for bugs without an incentive/penalty system for features completed leads to paralysis. And a complicated incentive system leads to game playing over productivity. It's an old saying: be careful what you measure because you'll get a lot more of it.
      - jboy55 11y ago
        I'm beginning to think that the best incentive system is just a good base pay, with an emphasis on personal learning and growth. Have engineers strive to become better engineers by taking courses, attending meetings and publishing content. Quality will fall out of that as a positive side-effect.
  - Splines 11y ago
    The solution to this is to require a bug/ticket/tracking item as part of the change. Since QA knows that dev needs to create one anyway, they might as well get the credit. Where this starts to break apart is that QA can get overzealous and file bugs for different incarnations of the same bug. It's a bit of give and take.
- kyllo 11y ago
  The top-level QA manager and the top-level Dev manager should both report to the CTO Having a top-level manager who is responsible for a whole discipline doesn't work, it just creates a high wall to throw shit over. Developers and QA testers really should report to the same first line manager.
- benihana 11y ago
  >and everyone should be rewarded or penalized based on the number of issues found post-release. this sounds absolutely horrible. What a great way to remove all agency and ownership from an organization and incentivize people to pass the buck off to literally anyone but themselves and to work on easy, cupcake features with no risk. >This keeps the devs incentivized to make sure everything works before the code goes to QA, and it keeps everyone incentivized to eliminate as many bugs as possible before release. What kind of organization are you working in where you think devs have to be motivated by punishment or reward? Do you not hire the kind of people who are intrinsically motivated and want to produce software that doesn't have bugs? Do you think that if you don't explicitly incentivize developers not to write bugs that they would just fuck around and make shitty, buggy software? I just can't imagine working in an organization so dysfunctional that leadership thinks the default state is being happy with shitty buggy software.
- room271 11y ago
  That sounds like a terrible strategy for a lot of us, where time to fix is much more important than bug count. This is because there is likely to be a tradeoff between speedy development and bug count. If you are shipping physical CDs of software, or releases which you can't update easily, or if you are working in a critical environment where mistakes are disastrous (finance, health, space, etc.), then it is fair enough to be so concerned about bugs. But many if not most developers work on the web in areas where the product is uncertain and evolving. Prioritising rate of change is more important here. Rather than extensive QA, monitoring and fast rollback are better choices.
  - DougWebb 11y ago
    A formal QA process, which goes along with a formal release process and release schedule, certainly does slow things down. I think for large-scale projects with a large feature/requirement count that's ok, because there's a lot of little details to keep track of and the formal process helps with that. But you also need a fast-track for fixing small things quickly, and for that Dev, QA, and Ops need to work together to quickly diagnose, fix, test, and release changes. My experience with the formal process I've been talking about was in a company whose product is a large-scale web application that interacted with a complex custom back end search engine and a proprietary content database. The QA and Dev teams had a friendly adversarial relationship, but ultimately we all felt we were on the same team with the same goals: to produce the best software we could. It was a fantastic place to work.
tootie 11y ago
My experience is the opposite. Having independent testers means you're going to be stuck addressing 100 defects if you don't watch what you're doing the first time around.

by jboy55 11y ago

DougWebb 11y ago

That's a management problem, not a problem with having a QA team. The top-level QA manager and the top-level Dev manager should both report to the CTO, and the Dev manager should be judged by how many issues the QA team finds. To keep things fair, the QA manager should not be rewarded nor penalized based on the number of issues found pre-release, and everyone should be rewarded or penalized based on the number of issues found post-release. This keeps the devs incentivized to make sure everything works before the code goes to QA, and it keeps everyone incentivized to eliminate as many bugs as possible before release.

fein 11y ago
Have you ever worked for a fortune 100 company? The dev manager has probably never met the CTO. Im in this right now with my currrent job. I still keep my builds and test them longer than i should. Less broken shit gets through, but i regularly delay my qa ba's because of it.
- DougWebb 11y ago
  So, not the CTO, but some shared higher-level manager. The point is that the QA lead cannot report into Development, and Development cannot report into QA. Both of those arrangements can lead to a bad actor stopping the other group from doing the right thing. (Eg: if QA reports to Dev, then Dev manager can force bad code to ship, and if Dev reports to QA than QA manager can prevent shipping and force Dev to waste time on excessive/pointless test code development that's not cost-justified.)
  - zhte415 11y ago
    I've met people that call something similar 'fake agile'. Surely everyone should report into the business (end-user on the business side) lead responsible for making this functionality happen. This includes Dev, QA, business. QA should be business? This is no longer 1985, there is no reason business (product owner) shouldn't be a capable PM and BA for their own product.
    - DougWebb 11y ago
      In the company I used to work for, the CTO and VP of Product Management were equals, both reporting to the CEO. The CTO's reports included the QA director, Dev director, and Project Management director. On any given project, we had a Product Manager who set the business requirements, a Project Manager who ran the project, a Dev lead in charge of the development team, and a QA lead in charge of QA. None of these people could override the others because of the reporting arrangement, so we had to collaborate instead to make sure we all met our goals. Early on this arrangement worked really well. But it started falling apart when ownership of the company changed and politics became a factor as people who cared more about moving up through the organization than about the success of the company and quality of the products were hired and promoted. My take is that bad management will destroy any approach you might take to producing good software. That's why I eventually left.
- rikkus 11y ago
  I work for a FTSE 100 company and the dev manager and CTO in the division where I work are very well connected. I don't think the size of company is necessarily the only factor here.
dragonwriter 11y ago
> The top-level QA manager and the top-level Dev manager should both report to the CTO For an organization with a single product (and, particularly, one small enough that the "top-level" QA and Dev manager are also first level managers), this makes some sense. Otherwise, this means that each product has no common level of technical management below the CTO, meaning it has no meaningful technical ownership.
- DougWebb 11y ago
  It doesn't have to be the CTO; as I mentioned in another reply my intent was that QA and Dev need to be equals in the reporting chain, rather than one reporting to the other. That can work at any scale; even on an agile team with one QA and one Dev, they both report to the same boss who's responsible for both sides.
  - knughit 11y ago
    Of you are fine tuning the reporting chain, your org is already hopelessly broken. That is the central planning Communist model.
vkou 11y ago
What will happen is that devs will ask their QA counterparts to report issues through an undocumented side-channel. If you give an engineer a career incentive to optimize something, you'd be surprised how seriously some will take it.
- jboy55 11y ago
  Incentives often cause really nasty political wars within orgs. When a site gives a 500 because a database went down and the web app couldn't connect to it... is that a bug, a missed test case or should ops take a hit on down time? Furthermore, if you argue that the dev team should have reasonable failsafes in code to connect to a db, in a 10 year old organization, should the current Dev team pay for something that could have been in the code for years? If you set up a system where devs hand off to QA then they hand off to ops to deploy. Furthermore every step is incentivised somewhat against each other. Even if all teams are equal, it ends up in my opinion, to the path to CYA and Waterfall. Everyone is more concerned with problems not being 'their' fault than shipping good code.
  - jrumbut 11y ago
    You're absolutely right, assuming we're talking about business/consumer apps and not pacemakers or rocket ships. An incentive/penalty system for bugs without an incentive/penalty system for features completed leads to paralysis. And a complicated incentive system leads to game playing over productivity. It's an old saying: be careful what you measure because you'll get a lot more of it.
    - jboy55 11y ago
      I'm beginning to think that the best incentive system is just a good base pay, with an emphasis on personal learning and growth. Have engineers strive to become better engineers by taking courses, attending meetings and publishing content. Quality will fall out of that as a positive side-effect.
- Splines 11y ago
  The solution to this is to require a bug/ticket/tracking item as part of the change. Since QA knows that dev needs to create one anyway, they might as well get the credit. Where this starts to break apart is that QA can get overzealous and file bugs for different incarnations of the same bug. It's a bit of give and take.
kyllo 11y ago
The top-level QA manager and the top-level Dev manager should both report to the CTO Having a top-level manager who is responsible for a whole discipline doesn't work, it just creates a high wall to throw shit over. Developers and QA testers really should report to the same first line manager.
benihana 11y ago
>and everyone should be rewarded or penalized based on the number of issues found post-release. this sounds absolutely horrible. What a great way to remove all agency and ownership from an organization and incentivize people to pass the buck off to literally anyone but themselves and to work on easy, cupcake features with no risk. >This keeps the devs incentivized to make sure everything works before the code goes to QA, and it keeps everyone incentivized to eliminate as many bugs as possible before release. What kind of organization are you working in where you think devs have to be motivated by punishment or reward? Do you not hire the kind of people who are intrinsically motivated and want to produce software that doesn't have bugs? Do you think that if you don't explicitly incentivize developers not to write bugs that they would just fuck around and make shitty, buggy software? I just can't imagine working in an organization so dysfunctional that leadership thinks the default state is being happy with shitty buggy software.
room271 11y ago
That sounds like a terrible strategy for a lot of us, where time to fix is much more important than bug count. This is because there is likely to be a tradeoff between speedy development and bug count. If you are shipping physical CDs of software, or releases which you can't update easily, or if you are working in a critical environment where mistakes are disastrous (finance, health, space, etc.), then it is fair enough to be so concerned about bugs. But many if not most developers work on the web in areas where the product is uncertain and evolving. Prioritising rate of change is more important here. Rather than extensive QA, monitoring and fast rollback are better choices.
- DougWebb 11y ago
  A formal QA process, which goes along with a formal release process and release schedule, certainly does slow things down. I think for large-scale projects with a large feature/requirement count that's ok, because there's a lot of little details to keep track of and the formal process helps with that. But you also need a fast-track for fixing small things quickly, and for that Dev, QA, and Ops need to work together to quickly diagnose, fix, test, and release changes. My experience with the formal process I've been talking about was in a company whose product is a large-scale web application that interacted with a complex custom back end search engine and a proprietary content database. The QA and Dev teams had a friendly adversarial relationship, but ultimately we all felt we were on the same team with the same goals: to produce the best software we could. It was a fantastic place to work.

tootie 11y ago

My experience is the opposite. Having independent testers means you're going to be stuck addressing 100 defects if you don't watch what you're doing the first time around.