4 ms·

From what I've seen, the value in QA is product familiarity. Good QA'ers know more about how the product actually works than anybody else. More than PM's, more

From what I've seen, the value in QA is product familiarity. Good QA'ers know more about how the product actually works than anybody else. More than PM's, more than sales, and more than most dev teams. They have a holistic knowledge of the entire user-facing system and can tell you exactly what to expect when any button gets pushed. Bad QA'ers are indeed a source of noise. But so are bad devs, sysadmins, T1/2 support, etc.

wmichelin 3y ago
Agreed! I did have some good experiences at my last job with the QA team, but it was definitely a unique model. They were really a "Customer Success" team, it was a mix of QA, sales, and customer support. These "Customer Support" reps, when functioning as QA, knew the product better than product or eng, exactly how you're describing. I did enjoy that model, but they also did not write tests for us. They primarily executed manual test plans, after deploys, in production. They did provide more value than creating noise, but the engineering team still was QA, at least from an automated test standpoint.
- mbb70 3y ago
  We had no dedicated QA, but would consistently poach "Customer Success" team members for critical QA work for the exact reasons your listed. Worked quite well for us. Especially for complex products that are based on users chaining many building blocks together to create something useful, devs generally have no visibility into how users work and how to test.
hasoleju 3y ago
I completely agree with that. It really comes down to having the right skills as a QA person. If you don't know how the product is used and only click on some buttons, you will never reach the states in the software that real users reach and therefor you will also not be able to reproduce them.
cableshaft 3y ago
> Good QA'ers know more about how the product actually works than anybody else. More than PM's, more than sales, and more than most dev teams. Not disagreeing with this, but there's one thing they won't always be aware of. They won't always know what code a dev touched underneath the hood and what they might need to recheck (short of a full regression test every single time) to verify everything is still working. I know that the component I adjusted for this feature might have also affected the component over in spots X, Y, and Z, because I looked at that code, and probably did a code search or a 'find references' check at some point to see where else it's getting called, and also I usually retest those other places as well (not every dev does, though. I've met some devs that think it's a waste of time and money for them to test anything and that's entirely QA's job). A good QA person might also intuit other places that might be affected if it's a visible component that looks the same (but either I haven't worked with too many good QA people or that intuition is pretty rare, I'm guessing it's the latter because I believe I have worked with people who were good at QA). Because of that, I do my best to be proactive and go "oh by the way this code might have affected these other places, please include those in your tests".
- hysan 3y ago
  > They won't always know what code a dev touched underneath the hood and what they might need to recheck (short of a full regression test every single time) to verify everything is still working. This is a good point, but there are some QA that do review code (source: me - started career in QA and transitioned to dev). When making a test plan, an important factor is risk assessment. If QA has a hunch, or better when the dev lead flags complex changes, the test plan should be created and then the code diffs should be reviewed to assess whether or not the plan needed revising. For example, maybe the QA env doesn’t have a full replica of prod but a query is introduced that could be impacted if one of the joining tables is huge (like in prod). So maybe we’d adjust the plan to run some benchmarks on a similar scale environment. I’m definitely biased since I started in QA and loved it. To me, good QA is a cross section of many of the things people have mentioned - technical, product, ops, security - with a healthy dash of liking to break things. However, reality is that the trend has been to split that responsibility among people in each of those roles and get rid of QA. Works great if people in each of those job functions has the bandwidth to take on that QA work (they’ll all have a much deeper knowledge of their respective domains). But you’ll lose coverage if any one of those people don’t have time to dedicate to proper QA. (I’ll also completely acknowledge that it’s rare to have a few, let alone a full team, of QA people who can do that.)
bbarn 3y ago
This is a great model, until those people so familiar with the business needs end up.. doing business things instead. It's really hard to keep people like that in a QA role once the business recognizes their value. Kind of the same problem with QA automation people - once they become really good at test automation, they are effectively software developers, and want to go there.
- importantbrian 3y ago
  I think that's a compensation problem more than anything else. I've known some QA folks who enjoyed QA and would have stayed in that role if they could have justified the massive differential in comp between QA and SWE or product development. If we valued QA and compensated it at the same level we do those other roles then there would be a lot less difficulty retaining good QA folks.
- taurath 3y ago
  I have never once heard of a problem that QA folks end up in project or product management too often, and almost always have the problem of not being able to escape the QA org despite many years. Most companies are extremely resistant to people moving tracks, especially from a “lower status” org like QA or CS. It’s the exception not the rule.
taylodl 3y ago
To your point, the QA team is the customer's advocate. As you say, they know the product, from the customer's perspective, better than anyone else in the development organization. Where I've seen QA teams most effective is providing more function than "just" QA. I've seen them used for 2nd tier support. I've seen them used to support sales engineers. I've also seen QA teams that take their manual test plans and automate their execution (think Selenium or UiPath) and have seen those automations included in dev pipelines. Finally, the QA team are the masters and caretakers of your test environment(s), all the different types of accounts you need for testing, they should have the knowledge of all the different browsers and OSes your customers are using, and so forth. That's a lot for the dev team to take on.
- kgermino 3y ago
  That also means they test from a different perspective than the dev does. If I get a requirement my build is based on my understanding of the requirement, and so is my testing. A separate QA person coming at it from the customer's perspective will do a test that's much more likely to reflect reality.

wmichelin 3y ago

Agreed! I did have some good experiences at my last job with the QA team, but it was definitely a unique model. They were really a "Customer Success" team, it was a mix of QA, sales, and customer support. These "Customer Support" reps, when functioning as QA, knew the product better than product or eng, exactly how you're describing. I did enjoy that model, but they also did not write tests for us. They primarily executed manual test plans, after deploys, in production. They did provide more value than creating noise, but the engineering team still was QA, at least from an automated test standpoint.

mbb70 3y ago
We had no dedicated QA, but would consistently poach "Customer Success" team members for critical QA work for the exact reasons your listed. Worked quite well for us. Especially for complex products that are based on users chaining many building blocks together to create something useful, devs generally have no visibility into how users work and how to test.

hasoleju 3y ago

I completely agree with that. It really comes down to having the right skills as a QA person. If you don't know how the product is used and only click on some buttons, you will never reach the states in the software that real users reach and therefor you will also not be able to reproduce them.

cableshaft 3y ago

> Good QA'ers know more about how the product actually works than anybody else. More than PM's, more than sales, and more than most dev teams. Not disagreeing with this, but there's one thing they won't always be aware of. They won't always know what code a dev touched underneath the hood and what they might need to recheck (short of a full regression test every single time) to verify everything is still working. I know that the component I adjusted for this feature might have also affected the component over in spots X, Y, and Z, because I looked at that code, and probably did a code search or a 'find references' check at some point to see where else it's getting called, and also I usually retest those other places as well (not every dev does, though. I've met some devs that think it's a waste of time and money for them to test anything and that's entirely QA's job). A good QA person might also intuit other places that might be affected if it's a visible component that looks the same (but either I haven't worked with too many good QA people or that intuition is pretty rare, I'm guessing it's the latter because I believe I have worked with people who were good at QA). Because of that, I do my best to be proactive and go "oh by the way this code might have affected these other places, please include those in your tests".

hysan 3y ago
> They won't always know what code a dev touched underneath the hood and what they might need to recheck (short of a full regression test every single time) to verify everything is still working. This is a good point, but there are some QA that do review code (source: me - started career in QA and transitioned to dev). When making a test plan, an important factor is risk assessment. If QA has a hunch, or better when the dev lead flags complex changes, the test plan should be created and then the code diffs should be reviewed to assess whether or not the plan needed revising. For example, maybe the QA env doesn’t have a full replica of prod but a query is introduced that could be impacted if one of the joining tables is huge (like in prod). So maybe we’d adjust the plan to run some benchmarks on a similar scale environment. I’m definitely biased since I started in QA and loved it. To me, good QA is a cross section of many of the things people have mentioned - technical, product, ops, security - with a healthy dash of liking to break things. However, reality is that the trend has been to split that responsibility among people in each of those roles and get rid of QA. Works great if people in each of those job functions has the bandwidth to take on that QA work (they’ll all have a much deeper knowledge of their respective domains). But you’ll lose coverage if any one of those people don’t have time to dedicate to proper QA. (I’ll also completely acknowledge that it’s rare to have a few, let alone a full team, of QA people who can do that.)

bbarn 3y ago

This is a great model, until those people so familiar with the business needs end up.. doing business things instead. It's really hard to keep people like that in a QA role once the business recognizes their value. Kind of the same problem with QA automation people - once they become really good at test automation, they are effectively software developers, and want to go there.

importantbrian 3y ago
I think that's a compensation problem more than anything else. I've known some QA folks who enjoyed QA and would have stayed in that role if they could have justified the massive differential in comp between QA and SWE or product development. If we valued QA and compensated it at the same level we do those other roles then there would be a lot less difficulty retaining good QA folks.
taurath 3y ago
I have never once heard of a problem that QA folks end up in project or product management too often, and almost always have the problem of not being able to escape the QA org despite many years. Most companies are extremely resistant to people moving tracks, especially from a “lower status” org like QA or CS. It’s the exception not the rule.

taylodl 3y ago

To your point, the QA team is the customer's advocate. As you say, they know the product, from the customer's perspective, better than anyone else in the development organization. Where I've seen QA teams most effective is providing more function than "just" QA. I've seen them used for 2nd tier support. I've seen them used to support sales engineers. I've also seen QA teams that take their manual test plans and automate their execution (think Selenium or UiPath) and have seen those automations included in dev pipelines. Finally, the QA team are the masters and caretakers of your test environment(s), all the different types of accounts you need for testing, they should have the knowledge of all the different browsers and OSes your customers are using, and so forth. That's a lot for the dev team to take on.

kgermino 3y ago
That also means they test from a different perspective than the dev does. If I get a requirement my build is based on my understanding of the requirement, and so is my testing. A separate QA person coming at it from the customer's perspective will do a test that's much more likely to reflect reality.