9 ms·

The conversation is usually: devs can write their own tests. We don't need QA. And the first part is true. We can. But that's not why we have (had) QA. First:

The conversation is usually: devs can write their own tests. We don't need QA. And the first part is true. We can. But that's not why we have (had) QA. First: it's not the best use of our time. I believe dev and QA are separate skillset. Of course there is overlap. Second, and most important: it's a separate person, an additional person who can question the ticket, and who can question my translation of the ticket into software. And lastly: they don't suffer from the curse of knowledge on how I implemented the ticket. I miss my QA colleagues. When I joined my current employer there were 8 or so. Initially I was afraid to give them my work, afraid of bad feedback. Never have I met such graceful people who took the time in understanding something, and talking to me to figure out where there was a mismatch. And then they were deemed not needed.

torginus 9mo ago
I very rarely worked with good QA. In my mind a good QA understands the feature we're working on, deploys the correct version, thoroughly tests the feature understanding what it's supposed and not supposed to do, and if they happen to find a bug, they create a bug ticket where they describe the environment in full and what steps are necessary to reproduce it. For automation tests, very few are capable of writing tests that test the spec, not implementation, contain sound technical practices, and properly address flakiness. For example it's very common to see a test that clicks the login button and instead of waiting for the login, the wait 20 seconds. Which is both too much, and 1% of the time too little. Whenever I worked with devs, they almost always managed to do all this, sometimes they needed a bit of guidance, but that's it. Very very few QA ever did (not that they seemed to bothered by that). A lot of QA have expressed that devs 'look down' on them. I can't comment on that, but the signal-to-noise ratio of bug tickets is so low, that often it's you have to do their job and repeat everything as well. This has been a repeated experience for me with multiple companies and a lot of places don't have proper feedback loops, so it doesn't even bother them as they're not affected by the poor quality of bug reports, but devs have to spend the extra time.
- bbayles 9mo ago
  I used to work with a QA person who really drove me nuts. They would misunderstand the point of a feature, and then write pages and pages of misguided commentary about what they saw when trying to test it. We'd repeat this a few times for every release. This forced me to start making my feature proposals as small as possible. I would defensively document everything, and sprinkle in little summaries to make things as clear as possible. I started writing scripts to help isolate the new behavior during testing. ...eventually I realized that this person was somehow the best QA person I'd ever worked with.
  - philk10 9mo ago
    how did misunderstanding a feature and writing pages on it help, not sure I follow the logic of why this made them a good QA person? Do you mean the features were not written well and so writing code for them was going to produce errors?
    - RHSeeger 9mo ago
      I worked with someone a little while ago that tended to do this; point out things that weren't really related to the ticket. And I was happy with their work. I think the main thing to remember is that the following are two different things - Understanding what is important to / related to the functionality of a given ticket - Thoroughly testing what is important to / related to the functionality of a given ticket Sure, the first one can waste some time by causing discussion of things that don't matter. But being REALLY good at the second one can mean far less bugs slip through.
      - tharkun__ 9mo ago
        I think noticing other bugs that aren't related to the ticket at hand is actually a good thing. That's how you notice them, by "being in the area" anyway. What many QAs can't do / for me separates the good and the not so good ones, is that they actually understand when they're not related and just report them as separate bugs to be tackled independently instead of starting long discussions on the current ticket at hand.
blindriver 9mo ago
I've worked in enterprise software development with the full lifecycle for over 30 years. I have found QA to be mostly unnecessary friction throughout my career, and I've never been more productive than when QA and writing tests became my responsibility. This is usually what has happened during a release cycle. 1) Devs come up with a list of features and a timeline. 2) QA will go through the list and about 1/2 of the features will get cut because they claim they don't have time to test everything based on their timeline. 3) The cycle begins and devs will start adding features into the codebase and it's radio silence from the QA. 4) Throughout the release QA will force more features to get dropped. By the end of the release cycle, another 1/4 of the original number of features get dropped leaving about 1/4 of the original features that were planned. "It will get done in a dot release." 5) Near the end of the release, everything gets tested and a mountain of bugs come in near the deadline and everyone is forced to scramble. The deadline gets pushed back and QA pushes the blame onto the devs. 6) After everything gets resolved, the next release cycle begins. This is at quite a few enterprise software companies that most people in Silicon Valley have heard of if you've been working for more than 10 years.
- Kinrany 9mo ago
  Release cycles are the problem
jonfw 9mo ago
Like all other job functions tangential to development- it can be difficult to organize the labor needed to accomplish this within a single team, and it can be difficult to align incentives when the labor is spread across multiple teams. This gets more and more difficult with modern development practices. Development benefits greatly from fast release cycles and quick iteration- the other job functions do not! QA is certainly included there. I think that inherent conflict is what is causing developers to increasingly managing their own operations, technical writing, testing, etc.
- daotoad 9mo ago
  In my experience, what works best is having QA and a tech writer assigned to a few teams. That way there is an ongoing, close relationship that makes interactions go smoothly. In a larger org, it may also make sense to have a separate QA group that handles tasks across all teams and focuses on the product as a unified whole.
- zingar 9mo ago
  I can’t imagine any role in software that gets better delivering more work in longer cycles than less work in shorter cycles. And I can’t speak for technical writing, but developers do operations and testing because automation makes it possible and better than living in a dysfunctional world of fire and forget coding.
marcosdumay 9mo ago
> I believe dev and QA are separate skillset. I'm not sure it's a separate skillset. You need the other side's skills all the time in each of those positions. But it's certainly a separate mindset. People must hold different values in each of them. One just can't competently do both at the same time. And "time" is quantized here in months-long intervals, because it takes many days to internalize a mindset, if one is flexible enough to ever do it.
cindyllm 9mo ago
[dead]
jghn 9mo ago
There are layers to this: 1) There are different types of tests, for different purposes. Devs should be writing some of them. Other types & forms of testing, I agree that this is not in many dev's sweet spot. In other words, by the time code gets thrown over the wall to QA, it should already be fairly well vetted at least in the small. 2) Many, but far from all, QA people are just not skilled. It wasn't that long ago that most QA people were washed out devs. My experience has that while testing isn't in the sweet spot of many devs, that they've been better at it than the typical QA person 3) High quality QA people are worth their weight in gold. 4) Too often devs look at QA groups as someone to whom they can offload their grunt work they don't want to do. Instead, QA groups should be partnering with dev teams to take up higher level and more advanced testing, helping devs to self-help with other types of testing, and other such tasks.
- astura 9mo ago
  >High quality QA people are worth their weight in gold. They absolutely are, but I've only met a couple high quality QA people in my career.
  - daotoad 9mo ago
    That's because we don't value QA in the way that matters. If you're a talented SDET, you're probably also, at least, a good SDE. If you'll make more money and have more opportunity as an SDE, which career path will you follow?
    - theLiminator 9mo ago
      Also, for most people passionate about software, they'd rather be building than testing, especially if pay is at least equal.
      - MoreQARespect 9mo ago
        Testing is probably my favorite topic in development and I kind of wish I could make it my "official" specialty but no way in hell am I taking a pay cut and joining the part of the org nobody listens to.
RHSeeger 9mo ago
I agree with everything you're saying here. The only thing I would add is Before I hand a ticket off to QA, I write up 1. What I understood the requirements to be, 2. What I implemented, 3. How to interact with it (where it is in the tool, etc), and 4. What _other_ areas of the code, besides the obvious, are touched; so they can regression test any areas that they feel are important Doing that writeup makes sure I understand everything about my implementation and that I didn't miss anything. I find it extremely valuable both for QA and myself.
- mixmastamyk 9mo ago
  Rubber-ducky in human form.
- weinzierl 9mo ago
  "What I understood the requirements to be" That is a way to get your changed approved quickly, so it is good for you. It is terrible for a project that values quality. A tremendous value of a QA team is that they interpret the requirements independently and if in the end they approve you can be pretty confident you implemented something that conforms to the commonly understood meaning of requirements and not your developer biased view.
  - RHSeeger 9mo ago
    Challenge. I use it as a way to double check, "Did the way I understood the ticket/requirement match the way that QA did?". They're testing what the ticket says is required. But part of that is testing that I understood the ticket correctly.
leptons 9mo ago
No QA is better than bad QA. I've had great QA teams and just awful QA teams. Most of them are somewhere in the middle. I'll take no QA over bad QA every time. Filing bugs that aren't bugs, not understanding the most basic things like what a Jpeg file is for a product centered around images, etc. QA for the sake of QA doesn't always yield results, and can cause a lot of distraction for a competent dev team.
javier2 9mo ago
Yeah. As a dev, it is simply not always a great idea that the same person that built the feature is the one testing it. Sometimes I already tested it 100 times, and by the 110th time I basically become blind to it because I know it too well. Then its great to have someone with fresh eyes and without the detailed knowledge do the testing to see if it works and if it works for our customers.
m463 9mo ago
dev tests - whitebox tests qa tests - blackbox tests there is a place for both. I think the problem is that usually the business doesn't care enough about that level of quality (unless nasa or avaiation)
locknitpicker 9mo ago
> First: it's not the best use of our time. I believe dev and QA are separate skillset. Of course there is overlap. I've heard this comment before,and it fails to hold in practice. You are not making the best use of your time if your ticket is blocked because you were paired with a QA who is sitting on a ticket, or is oblivious to the problem domain to the point he struggles to put together a single test that is relevant. You are also not making the best use of your time if a test suite is not maintained by said QAs and fails in pipelines without anyone acting on it. For verification and validation tasks, and if QAs treat pipeline maintenance as an OnCall rotation, that would hold true. Anything beyond this, a QA can waste far more time than the one they could hypothetically save you.
scott_w 9mo ago
> First: it's not the best use of our time. I want to push back strongly on this. I think this attitude leads to more bugs as QA cannot possibly keep up with the rate of change. I understand that you, personally, may not have exhibited this based on your elaboration. However that doesn’t change the fact that many devs do take exactly this attitude with QA. To take a slightly different comparison, I would liken it to the old school of “throwing it over the wall” to Ops to handle deployment. Paying attention to how the code gets to production and subsequently runs on prod isn’t a “good use of developer time,” either. Except we discarded that view a decade ago, for good reason.

torginus 9mo ago

I very rarely worked with good QA. In my mind a good QA understands the feature we're working on, deploys the correct version, thoroughly tests the feature understanding what it's supposed and not supposed to do, and if they happen to find a bug, they create a bug ticket where they describe the environment in full and what steps are necessary to reproduce it. For automation tests, very few are capable of writing tests that test the spec, not implementation, contain sound technical practices, and properly address flakiness. For example it's very common to see a test that clicks the login button and instead of waiting for the login, the wait 20 seconds. Which is both too much, and 1% of the time too little. Whenever I worked with devs, they almost always managed to do all this, sometimes they needed a bit of guidance, but that's it. Very very few QA ever did (not that they seemed to bothered by that). A lot of QA have expressed that devs 'look down' on them. I can't comment on that, but the signal-to-noise ratio of bug tickets is so low, that often it's you have to do their job and repeat everything as well. This has been a repeated experience for me with multiple companies and a lot of places don't have proper feedback loops, so it doesn't even bother them as they're not affected by the poor quality of bug reports, but devs have to spend the extra time.

bbayles 9mo ago
I used to work with a QA person who really drove me nuts. They would misunderstand the point of a feature, and then write pages and pages of misguided commentary about what they saw when trying to test it. We'd repeat this a few times for every release. This forced me to start making my feature proposals as small as possible. I would defensively document everything, and sprinkle in little summaries to make things as clear as possible. I started writing scripts to help isolate the new behavior during testing. ...eventually I realized that this person was somehow the best QA person I'd ever worked with.
- philk10 9mo ago
  how did misunderstanding a feature and writing pages on it help, not sure I follow the logic of why this made them a good QA person? Do you mean the features were not written well and so writing code for them was going to produce errors?
  - RHSeeger 9mo ago
    I worked with someone a little while ago that tended to do this; point out things that weren't really related to the ticket. And I was happy with their work. I think the main thing to remember is that the following are two different things - Understanding what is important to / related to the functionality of a given ticket - Thoroughly testing what is important to / related to the functionality of a given ticket Sure, the first one can waste some time by causing discussion of things that don't matter. But being REALLY good at the second one can mean far less bugs slip through.
    - tharkun__ 9mo ago
      I think noticing other bugs that aren't related to the ticket at hand is actually a good thing. That's how you notice them, by "being in the area" anyway. What many QAs can't do / for me separates the good and the not so good ones, is that they actually understand when they're not related and just report them as separate bugs to be tackled independently instead of starting long discussions on the current ticket at hand.

blindriver 9mo ago

I've worked in enterprise software development with the full lifecycle for over 30 years. I have found QA to be mostly unnecessary friction throughout my career, and I've never been more productive than when QA and writing tests became my responsibility. This is usually what has happened during a release cycle. 1) Devs come up with a list of features and a timeline. 2) QA will go through the list and about 1/2 of the features will get cut because they claim they don't have time to test everything based on their timeline. 3) The cycle begins and devs will start adding features into the codebase and it's radio silence from the QA. 4) Throughout the release QA will force more features to get dropped. By the end of the release cycle, another 1/4 of the original number of features get dropped leaving about 1/4 of the original features that were planned. "It will get done in a dot release." 5) Near the end of the release, everything gets tested and a mountain of bugs come in near the deadline and everyone is forced to scramble. The deadline gets pushed back and QA pushes the blame onto the devs. 6) After everything gets resolved, the next release cycle begins. This is at quite a few enterprise software companies that most people in Silicon Valley have heard of if you've been working for more than 10 years.

Kinrany 9mo ago
Release cycles are the problem

jonfw 9mo ago

Like all other job functions tangential to development- it can be difficult to organize the labor needed to accomplish this within a single team, and it can be difficult to align incentives when the labor is spread across multiple teams. This gets more and more difficult with modern development practices. Development benefits greatly from fast release cycles and quick iteration- the other job functions do not! QA is certainly included there. I think that inherent conflict is what is causing developers to increasingly managing their own operations, technical writing, testing, etc.

daotoad 9mo ago
In my experience, what works best is having QA and a tech writer assigned to a few teams. That way there is an ongoing, close relationship that makes interactions go smoothly. In a larger org, it may also make sense to have a separate QA group that handles tasks across all teams and focuses on the product as a unified whole.
zingar 9mo ago
I can’t imagine any role in software that gets better delivering more work in longer cycles than less work in shorter cycles. And I can’t speak for technical writing, but developers do operations and testing because automation makes it possible and better than living in a dysfunctional world of fire and forget coding.

marcosdumay 9mo ago

> I believe dev and QA are separate skillset. I'm not sure it's a separate skillset. You need the other side's skills all the time in each of those positions. But it's certainly a separate mindset. People must hold different values in each of them. One just can't competently do both at the same time. And "time" is quantized here in months-long intervals, because it takes many days to internalize a mindset, if one is flexible enough to ever do it.

cindyllm 9mo ago

[dead]

jghn 9mo ago

There are layers to this: 1) There are different types of tests, for different purposes. Devs should be writing some of them. Other types & forms of testing, I agree that this is not in many dev's sweet spot. In other words, by the time code gets thrown over the wall to QA, it should already be fairly well vetted at least in the small. 2) Many, but far from all, QA people are just not skilled. It wasn't that long ago that most QA people were washed out devs. My experience has that while testing isn't in the sweet spot of many devs, that they've been better at it than the typical QA person 3) High quality QA people are worth their weight in gold. 4) Too often devs look at QA groups as someone to whom they can offload their grunt work they don't want to do. Instead, QA groups should be partnering with dev teams to take up higher level and more advanced testing, helping devs to self-help with other types of testing, and other such tasks.

astura 9mo ago
>High quality QA people are worth their weight in gold. They absolutely are, but I've only met a couple high quality QA people in my career.
- daotoad 9mo ago
  That's because we don't value QA in the way that matters. If you're a talented SDET, you're probably also, at least, a good SDE. If you'll make more money and have more opportunity as an SDE, which career path will you follow?
  - theLiminator 9mo ago
    Also, for most people passionate about software, they'd rather be building than testing, especially if pay is at least equal.
    - MoreQARespect 9mo ago
      Testing is probably my favorite topic in development and I kind of wish I could make it my "official" specialty but no way in hell am I taking a pay cut and joining the part of the org nobody listens to.

RHSeeger 9mo ago

I agree with everything you're saying here. The only thing I would add is Before I hand a ticket off to QA, I write up 1. What I understood the requirements to be, 2. What I implemented, 3. How to interact with it (where it is in the tool, etc), and 4. What _other_ areas of the code, besides the obvious, are touched; so they can regression test any areas that they feel are important Doing that writeup makes sure I understand everything about my implementation and that I didn't miss anything. I find it extremely valuable both for QA and myself.

mixmastamyk 9mo ago
Rubber-ducky in human form.
weinzierl 9mo ago
"What I understood the requirements to be" That is a way to get your changed approved quickly, so it is good for you. It is terrible for a project that values quality. A tremendous value of a QA team is that they interpret the requirements independently and if in the end they approve you can be pretty confident you implemented something that conforms to the commonly understood meaning of requirements and not your developer biased view.
- RHSeeger 9mo ago
  Challenge. I use it as a way to double check, "Did the way I understood the ticket/requirement match the way that QA did?". They're testing what the ticket says is required. But part of that is testing that I understood the ticket correctly.

leptons 9mo ago

No QA is better than bad QA. I've had great QA teams and just awful QA teams. Most of them are somewhere in the middle. I'll take no QA over bad QA every time. Filing bugs that aren't bugs, not understanding the most basic things like what a Jpeg file is for a product centered around images, etc. QA for the sake of QA doesn't always yield results, and can cause a lot of distraction for a competent dev team.

javier2 9mo ago

Yeah. As a dev, it is simply not always a great idea that the same person that built the feature is the one testing it. Sometimes I already tested it 100 times, and by the 110th time I basically become blind to it because I know it too well. Then its great to have someone with fresh eyes and without the detailed knowledge do the testing to see if it works and if it works for our customers.

m463 9mo ago

dev tests - whitebox tests qa tests - blackbox tests there is a place for both. I think the problem is that usually the business doesn't care enough about that level of quality (unless nasa or avaiation)

locknitpicker 9mo ago

> First: it's not the best use of our time. I believe dev and QA are separate skillset. Of course there is overlap. I've heard this comment before,and it fails to hold in practice. You are not making the best use of your time if your ticket is blocked because you were paired with a QA who is sitting on a ticket, or is oblivious to the problem domain to the point he struggles to put together a single test that is relevant. You are also not making the best use of your time if a test suite is not maintained by said QAs and fails in pipelines without anyone acting on it. For verification and validation tasks, and if QAs treat pipeline maintenance as an OnCall rotation, that would hold true. Anything beyond this, a QA can waste far more time than the one they could hypothetically save you.

scott_w 9mo ago

> First: it's not the best use of our time. I want to push back strongly on this. I think this attitude leads to more bugs as QA cannot possibly keep up with the rate of change. I understand that you, personally, may not have exhibited this based on your elaboration. However that doesn’t change the fact that many devs do take exactly this attitude with QA. To take a slightly different comparison, I would liken it to the old school of “throwing it over the wall” to Ops to handle deployment. Paying attention to how the code gets to production and subsequently runs on prod isn’t a “good use of developer time,” either. Except we discarded that view a decade ago, for good reason.