3 ms·

> Good QA'ers know more about how the product actually works than anybody else. More than PM's, more than sales, and more than most dev teams. Not disagreeing

> Good QA'ers know more about how the product actually works than anybody else. More than PM's, more than sales, and more than most dev teams. Not disagreeing with this, but there's one thing they won't always be aware of. They won't always know what code a dev touched underneath the hood and what they might need to recheck (short of a full regression test every single time) to verify everything is still working. I know that the component I adjusted for this feature might have also affected the component over in spots X, Y, and Z, because I looked at that code, and probably did a code search or a 'find references' check at some point to see where else it's getting called, and also I usually retest those other places as well (not every dev does, though. I've met some devs that think it's a waste of time and money for them to test anything and that's entirely QA's job). A good QA person might also intuit other places that might be affected if it's a visible component that looks the same (but either I haven't worked with too many good QA people or that intuition is pretty rare, I'm guessing it's the latter because I believe I have worked with people who were good at QA). Because of that, I do my best to be proactive and go "oh by the way this code might have affected these other places, please include those in your tests".

hysan 3y ago
> They won't always know what code a dev touched underneath the hood and what they might need to recheck (short of a full regression test every single time) to verify everything is still working. This is a good point, but there are some QA that do review code (source: me - started career in QA and transitioned to dev). When making a test plan, an important factor is risk assessment. If QA has a hunch, or better when the dev lead flags complex changes, the test plan should be created and then the code diffs should be reviewed to assess whether or not the plan needed revising. For example, maybe the QA env doesn’t have a full replica of prod but a query is introduced that could be impacted if one of the joining tables is huge (like in prod). So maybe we’d adjust the plan to run some benchmarks on a similar scale environment. I’m definitely biased since I started in QA and loved it. To me, good QA is a cross section of many of the things people have mentioned - technical, product, ops, security - with a healthy dash of liking to break things. However, reality is that the trend has been to split that responsibility among people in each of those roles and get rid of QA. Works great if people in each of those job functions has the bandwidth to take on that QA work (they’ll all have a much deeper knowledge of their respective domains). But you’ll lose coverage if any one of those people don’t have time to dedicate to proper QA. (I’ll also completely acknowledge that it’s rare to have a few, let alone a full team, of QA people who can do that.)
giantrobot 3y ago
> Not disagreeing with this, but there's one thing they won't always be aware of. They won't always know what code a dev touched underneath the hood and what they might need to recheck (short of a full regression test every single time) to verify everything is still working. It doesn't necessarily matter what code was changed, a change in code in Module A can cause a bug in Module B that hasn't been changed in a year. A QA test plan should cover the surface area of the product as used by consumers whoever they might be. While knowing some module had fixes can inform the test plan or focus areas when the test schedule is constrained, only testing changes is the road to tears.
- cableshaft 3y ago
  Test plans never account for everything, at least in my experience, especially edge cases. And it's rare that I've seen any QA team do a full regression test of the entire site. There's only been a few times where I've seen it authorized, and that's usually after a major refactoring or rewrite. I'm not in QA, I write code, so I defer to whatever they decide for these things usually, these are just observations from what I've seen. I just try to make sure I test my code enough that there isn't anything terribly broken when I check it in and fixes I need to make tend to be relatively minor (with a few exceptions in my past). Also I'm not necessarily talking basic functionality here. I'm currently working for a client that's very picky about the look and feel, so if a few pixels in padding get adjusted where it's noticeable, or a font color or size gets adjusted a bit, in one place and it affects something else, there could be complaints. And a test plan is not likely to catch that, at least not any on any projects I've worked on.
asadotzler 3y ago
>They won't always know what code a dev touched underneath the hood and what they might need to recheck (short of a full regression test every single time) to verify everything is still working. Not really. As QA I always reviewed the checkins since yesterday before opening up the daily build. Between the bug comments and the patch comments, even if the patch itself is a bit Greek to me, I can tell what was going on enough to be a better tester of that area.
- cableshaft 3y ago
  Let's say that there's some sort of list component that's used in like six other places (that's maybe not obvious because there's different configurations or something). Six months later I work on a new page that incorporates that component that requires some sort of change in that component that could theoretically break it elsewhere, at least how it looks or something, like alters the padding or something in a way that shoves things onto the next line in one of the instances. If I don't mention what other places that component is used in my comments, are you going to know where all to look for the other instances of that component? Even if you look at the source code, it will only show changes, it won't say 'oh this component is used elsewhere' unless you actively bring up the source code yourself and do a code search yourself. Because it's a visible component, maybe you'll just have the memory that something that looks like this is elsewhere, but maybe you won't remember. Or maybe you're fairly new to the project and aren't aware of these other parts of the site using it. What about something that's even less visible, like something that takes in data and then sorts, transforms, and adds metadata to it? Then I make a change to it to accomodate for some new requirement on a new page. Maybe it's used for four other locations, but the changes only shows the function being altered and the new page using it. That change I made broke it elsewhere, but I don't mention where else to check in my notes. Are you going to think to check each place where data is loaded to make sure it's working correctly, without me explicitly telling you to check those spots? If you can, great, you've got a real intuition for this that I doubt many QA people have. I don't even know to check these other places sometimes, and I worked on the code myself and can do a quick text search to see where else it's being used. This was why I said what I did. It's not just about 'that area', the change could potentially affect another application in different repository entirely sometimes, especially if a shared library is involved. I've literally had like four or five of these situations happen in the past six months of my current project, where my change broke something elsewhere. Usually I caught it, but I also told QA to check it explicitly anyway since I don't usually test things 100% (since I'm not QA). When I was a more junior developer, I missed these things more often and QA would sometimes miss it also since they didn't know to retest something.

hysan 3y ago

> They won't always know what code a dev touched underneath the hood and what they might need to recheck (short of a full regression test every single time) to verify everything is still working. This is a good point, but there are some QA that do review code (source: me - started career in QA and transitioned to dev). When making a test plan, an important factor is risk assessment. If QA has a hunch, or better when the dev lead flags complex changes, the test plan should be created and then the code diffs should be reviewed to assess whether or not the plan needed revising. For example, maybe the QA env doesn’t have a full replica of prod but a query is introduced that could be impacted if one of the joining tables is huge (like in prod). So maybe we’d adjust the plan to run some benchmarks on a similar scale environment. I’m definitely biased since I started in QA and loved it. To me, good QA is a cross section of many of the things people have mentioned - technical, product, ops, security - with a healthy dash of liking to break things. However, reality is that the trend has been to split that responsibility among people in each of those roles and get rid of QA. Works great if people in each of those job functions has the bandwidth to take on that QA work (they’ll all have a much deeper knowledge of their respective domains). But you’ll lose coverage if any one of those people don’t have time to dedicate to proper QA. (I’ll also completely acknowledge that it’s rare to have a few, let alone a full team, of QA people who can do that.)

giantrobot 3y ago

> Not disagreeing with this, but there's one thing they won't always be aware of. They won't always know what code a dev touched underneath the hood and what they might need to recheck (short of a full regression test every single time) to verify everything is still working. It doesn't necessarily matter what code was changed, a change in code in Module A can cause a bug in Module B that hasn't been changed in a year. A QA test plan should cover the surface area of the product as used by consumers whoever they might be. While knowing some module had fixes can inform the test plan or focus areas when the test schedule is constrained, only testing changes is the road to tears.

cableshaft 3y ago
Test plans never account for everything, at least in my experience, especially edge cases. And it's rare that I've seen any QA team do a full regression test of the entire site. There's only been a few times where I've seen it authorized, and that's usually after a major refactoring or rewrite. I'm not in QA, I write code, so I defer to whatever they decide for these things usually, these are just observations from what I've seen. I just try to make sure I test my code enough that there isn't anything terribly broken when I check it in and fixes I need to make tend to be relatively minor (with a few exceptions in my past). Also I'm not necessarily talking basic functionality here. I'm currently working for a client that's very picky about the look and feel, so if a few pixels in padding get adjusted where it's noticeable, or a font color or size gets adjusted a bit, in one place and it affects something else, there could be complaints. And a test plan is not likely to catch that, at least not any on any projects I've worked on.

asadotzler 3y ago

>They won't always know what code a dev touched underneath the hood and what they might need to recheck (short of a full regression test every single time) to verify everything is still working. Not really. As QA I always reviewed the checkins since yesterday before opening up the daily build. Between the bug comments and the patch comments, even if the patch itself is a bit Greek to me, I can tell what was going on enough to be a better tester of that area.

cableshaft 3y ago
Let's say that there's some sort of list component that's used in like six other places (that's maybe not obvious because there's different configurations or something). Six months later I work on a new page that incorporates that component that requires some sort of change in that component that could theoretically break it elsewhere, at least how it looks or something, like alters the padding or something in a way that shoves things onto the next line in one of the instances. If I don't mention what other places that component is used in my comments, are you going to know where all to look for the other instances of that component? Even if you look at the source code, it will only show changes, it won't say 'oh this component is used elsewhere' unless you actively bring up the source code yourself and do a code search yourself. Because it's a visible component, maybe you'll just have the memory that something that looks like this is elsewhere, but maybe you won't remember. Or maybe you're fairly new to the project and aren't aware of these other parts of the site using it. What about something that's even less visible, like something that takes in data and then sorts, transforms, and adds metadata to it? Then I make a change to it to accomodate for some new requirement on a new page. Maybe it's used for four other locations, but the changes only shows the function being altered and the new page using it. That change I made broke it elsewhere, but I don't mention where else to check in my notes. Are you going to think to check each place where data is loaded to make sure it's working correctly, without me explicitly telling you to check those spots? If you can, great, you've got a real intuition for this that I doubt many QA people have. I don't even know to check these other places sometimes, and I worked on the code myself and can do a quick text search to see where else it's being used. This was why I said what I did. It's not just about 'that area', the change could potentially affect another application in different repository entirely sometimes, especially if a shared library is involved. I've literally had like four or five of these situations happen in the past six months of my current project, where my change broke something elsewhere. Usually I caught it, but I also told QA to check it explicitly anyway since I don't usually test things 100% (since I'm not QA). When I was a more junior developer, I missed these things more often and QA would sometimes miss it also since they didn't know to retest something.