5 ms·

I’ve met Elisabeth, and i think she’s a fantastic thought leader for exploratory testing. Sadly my efforts to promote this practice were met with a dismissive

I’ve met Elisabeth, and i think she’s a fantastic thought leader for exploratory testing. Sadly my efforts to promote this practice were met with a dismissive response: “QA leads to lower quality, according to this study”. I just didn’t have the energy to try to change this person’s opinion. I have seen exploratory testing in concert with good developer testing work extremely well. Old school SQA isn’t really my thing, but modern exploratory testers are tremendously valuable.

jdlshore 3y ago
Elisabeth is the author of the article they may have been unknowingly quoting: “Better Testing, Worse Quality.”
- switchbak 3y ago
  I don't think so, I don't know the exact article they were referencing but it was definitely post-2000. I think something more along the lines of this: https://www.forrester.com/blogs/11-02-17-want_better_quality_fire_your_qa_team/ https://www.forrester.com/blogs/11-02-17-want_better_quality... It was a very strawman argument, as if we could only have a 90's style split where QA only did testing, or you could have modern developer testing and no QA. I think modern dev testing + exploratory testing is the best I've experienced, but it has been challenging to explain that. Software organizations are imperfect places, and sometimes it's hard to push against popular trends.
shiroiuma 3y ago
>“QA leads to lower quality, according to this study”. Was this produced by the same people who published a study claiming that flossing your teeth is bad for you, or the study that claimed that wearing a bicycle helmet is dangerous?
- YZF 3y ago
  "QA" isn't as precise as wear a helmet while riding your bicycle or floss your teeth every night before going to sleep. It refers to vague practices that vary greatly from organization to organization and team to team. There are several anti-patterns that sometimes fit under this vagueness: - Throw garbage over the fence. I.e. low quality products thrown over the fence to a QA team. - Over-reliance on manual testing. - Quality is somebody else's problem (sort of related to the first anti-pattern but not exactly the same). Even the term "Quality" is usually too vague to make sense of. What is quality exactly? How do you tell if your product is higher or lower quality than the competing product? Organizations tend to declare they're the best, regardless of practices and with no real benchmark.
  - shiroiuma 3y ago
    I don't see how having a QA team can actually result in worse quality though. Even for your "throw garbage over the fence" (something I've seen first-hand actually), the QA team is improving quality by finding the bugs in all the garbage thrown over the fence. Sure, getting the dev teams to stop throwing garbage would be better, but if management doesn't want to do that for some weird reason, and wants QA to serve as the bug-fixing org in place of having devs actually test their code before merging to master, not having the QA team at all would be measurably worse. It's a bit like the bike helmet controversy. The anti-helmet people from Netherlands all claim that helmets somehow force cyclists to ride faster or more dangerously, or force car drivers to take greater risks around them. (Incidentally, this is very similar to the anti-seatbelt arguments I heard decades ago.) The helmet isn't the problem here though, even if there's any truth to these claims: it's the humans. The helmet will absolutely protect your skull better when it impacts concrete than no helmet.
    - YZF 3y ago
      The problem is that these are systems. I.e. you're right that in a given system if all else is constant the presence of some QA/testing step should improve quality. But everything else is not constant. But yeah, point taken on the bike helmet arguments, though I think that's a little more clear cut. I think QA (manual software testing by professional testers) has an important role in producing quality software but there are organizational corner cases where the idea of "QA" in software promotes the wrong thinking/approach to quality that results in an overall poor outcome. Quality in software starts before a single line of code is written and has to be in mind throughout the development process so you shouldn't fall into a trap that somehow slapping "Quality Assurance" on top of bad engineering can create quality after the fact.
      - shiroiuma 3y ago
        Just because some people get the wrong idea when something exists, doesn't mean the existence of that thing is an actual problem or detriment. It's true, slapping "QA" on top of bad engineering isn't going to create quality. However, "having quality in mind throughout the development process" isn't going to guarantee it either, especially in a large and complex codebase with modules from many different teams. A QA org is useful for doing things like full-system testing, which can catch bugs that those individual teams might miss because they aren't testing the entire system. It's also useful to have dedicated testers because they'll test things out in ways the devs never thought of. Again, this isn't a replacement for quality engineering at the outset, but that's a management problem.

jdlshore 3y ago

Elisabeth is the author of the article they may have been unknowingly quoting: “Better Testing, Worse Quality.”

switchbak 3y ago
I don't think so, I don't know the exact article they were referencing but it was definitely post-2000. I think something more along the lines of this: https://www.forrester.com/blogs/11-02-17-want_better_quality_fire_your_qa_team/ https://www.forrester.com/blogs/11-02-17-want_better_quality... It was a very strawman argument, as if we could only have a 90's style split where QA only did testing, or you could have modern developer testing and no QA. I think modern dev testing + exploratory testing is the best I've experienced, but it has been challenging to explain that. Software organizations are imperfect places, and sometimes it's hard to push against popular trends.

shiroiuma 3y ago

>“QA leads to lower quality, according to this study”. Was this produced by the same people who published a study claiming that flossing your teeth is bad for you, or the study that claimed that wearing a bicycle helmet is dangerous?

YZF 3y ago
"QA" isn't as precise as wear a helmet while riding your bicycle or floss your teeth every night before going to sleep. It refers to vague practices that vary greatly from organization to organization and team to team. There are several anti-patterns that sometimes fit under this vagueness: - Throw garbage over the fence. I.e. low quality products thrown over the fence to a QA team. - Over-reliance on manual testing. - Quality is somebody else's problem (sort of related to the first anti-pattern but not exactly the same). Even the term "Quality" is usually too vague to make sense of. What is quality exactly? How do you tell if your product is higher or lower quality than the competing product? Organizations tend to declare they're the best, regardless of practices and with no real benchmark.
- shiroiuma 3y ago
  I don't see how having a QA team can actually result in worse quality though. Even for your "throw garbage over the fence" (something I've seen first-hand actually), the QA team is improving quality by finding the bugs in all the garbage thrown over the fence. Sure, getting the dev teams to stop throwing garbage would be better, but if management doesn't want to do that for some weird reason, and wants QA to serve as the bug-fixing org in place of having devs actually test their code before merging to master, not having the QA team at all would be measurably worse. It's a bit like the bike helmet controversy. The anti-helmet people from Netherlands all claim that helmets somehow force cyclists to ride faster or more dangerously, or force car drivers to take greater risks around them. (Incidentally, this is very similar to the anti-seatbelt arguments I heard decades ago.) The helmet isn't the problem here though, even if there's any truth to these claims: it's the humans. The helmet will absolutely protect your skull better when it impacts concrete than no helmet.
  - YZF 3y ago
    The problem is that these are systems. I.e. you're right that in a given system if all else is constant the presence of some QA/testing step should improve quality. But everything else is not constant. But yeah, point taken on the bike helmet arguments, though I think that's a little more clear cut. I think QA (manual software testing by professional testers) has an important role in producing quality software but there are organizational corner cases where the idea of "QA" in software promotes the wrong thinking/approach to quality that results in an overall poor outcome. Quality in software starts before a single line of code is written and has to be in mind throughout the development process so you shouldn't fall into a trap that somehow slapping "Quality Assurance" on top of bad engineering can create quality after the fact.
    - shiroiuma 3y ago
      Just because some people get the wrong idea when something exists, doesn't mean the existence of that thing is an actual problem or detriment. It's true, slapping "QA" on top of bad engineering isn't going to create quality. However, "having quality in mind throughout the development process" isn't going to guarantee it either, especially in a large and complex codebase with modules from many different teams. A QA org is useful for doing things like full-system testing, which can catch bugs that those individual teams might miss because they aren't testing the entire system. It's also useful to have dedicated testers because they'll test things out in ways the devs never thought of. Again, this isn't a replacement for quality engineering at the outset, but that's a management problem.