6 ms·

Exploratory testing by experienced testers is extremely valuable. I am a dev but have done some testing. I could break the systems in various ways because I had

Exploratory testing by experienced testers is extremely valuable. I am a dev but have done some testing. I could break the systems in various ways because I had a pretty good feel for what can go wrong. Beginner users are also very good because they do stuff you never expected.

eru 3y ago
> Beginner users are also very good because they do stuff you never expected. People using software for a while undergo operant conditioning: consciously or subconsciously they quickly learn to internalise what makes the software 'angry', and learn to avoid that. That makes them more productive users, but also less useful as explorers. See https://blog.regehr.org/archives/861 https://blog.regehr.org/archives/861 > Have you ever used a new program or system and found it to be obnoxiously buggy, but then after a while you didn’t notice the bugs anymore? If so, then congratulations: you have been trained by the computer to avoid some of its problems. For example, I used to have a laptop that would lock up to the point where the battery needed to be removed when I scrolled down a web page for too long (I’m guessing the video driver’s logic for handling a full command queue was defective). Messing with the driver version did not solve the problem and I soon learned to take little breaks when scrolling down a long web page. To this day I occasionally feel a twinge of guilt or fear when rapidly scrolling a web page. > The extent to which we technical people have become conditioned by computers became apparent to me when one of my kids, probably three years old at the time, sat down at a Windows machine and within minutes rendered the GUI unresponsive. Even after watching which keys he pressed, I was unable to reproduce this behavior, at least partially because decades of training in how to use a computer have made it very hard for me to use one in such an inappropriate fashion. By now, this child (at 8 years old) has been brought into the fold: like millions of other people he can use a Windows machine for hours at a time without killing it.
- julian_t 3y ago
  I think this is true whenever we learn to interact with a system. When I was learning the saxophone I had real trouble getting one note to sound properly, and it seemed to me that there was a leak on that pad. My teacher couldn't get my instrument to misbehave at all. He said there probably was something slightly wrong with that pad, but over the years he'd learned to play 'with' an instrument, adjusting for its particular behavior. The same goes for things like driving a temperamental car.
switchbak 3y ago
I’ve met Elisabeth, and i think she’s a fantastic thought leader for exploratory testing. Sadly my efforts to promote this practice were met with a dismissive response: “QA leads to lower quality, according to this study”. I just didn’t have the energy to try to change this person’s opinion. I have seen exploratory testing in concert with good developer testing work extremely well. Old school SQA isn’t really my thing, but modern exploratory testers are tremendously valuable.
- jdlshore 3y ago
  Elisabeth is the author of the article they may have been unknowingly quoting: “Better Testing, Worse Quality.”
  - switchbak 3y ago
    I don't think so, I don't know the exact article they were referencing but it was definitely post-2000. I think something more along the lines of this: https://www.forrester.com/blogs/11-02-17-want_better_quality_fire_your_qa_team/ https://www.forrester.com/blogs/11-02-17-want_better_quality... It was a very strawman argument, as if we could only have a 90's style split where QA only did testing, or you could have modern developer testing and no QA. I think modern dev testing + exploratory testing is the best I've experienced, but it has been challenging to explain that. Software organizations are imperfect places, and sometimes it's hard to push against popular trends.
- shiroiuma 3y ago
  >“QA leads to lower quality, according to this study”. Was this produced by the same people who published a study claiming that flossing your teeth is bad for you, or the study that claimed that wearing a bicycle helmet is dangerous?
  - YZF 3y ago
    "QA" isn't as precise as wear a helmet while riding your bicycle or floss your teeth every night before going to sleep. It refers to vague practices that vary greatly from organization to organization and team to team. There are several anti-patterns that sometimes fit under this vagueness: - Throw garbage over the fence. I.e. low quality products thrown over the fence to a QA team. - Over-reliance on manual testing. - Quality is somebody else's problem (sort of related to the first anti-pattern but not exactly the same). Even the term "Quality" is usually too vague to make sense of. What is quality exactly? How do you tell if your product is higher or lower quality than the competing product? Organizations tend to declare they're the best, regardless of practices and with no real benchmark.

eru 3y ago

> Beginner users are also very good because they do stuff you never expected. People using software for a while undergo operant conditioning: consciously or subconsciously they quickly learn to internalise what makes the software 'angry', and learn to avoid that. That makes them more productive users, but also less useful as explorers. See https://blog.regehr.org/archives/861 https://blog.regehr.org/archives/861 > Have you ever used a new program or system and found it to be obnoxiously buggy, but then after a while you didn’t notice the bugs anymore? If so, then congratulations: you have been trained by the computer to avoid some of its problems. For example, I used to have a laptop that would lock up to the point where the battery needed to be removed when I scrolled down a web page for too long (I’m guessing the video driver’s logic for handling a full command queue was defective). Messing with the driver version did not solve the problem and I soon learned to take little breaks when scrolling down a long web page. To this day I occasionally feel a twinge of guilt or fear when rapidly scrolling a web page. > The extent to which we technical people have become conditioned by computers became apparent to me when one of my kids, probably three years old at the time, sat down at a Windows machine and within minutes rendered the GUI unresponsive. Even after watching which keys he pressed, I was unable to reproduce this behavior, at least partially because decades of training in how to use a computer have made it very hard for me to use one in such an inappropriate fashion. By now, this child (at 8 years old) has been brought into the fold: like millions of other people he can use a Windows machine for hours at a time without killing it.

julian_t 3y ago
I think this is true whenever we learn to interact with a system. When I was learning the saxophone I had real trouble getting one note to sound properly, and it seemed to me that there was a leak on that pad. My teacher couldn't get my instrument to misbehave at all. He said there probably was something slightly wrong with that pad, but over the years he'd learned to play 'with' an instrument, adjusting for its particular behavior. The same goes for things like driving a temperamental car.

switchbak 3y ago

I’ve met Elisabeth, and i think she’s a fantastic thought leader for exploratory testing. Sadly my efforts to promote this practice were met with a dismissive response: “QA leads to lower quality, according to this study”. I just didn’t have the energy to try to change this person’s opinion. I have seen exploratory testing in concert with good developer testing work extremely well. Old school SQA isn’t really my thing, but modern exploratory testers are tremendously valuable.

jdlshore 3y ago
Elisabeth is the author of the article they may have been unknowingly quoting: “Better Testing, Worse Quality.”
- switchbak 3y ago
  I don't think so, I don't know the exact article they were referencing but it was definitely post-2000. I think something more along the lines of this: https://www.forrester.com/blogs/11-02-17-want_better_quality_fire_your_qa_team/ https://www.forrester.com/blogs/11-02-17-want_better_quality... It was a very strawman argument, as if we could only have a 90's style split where QA only did testing, or you could have modern developer testing and no QA. I think modern dev testing + exploratory testing is the best I've experienced, but it has been challenging to explain that. Software organizations are imperfect places, and sometimes it's hard to push against popular trends.
shiroiuma 3y ago
>“QA leads to lower quality, according to this study”. Was this produced by the same people who published a study claiming that flossing your teeth is bad for you, or the study that claimed that wearing a bicycle helmet is dangerous?
- YZF 3y ago
  "QA" isn't as precise as wear a helmet while riding your bicycle or floss your teeth every night before going to sleep. It refers to vague practices that vary greatly from organization to organization and team to team. There are several anti-patterns that sometimes fit under this vagueness: - Throw garbage over the fence. I.e. low quality products thrown over the fence to a QA team. - Over-reliance on manual testing. - Quality is somebody else's problem (sort of related to the first anti-pattern but not exactly the same). Even the term "Quality" is usually too vague to make sense of. What is quality exactly? How do you tell if your product is higher or lower quality than the competing product? Organizations tend to declare they're the best, regardless of practices and with no real benchmark.