5 ms·

> The Joel Test is now considered to be obsolete because it awards points for things like "Do you have testers?", which aren't considered best practices by most

> The Joel Test is now considered to be obsolete because it awards points for things like "Do you have testers?", which aren't considered best practices by most devs today. Is having dedicated testers really not a best practice? I mean if I compare my experience between projects where we had testers and projects where we didn't, I felt we had better results when we did, if only because it'd free up lot of time from the developpers, who don't have to run a lot of tests and the testers usually think of edge case that we overlook. One downside of having testers is that it kinda slowed down the process of test automation, but in that particular case, it's not the only cause.

rsj_hn 5y ago
> Is having dedicated testers really not a best practice? It is still a best practice. A lot of companies are doing crazy things that Joel would not agree with -- for example, adopting open office layouts, dropping their QA department, and constantly rewriting code. Joel is not wrong that these are bad ideas, regardless of whether they are widely used or not. Rather, I think it's just more evidence of the growing power of monopolies, that firms can adopt inefficient practices because they are so entrenched.
- jonas21 5y ago
  Well, that's convenient. Whenever companies do things I disagree with and succeed, it's evidence they must be monopolies!
  - adamisom 5y ago
    Indeed that could be evidence depending on the strength of the beliefs underlying the disagreement. Imagine the parent commenter simply sought out expert consensus and that the consensus was unmistakable; surely you’d agree that if some practice “disagreed”, there’s a higher than average chance something unusual is going on, like monopoly; that’s “evidence”.
    - fastball 5y ago
      Only if the "experts" had strong objective evidence for their consensus in the first place, which is usually not the case. If you have objective evidence you don't need experts.
    - roenxi 5y ago
      It isn't really evidence of much. One of the most reliable consensus points of experts is "you need to hire lots of people in my field of speciality and they should be treated really well and their work should be regarded as important!". Not an iron-clad law, but a reliable trend. I'll single out "dropping their QA department" as an example of exactly the sort of situation where expert consensus would quite possibly sit in opposition to a good business outcome. If someone wants to argue there is a monopoly, the evidence should be of a "competition is illegal or physically impossible" nature. Not disagreeing with business practices. Business practices are important but not the be all and end all of success.
mattchamb 5y ago
Testers absolutely still have a role to play, but not as manual test execution robots. That stuff can be automated. I find value in skilled testers in exploratory testing, sanity checks on if product flows make sense and helping define scope on what things are best covered by automation
Zababa 5y ago
At work we usually manual tests, and we try to have them done by someone that didn't develop this specific feature. I feel like some testers would help here, and would also help reduce the need for developers that they can't recruit fast enough. They transitionned a decade ago from waterfall to agile, and removed the testing team. Since what we're doing is basically waterfall on a 2 week cycle, I think dedicated testers would make sense. We have dedicated documentation and localization people. I feel like we're underestimate the work of a good tester by thinking it can easily be done by a developer, and that we're undervaluing the time of the developers.
kragen 5y ago
Having dedicated testers is great, but not the kind Joel is talking about. He's talking about low-paid manual laborers whose work has mostly been automated, not accomplished programmers whose work is considered just as valuable as that of writing the production code they're responsible for testing, and who spend a lot of their time on automating the test process. Joel's argument is specifically that testers are cheaper than programmers, so it's silly to have programmers test their own code instead of having testers do it. Contrast Dan Luu's internship work as a tester: > Most of them were only slightly less manual than "testing" a phone by mashing buttons, but I did one not-totally-uninteresting task, which was to verify that a flash chip's controller behaved correctly. Unlike my other tasks, this was amenable to automation and I was able to write a Perl script to do the testing for me. ... they had me do verification on the logic they were writing. They had a relatively new tool called SixthSense, which you can think of as a souped-up quickcheck. The obvious skill I learned was how to write tests using a fancy testing framework, but the meta-thing I learned which has been even more useful is the fact that writing a test-case generator and a checker is often much more productive than the manual test-case writing that passes for automated testing in most places
- 3np 5y ago
  It also depends on what you're developing. In games, in particular, dedicated testers/QA are very much part of the norm and best practices in a different way compared to web apps.
  - kragen 5y ago
    That is an excellent point.
- dfabulich 5y ago
  Joel is specifically referring to the kind of testers that Microsoft had at the time the Joel Test was written (Aug 2000). At that time, for Windows and Office, Microsoft had testers who would write their own test plans (and then execute the tests themselves). This is not the low-paid manual labor you're thinking of where a test manager writes a test plan and then delegates it to folks who are expected to just follow instructions. The "manual labor" style of testing was/is typical of game dev companies, where it's easy to trick inexperienced kids into working for very cheap or even free, just to have early access to upcoming games and the (mostly illusory) opportunity to land a "real job" in the game industry.
rpastuszak 5y ago
In my experience xp with pair programming and TDD turned out to be much faster than relying on testers. Relying on QA (besides certain, rare, edge cases) feels like a bottleneck now.
- telman17 5y ago
  I agree with you on the bottleneck. Our QA is split into automation coders/testers and manual testers, the former of whom never have enough people for complete coverage and the latter of whom are not very technical and the feature developers often have to walk them through exactly how to test the features, which takes even more time away from core feature building.
Bombthecat 5y ago
Not having testers is cheap though.. At first at least..
KnobbleMcKnees 5y ago
We have dedicated QAs and QEs, but they are not there to write tests. It's possible that's what he's driving at as I think the Joel test saw testers as an integrated part of the software delivery pipeline. Nowadays, in big tech, they are more there to drive high level tooling and testing culture - engineers are responsible for writing tests. At least that's my experience.

rsj_hn 5y ago

> Is having dedicated testers really not a best practice? It is still a best practice. A lot of companies are doing crazy things that Joel would not agree with -- for example, adopting open office layouts, dropping their QA department, and constantly rewriting code. Joel is not wrong that these are bad ideas, regardless of whether they are widely used or not. Rather, I think it's just more evidence of the growing power of monopolies, that firms can adopt inefficient practices because they are so entrenched.

jonas21 5y ago
Well, that's convenient. Whenever companies do things I disagree with and succeed, it's evidence they must be monopolies!
- adamisom 5y ago
  Indeed that could be evidence depending on the strength of the beliefs underlying the disagreement. Imagine the parent commenter simply sought out expert consensus and that the consensus was unmistakable; surely you’d agree that if some practice “disagreed”, there’s a higher than average chance something unusual is going on, like monopoly; that’s “evidence”.
  - fastball 5y ago
    Only if the "experts" had strong objective evidence for their consensus in the first place, which is usually not the case. If you have objective evidence you don't need experts.
  - roenxi 5y ago
    It isn't really evidence of much. One of the most reliable consensus points of experts is "you need to hire lots of people in my field of speciality and they should be treated really well and their work should be regarded as important!". Not an iron-clad law, but a reliable trend. I'll single out "dropping their QA department" as an example of exactly the sort of situation where expert consensus would quite possibly sit in opposition to a good business outcome. If someone wants to argue there is a monopoly, the evidence should be of a "competition is illegal or physically impossible" nature. Not disagreeing with business practices. Business practices are important but not the be all and end all of success.

mattchamb 5y ago

Testers absolutely still have a role to play, but not as manual test execution robots. That stuff can be automated. I find value in skilled testers in exploratory testing, sanity checks on if product flows make sense and helping define scope on what things are best covered by automation

Zababa 5y ago

At work we usually manual tests, and we try to have them done by someone that didn't develop this specific feature. I feel like some testers would help here, and would also help reduce the need for developers that they can't recruit fast enough. They transitionned a decade ago from waterfall to agile, and removed the testing team. Since what we're doing is basically waterfall on a 2 week cycle, I think dedicated testers would make sense. We have dedicated documentation and localization people. I feel like we're underestimate the work of a good tester by thinking it can easily be done by a developer, and that we're undervaluing the time of the developers.

kragen 5y ago

Having dedicated testers is great, but not the kind Joel is talking about. He's talking about low-paid manual laborers whose work has mostly been automated, not accomplished programmers whose work is considered just as valuable as that of writing the production code they're responsible for testing, and who spend a lot of their time on automating the test process. Joel's argument is specifically that testers are cheaper than programmers, so it's silly to have programmers test their own code instead of having testers do it. Contrast Dan Luu's internship work as a tester: > Most of them were only slightly less manual than "testing" a phone by mashing buttons, but I did one not-totally-uninteresting task, which was to verify that a flash chip's controller behaved correctly. Unlike my other tasks, this was amenable to automation and I was able to write a Perl script to do the testing for me. ... they had me do verification on the logic they were writing. They had a relatively new tool called SixthSense, which you can think of as a souped-up quickcheck. The obvious skill I learned was how to write tests using a fancy testing framework, but the meta-thing I learned which has been even more useful is the fact that writing a test-case generator and a checker is often much more productive than the manual test-case writing that passes for automated testing in most places

3np 5y ago
It also depends on what you're developing. In games, in particular, dedicated testers/QA are very much part of the norm and best practices in a different way compared to web apps.
- kragen 5y ago
  That is an excellent point.
dfabulich 5y ago
Joel is specifically referring to the kind of testers that Microsoft had at the time the Joel Test was written (Aug 2000). At that time, for Windows and Office, Microsoft had testers who would write their own test plans (and then execute the tests themselves). This is not the low-paid manual labor you're thinking of where a test manager writes a test plan and then delegates it to folks who are expected to just follow instructions. The "manual labor" style of testing was/is typical of game dev companies, where it's easy to trick inexperienced kids into working for very cheap or even free, just to have early access to upcoming games and the (mostly illusory) opportunity to land a "real job" in the game industry.

rpastuszak 5y ago

In my experience xp with pair programming and TDD turned out to be much faster than relying on testers. Relying on QA (besides certain, rare, edge cases) feels like a bottleneck now.

telman17 5y ago
I agree with you on the bottleneck. Our QA is split into automation coders/testers and manual testers, the former of whom never have enough people for complete coverage and the latter of whom are not very technical and the feature developers often have to walk them through exactly how to test the features, which takes even more time away from core feature building.

Bombthecat 5y ago

Not having testers is cheap though.. At first at least..

KnobbleMcKnees 5y ago

We have dedicated QAs and QEs, but they are not there to write tests. It's possible that's what he's driving at as I think the Joel test saw testers as an integrated part of the software delivery pipeline. Nowadays, in big tech, they are more there to drive high level tooling and testing culture - engineers are responsible for writing tests. At least that's my experience.