5 ms·

Google v. Microsoft, and the dev:test ratio debate

12 points by greyman 18y ago

aditya 18y ago
Eh? Isn't the sweet spot, like, every dev writing their own tests?! I think writing tests makes you think a lot more about your code and how it will be used than just writing code ever will, obviously this is missing from the kind of software that Microsoft has historically produced... I'm not a big fan of TATFT and/or purely agile methodologies, but, separating the dev/test roles just doesn't seem right anymore.
- Retric 18y ago
  You separate testers from coders because many bugs relate to the developer misunderstanding the goal. I have no problem with developers suggesting additional tests or writing their own test scrips, but you need someone else to look things over and say "that works". PS: You can also have developers testing other developers code, but IMO that's just semantics.
  - aditya 18y ago
    Wouldn't it be a better idea to make developers understand the goal better, then? It sounds backward to add another person to a team because the other people on it don't completely understand what they're doing?
    - Retric 18y ago
      How would you validate that the developers understood the goal? You don't need 1:1 dev / testers for this but someone else needs to look at the output or your users are going to be your testers. PS: Think about the novelist vs. editor divide.
- DenisM 18y ago
  Different jobs might require different mindset. Also, Google is desperately trying to hire away as many testers as they can - many testers I know at Microsoft were bombarded with Google's invitations for interview.
- snprbob86 18y ago
  They are really quite different jobs. The devs on my team (at Microsoft) write their own unit tests and are responsible for the quality of their code. The testers, however, own sign off. The tests they write are significantly different: they involve testing the product as a whole instead of just a single module. Especially in large GUI applications, such as Visual Studio, testers have the extremely challenging task of automating tests for high level end-to-end scenerios. I'm not sure if I have an opinion of what the "correct" ratio should be, but I do know that it is a good idea to have the person saying "OK, we can ship this" to be different than the person saying "Yeah, it works for me!" When you work on web software and customers start complaining about your shoddy update, you can issue a patch before the end of your coffee break. When you press your software onto millions of disks and promise to support those bits for a decade, you better be damn sure it works and meets the requirements.
lacker 18y ago
It really depends on what sort of software you are writing. If you are writing user interface code, there are many different ways a user can screw up your program. Maybe they are holding down "alt" while right-clicking and dragging to highlight half of a misspelled word. In this situation, you really need a lot of testers. And this is a lot of what Microsoft does, local applications like Office and the operating system itself. By contrast, websites avoid a lot of this sort of testing. You don't have to test what happens if a user clicks on a link in a strange way - that's the responsibility of the browser. Or if you work on a part of the Google search engine, you can replay query traffic to see if it crashes. That will test a heck of a lot more corner cases than any tester. With web-delivered services you can also fix glitches as you find them, which minimizes the cost of missing a bug.
blasdel 18y ago
The industry practice of having dedicated testers is intentional class stratification. They don't have to pay testers as much, testers are interchangeable, and they can use them as entry-level drudgery positions.

12 points by greyman 18y ago

Eh? Isn't the sweet spot, like, every dev writing their own tests?! I think writing tests makes you think a lot more about your code and how it will be used than just writing code ever will, obviously this is missing from the kind of software that Microsoft has historically produced... I'm not a big fan of TATFT and/or purely agile methodologies, but, separating the dev/test roles just doesn't seem right anymore.

Retric 18y ago
You separate testers from coders because many bugs relate to the developer misunderstanding the goal. I have no problem with developers suggesting additional tests or writing their own test scrips, but you need someone else to look things over and say "that works". PS: You can also have developers testing other developers code, but IMO that's just semantics.
- aditya 18y ago
  Wouldn't it be a better idea to make developers understand the goal better, then? It sounds backward to add another person to a team because the other people on it don't completely understand what they're doing?
  - Retric 18y ago
    How would you validate that the developers understood the goal? You don't need 1:1 dev / testers for this but someone else needs to look at the output or your users are going to be your testers. PS: Think about the novelist vs. editor divide.
DenisM 18y ago
Different jobs might require different mindset. Also, Google is desperately trying to hire away as many testers as they can - many testers I know at Microsoft were bombarded with Google's invitations for interview.
snprbob86 18y ago
They are really quite different jobs. The devs on my team (at Microsoft) write their own unit tests and are responsible for the quality of their code. The testers, however, own sign off. The tests they write are significantly different: they involve testing the product as a whole instead of just a single module. Especially in large GUI applications, such as Visual Studio, testers have the extremely challenging task of automating tests for high level end-to-end scenerios. I'm not sure if I have an opinion of what the "correct" ratio should be, but I do know that it is a good idea to have the person saying "OK, we can ship this" to be different than the person saying "Yeah, it works for me!" When you work on web software and customers start complaining about your shoddy update, you can issue a patch before the end of your coffee break. When you press your software onto millions of disks and promise to support those bits for a decade, you better be damn sure it works and meets the requirements.

lacker 18y ago

It really depends on what sort of software you are writing. If you are writing user interface code, there are many different ways a user can screw up your program. Maybe they are holding down "alt" while right-clicking and dragging to highlight half of a misspelled word. In this situation, you really need a lot of testers. And this is a lot of what Microsoft does, local applications like Office and the operating system itself. By contrast, websites avoid a lot of this sort of testing. You don't have to test what happens if a user clicks on a link in a strange way - that's the responsibility of the browser. Or if you work on a part of the Google search engine, you can replay query traffic to see if it crashes. That will test a heck of a lot more corner cases than any tester. With web-delivered services you can also fix glitches as you find them, which minimizes the cost of missing a bug.

blasdel 18y ago

The industry practice of having dedicated testers is intentional class stratification. They don't have to pay testers as much, testers are interchangeable, and they can use them as entry-level drudgery positions.