3 ms·

This is exactly what I thought would happen when they gutted their QA staff and reassigned the surviving testers as developers. Your employees act in accordance

This is exactly what I thought would happen when they gutted their QA staff and reassigned the surviving testers as developers. Your employees act in accordance with their incentives, and the incentives aren't there for QA. Microsoft made it clear that they only value feature development and shipping, and their employees now associate "doing QA" with "being fired en masse". To corrupt a phrase, they've crapped their bed and now they have to lie in it. Edited to add: I'm not some hardcore "M$FT SUXXXX" guy. I like and use their products, but I'm deeply concerned about the direction the company seems to be heading.

yuhong 12y ago
I think QA is still rewarded, but it probably will still not be done as well as it was done before. Eg. I wonder if the problems with KB3004394 for Win7 showed up on the dev machines with the test root installed.
- bcbrown 12y ago
  I disagree. Back before the reorg and layoffs, there was a clear prestige difference between Test and Dev, where most SDETs were levels 1 or 2 with one Senior per team, while the SDEs had 2-3 times as many Senios, and one Principal per team. It was clear that the ceiling for promotion was much lower in Test. I left before they began doing away with the SDET discipline, but I'm sure that QA work is still not very prestigious.
  - pacaro 12y ago
    I also get the impression that they have been handing out promotions like candy recently. I know that the system was broken before and people were withe leaving or threatening to leave to get promoted, but the flipside to fixing the review system too late is that you may be promoting the wrong people. Put another way, many of the level 64 SDEs I knew are now Principle (65+) for some this is long overdue, but for others it is unwarranted. Over-promotion can do horrible damage to a team, particularly when it moves someone into a broader decision making position than they ate capable of handling. Some of the QA issues may well stem from this
AceJohnny2 12y ago
This is something that deeply bothers me in the software industry, the segregation between Dev and QA, and the low value placed in QA. "Oh whatever, QA are low-paid techs that just run scripts!" No. QA is vital to the functionality of the product, and a proper QA person should be just as capable a programmer as a developer in order to 1) write and maintain a test suite and 2) understand the tested program and how best to test it. My previous company in Europe understood this, and that seemed the norm in the surrounding industry. My current in America doesn't, and that seems the norm in the Silicon Valley.
- geoelectric 12y ago
  I'm a Quality Engineer, and have worked for a number of large and small companies in the Valley after moving into quality from software engineering 13 years ago. I've seen a lot of teams. Problem is it's a feedback loop. A lot of companies maintain (or did maintain) armies of unskilled people who basically were script executors, with maybe a bit of barely-trained skill in finding equivalence classes and boundary testing so that basic test plans could be written. I wouldn't even call this QA, even if it was misidentified as such--it's Quality Control to catch problems after the fact, with no front end assurance whatsoever. It's honestly really only good for pissing off people when you stop the shipment at the last minute. And that lack of value has been broadly recognized. When you combine the high cost associated with the brute-force documentation associated with verbose test scripts, result templates, and other prehistoric process artifacts aimed at this level of tester (and which are usually simply unnecessary and actually detrimental to agility and pace) it's no wonder they're being cut. Problem is as classic brute force QA teams get gutted, the market has been flooded with these testers, and most of them kind of suck. Hiring a decent quality professional has gotten damned difficult. And when everyone you meet with "QA" in their resume is a non-technical, uninfluential yo-yo, you get a pretty bad impression of the discipline. In the meantime, everyone as skilled, interested, or immersed as you describe has been moved into software engineering, because QA was treated as an "entry-level" position and nobody wants to be a non-technical, uninfluential yo-yo. The rise of the Software Engineer in Test is as much a collective gambit to keep skilled people in quality as it is identifying a new skill-set. And for people in my line of work, identifying yourself as a SET has about a 30% pay differential over identifying yourself as a QA Engineer, so there's plenty of incentive to move in that direction. But even then, the rise of the SET reflects an over-reliance on automation, and an under-reliance on process-based quality improvement and the type of fuzzy testing and intelligent guessing only a human is going to reasonably provide. The net result is that edge and corner cases escape, because automation is pretty shitty at catching those unless a fuzzer or an automated monkey (which is essentially a UI fuzzer) can be used. Generally, you only find problems where you've programmed it to look, which will be necessarily be pretty narrow, and the automation cannot use discretion to vary off the script to where new problems actually might lie. People have conflated the value of QA with the value of bad QA, and have chucked it out the window without understanding the cost. I'd guess that many have never even seen good QA to understand what it can do. Nevertheless, this particular story is egregious. However much you cut back on quality control or assurance, the ONE FUCKING THING you always guarantee is that the user can get out of any problems that arise with minimum damage to money, data, time, and customer goodwill, roughly in that order of priority. Bugs in updates that kill your code signing stack and block further updates are just plain unforgivable.

yuhong 12y ago

I think QA is still rewarded, but it probably will still not be done as well as it was done before. Eg. I wonder if the problems with KB3004394 for Win7 showed up on the dev machines with the test root installed.

bcbrown 12y ago
I disagree. Back before the reorg and layoffs, there was a clear prestige difference between Test and Dev, where most SDETs were levels 1 or 2 with one Senior per team, while the SDEs had 2-3 times as many Senios, and one Principal per team. It was clear that the ceiling for promotion was much lower in Test. I left before they began doing away with the SDET discipline, but I'm sure that QA work is still not very prestigious.
- pacaro 12y ago
  I also get the impression that they have been handing out promotions like candy recently. I know that the system was broken before and people were withe leaving or threatening to leave to get promoted, but the flipside to fixing the review system too late is that you may be promoting the wrong people. Put another way, many of the level 64 SDEs I knew are now Principle (65+) for some this is long overdue, but for others it is unwarranted. Over-promotion can do horrible damage to a team, particularly when it moves someone into a broader decision making position than they ate capable of handling. Some of the QA issues may well stem from this

AceJohnny2 12y ago

This is something that deeply bothers me in the software industry, the segregation between Dev and QA, and the low value placed in QA. "Oh whatever, QA are low-paid techs that just run scripts!" No. QA is vital to the functionality of the product, and a proper QA person should be just as capable a programmer as a developer in order to 1) write and maintain a test suite and 2) understand the tested program and how best to test it. My previous company in Europe understood this, and that seemed the norm in the surrounding industry. My current in America doesn't, and that seems the norm in the Silicon Valley.

geoelectric 12y ago
I'm a Quality Engineer, and have worked for a number of large and small companies in the Valley after moving into quality from software engineering 13 years ago. I've seen a lot of teams. Problem is it's a feedback loop. A lot of companies maintain (or did maintain) armies of unskilled people who basically were script executors, with maybe a bit of barely-trained skill in finding equivalence classes and boundary testing so that basic test plans could be written. I wouldn't even call this QA, even if it was misidentified as such--it's Quality Control to catch problems after the fact, with no front end assurance whatsoever. It's honestly really only good for pissing off people when you stop the shipment at the last minute. And that lack of value has been broadly recognized. When you combine the high cost associated with the brute-force documentation associated with verbose test scripts, result templates, and other prehistoric process artifacts aimed at this level of tester (and which are usually simply unnecessary and actually detrimental to agility and pace) it's no wonder they're being cut. Problem is as classic brute force QA teams get gutted, the market has been flooded with these testers, and most of them kind of suck. Hiring a decent quality professional has gotten damned difficult. And when everyone you meet with "QA" in their resume is a non-technical, uninfluential yo-yo, you get a pretty bad impression of the discipline. In the meantime, everyone as skilled, interested, or immersed as you describe has been moved into software engineering, because QA was treated as an "entry-level" position and nobody wants to be a non-technical, uninfluential yo-yo. The rise of the Software Engineer in Test is as much a collective gambit to keep skilled people in quality as it is identifying a new skill-set. And for people in my line of work, identifying yourself as a SET has about a 30% pay differential over identifying yourself as a QA Engineer, so there's plenty of incentive to move in that direction. But even then, the rise of the SET reflects an over-reliance on automation, and an under-reliance on process-based quality improvement and the type of fuzzy testing and intelligent guessing only a human is going to reasonably provide. The net result is that edge and corner cases escape, because automation is pretty shitty at catching those unless a fuzzer or an automated monkey (which is essentially a UI fuzzer) can be used. Generally, you only find problems where you've programmed it to look, which will be necessarily be pretty narrow, and the automation cannot use discretion to vary off the script to where new problems actually might lie. People have conflated the value of QA with the value of bad QA, and have chucked it out the window without understanding the cost. I'd guess that many have never even seen good QA to understand what it can do. Nevertheless, this particular story is egregious. However much you cut back on quality control or assurance, the ONE FUCKING THING you always guarantee is that the user can get out of any problems that arise with minimum damage to money, data, time, and customer goodwill, roughly in that order of priority. Bugs in updates that kill your code signing stack and block further updates are just plain unforgivable.