4 ms·
The main problem with QA teams is the same problem with IT teams or even management. If they are doing their jobs well they appear to be doing nothing. This of
by godelski 3y ago
The main problem with QA teams is the same problem with IT teams or even management. If they are doing their jobs well they appear to be doing nothing.
This often creates a situation where people need to "justify" their jobs. Usually this happens due to an over reliance upon metrics (see Goodhart's Law) rather than understanding what the metrics are proxying and what the actual purpose of the job is. A bad QA team is one who is overly nitpicky, looking to ensure they have something to say. A good QA team simultaneous checks for quality as well as trains employees to produce higher quality.
I do feel like there is a lack of training going on in the workforce. We had the "95%-ile isn't that good"[0] post on the front page not long ago and literally it is saying "It's easy to get to the top 5% of performers in any field because most performers don't actively train or have active feedback." It's like the difference between being on an amateur sports team vs a professional. Shouldn't businesses be operating like the latter? Constantly training? Should make hiring be viewed differently too, as in "can we turn this person into a top performer" rather than "are they already" because the latter isn't as meaningful as it appears when your environment is vastly different than the one where success was demonstrated.
[0] https://news.ycombinator.com/item?id=38560345 https://news.ycombinator.com/item?id=38560345
- philk10 3y agoWhen I work with new devs I can often trip them up with basic tests of double-clicking, leading spaces, using the back button on Android. They then learn these and from then on these issues dont appear ( well, OK, it might take a couple of times of a ticket being rejected because of these but they do quickly learn all my tricks ) I don't get measured on bugs found so there's no pressure on me to find stupid bugs just to boost my figures.
- hotpotamus 3y agoI had a buddy who liked to do that kind of thing. I think his favorite trick was just to enter nothing in a form and hit enter to see what happens. It's probably his favorite because it deleted the database on some little thing I wrote at one point and we got a good laugh over it and I got a good lesson out of it.
- philk10 3y agoYep, I start off by entering nothing, then just spaces, then special characters and then finally get to entering some typical data
- godelski 3y ago> I don't get measured on bugs found so there's no pressure on me to find stupid bugs just to boost my figures. Sounds like the right incentive structure. If you don't mind, how are you judged? Do you feel like the system you're in is creating the appropriate incentives and actually being effective? Certainly this example is but I'd like to know more details from an expert so I can update my understanding.
- philk10 3y agoThe system is really effective, I wanted to work at a place where the cliche "everyone cares about quality" is actually true and I found it - devs test, designers test, I test, customer has the chance to test the latest build every 2 weeks so that we can check that our quality checks are aligning with theirs. It gets to be a game of 'can the devs get it past my checks' and 'can I find new ways to trip them up' which builds up confidence in each others skill levels.
- godelski 3y agoThat's cool to hear. I also like the periodic table on your team's site haha. Looks like great philosophy
- mattgreenrocks 3y ago> Shouldn't businesses be operating like the latter? Constantly training? There's a rampant cultural mind-virus that argues that 95%th percentile is somehow tons of work (rather than a lack of unforced mistakes), so everyone just writes it off. It's on full display at this very site. Just look on any post involving software quality, and read a bunch of comments suggesting widespread apathy from engineers. Obviously every situation is different, but people seem to be pretty okay with relinquishing agency on these things and just going along with whatever local maxima their org operates in. It's not totally their fault, but they're not blameless either.
- godelski 3y agoYeah it is weird that it is believed that there is a linear scale to work in and quality considering how well known pareto/power distributions are. These distributions are extremely prolific too. I mean we even codify that sentiment in the 80/20 rule or say that 20% of time is writing code and 80% is debugging it. What's interesting is this effect is scalable. Like you see this when comparing countries by population but the same distribution (general shape) exists when looking at populations of states/regions/cities (zooming in for higher resolution) for any given country or even down to the street level. > Obviously every situation is different, but people seem to be pretty okay with relinquishing agency on these things and just going along with whatever local maxima their org operates in. It's not totally their fault, but they're not blameless either. I agree here, to the letter. I don't blame low level employees for maximizing their local optima. But there's two main areas (among many) that just baffles me. The first is when this is top down. When a CEO and board are hyper focused on benchmarks rather than the evaluation. Being unable to distinguish the two (benchmarks and metrics are guides, not answers). The other is when you have highly trained and educated people actively ignoring this situation. To have an over-reliance on metrics and refusing to acknowledge that metrics are proxies and considering the nuances that they were explicitly trained to look for and is what meaningfully distinguishes them from less experienced people. I've been trying to coin the term Goodhart's Hell to describe this more general phenomena because I think it is a fairly apt and concise description. The general phenomena seems prolific, but I agree that the blame has higher weight to those issuing orders. Just like a soldier is not blameless for their participation in a war crime but the ones issuing the orders are going to receive higher critique due to the imbalance of power/knowledge. Ironically I think we need to embrace the chaos a bit more. But that is rather in recognizing that ambiguity and uncertainty is inescapable rather than abandonment of any form of metric all together. I think modern society has gotten so good at measuring that we often forget that our tools are imprecise whereas previously the imprecision was so apparent that it was difficult to ignore. One could call this laziness but considering its systematic I'm not sure that's the right word.
- JohnMakin 3y ago> This often creates a situation where people need to "justify" their jobs. On DevOps teams I see this constantly. Usually the best-compensated or most senior "Ops" guy or whatever they're called at the company spends a lot of his time extinguishing fires that were either entirely of his own creation/incompetence, which makes it look like he's "doing something." You automate away the majority of the toil there and this person doesn't have a job, yet this pattern is so insanely common. There's little incentive to do it right when doing it right means management thinks you sit there all day and do nothing.
- godelski 3y agoThat's a fantastic example of what I'm trying to describe. It's kinda like thinking hours worked is directly proportional to widgets produced. There certainly are jobs and situations where this relationship holds (can't sell widgets if you aren't manning the store or can't produce turn crank widgets if the crank isn't being turned). But modern world widgets don't work that way and are more abstract. Sometimes fewer hours creates more widgets, sometimes the reverse. But widget production is now stochastic and especially in fields where creativity and brain power are required. (Using widgets for generalization -- in the economic sense--, insert any appropriate product or ask and I'll clarify)
- tstrimple 3y agoWhich is one reason DevOps teams don't make sense. DevOps is a skill developers need to have. It needs to be embedded within the development team, not some other team's responsibility who only focuses on "DevOps" work. You create the build and deployment pipelines and move on to other project work. If you give someone a role and say their job is to do "DevOps" they will HAVE to invent things to do because that's such a small part of a project and once implemented doesn't need a ton of maintenance.
- ponector 3y agoIt is not about DevOps. In every organization if you do your work good and have no fuckups - you will not be recognized and promoted. But if you are hero-firefighter - managers will love you and help with promotion. Because visibility is a key! Key to everything in the corporate life. If your work is not visible for managers - you are doing nothing.
- bsder 3y ago> Shouldn't businesses be operating like the latter? Constantly training? "But then they'll leave for somewhere else for more money." </sarcasm> Literally every company I have worked for. Meaningful training was always an uphill battle. However, good training also requires someone good and broad in your technical ladder. They may not be the most up-to-date, but they need to be able to sniff out bullshit and call it out. FAANG is no exception, either.