8 ms·

Ask HN: Do you have a dedicated QA team?

At my current company they recently got rid of our QA resources. They were contractors and reassigned to do assist in other projects. However, the company now believes developers don't need QA since they can do their own testing. Devs should write code, do all their unit, int, system testings, run their code reviews, save testing artifacts, and work with key stakeholders across the company to close requirements gaps etc. We're basically the entire sdlc minus devops. This is great for small features, but it gives me pause for mid or large features. Are we the only ones doing this?

nitwit005 5y ago
In my team's current state, there is no point in a QA team as we're already aware things are broken.
melony 5y ago
My customers are the best QA team :)
IceMetalPunk 5y ago
We have a dedicated QA team. Recently they've been "rolled into" the dev team, but they're still a group of people dedicated to testing. The actual developers should be writing their unit tests and functionally validating before opening a PR, then the other devs can validate before approving that PR, but the full E2E, automation, etc. tests should have a dedicated team, because there is a lot involved in doing that properly, and developers need time to... ya know, develop. By putting all those responsibilities on the devs, they have essentially turned a parallel development system into a serial one, which is inefficient and, frankly, dumb.
MattGaiser 5y ago
This seems to be a pretty common setup. I’ve heard opinions on both sides about whether it is a good idea. One of the biggest challenges with it for me is that a dev who is a dev is a very different personality who a dev who manages the entire SDLC. The former placed in the latter environment is going to be lost and annoyed at all the moving parts. They signed up to write code and to trust that the process would deliver good requirements. The latter in the former environment is going to be frustrated by all the process. They signed up to really own the product and realize that they really just own a block of a process. I suspect either can work if you have people suited for each. Switching people between them sounds like a recipe for attrition though.
- UglyToad 5y ago
  A term I've seen floating around for the latter is a product engineer/developer. This makes the distinction between people who view their jobs as coding to meet requirements vs. solving business problems just happening to use code.
  - MattGaiser 5y ago
    Yep. I actually interviewed for a job like that and that was the title.
olvy0 5y ago
In my company we also have dedicated QA teams. I agree with all IcePunkMetal wrote above, getting rid of them is kind of dumb. Like he wrote, they are now rolled up into the dev team instead of being part of the separate "QA division", which is better IMO. But they're still not developers. In my team, they are fully integrated into the decision making process and roadmaps, they are CCd to almost all the team's internal emails, and a format version isn't released without their authorization. Now, there are some problems with being a QA, personal stuff like having a low self-esteem since "you're not smart enough to be a dev", and enmity between devs, PMs and QA due to feelings that QA hold up the teams. As a former QA myself I'm fully aware of them. In our particular team these issues are negligible (well, at least I hope they are). I personally fully respect the QA team and stop whatever I'm doing to help them with whatever they need or ask. I see them as my first line of defense against bugs, especially the kind of UX bugs that are difficult to test for.
etblg 5y ago
We didn't use to, but now we do. I think we did a better job when we didn't have the team TBH. Engineers here used to be the ones testing out the code, verifying their changes on pre-production environments, and keep an eye on the deployed code in production to make sure it's working well. Now I get a sense that engineers do the work up to the point of where QA steps in, and throwing it over to them and moving on. The QA team is new, and so not very well versed in the product, and rely on the engineers a great deal on how to test things, which I get the feeling also ends with the QA team not going out of their way to test things the engineers didn't explicitly say to test. I think it may be playing out in a way where our quality assurance could get worse, and our engineers will end up being less well-versed in the product, and feel less responsibility for delivering quality, while also requiring another team to manage, and spreading work over different people. As a disclaimer I think we may just have a not great setup more-so than saying all QA is bad.
- ivan_gammel 5y ago
  > the QA team not going out of their way to test things the engineers didn't explicitly say to test This is the exact reason why QA team should have sufficient autonomy to design test cases on their own. If QA team is dev-driven, they will hardly bring any value, but they surely bring a lot of overhead.
bberenberg 5y ago
My feel is that this pattern leads to quantitatively better software that is qualitatively worse. There are two reasons for this: 1. QA teams come up with new ways to break software in the way humans use it. Automated tests etc confirm the software works as designed. 2. Because the org making this choice isn't stupid, they measure bug frequency and impact and find that "only X users have Y impact where Y is ranked as low" per each bug. No one bug is ever worth fixing. But, user X aren't seeing that one bug, they get hit by tens or hundreds of these on a regular basis. As a result, their perceived experience with a given product is poor and it feels buggy. This idea that you can measure bugs in isolation and deliver perceived product quality is faulty. A user experiences a product as a whole, and if you can't evaluate that as well, you will be seen as delivering a poor product.
- thelittlenag 5y ago
  The most common pattern I've seen, which dovetails with this analysis, is that new features are thus treated as more important as long as there is a path around existing bugs. Bugs gets treated as technical debt, and given as much priority.
- sanitythrowaway 5y ago
  The best QA setup I've seen was a parallel one. Devs and QA were given the same requirements doc, and independently worked out their assumptions and implementation. QA was often ready before dev work was done so sometimes devs could even run their tests locally to see if they pass. It was really remarkable. It's frustrating its not so clean usually.
  - thebean11 5y ago
    That seems like it would only work on very specific types of software teams. I've never been given a requirements doc that prescribed exact interfaces or anything even close.
    - ivan_gammel 5y ago
      True, but documented test cases can help too, because developers can use them along with requirements for better understanding of the problem.
ineptech 5y ago
Devs can add automation tests to a suite just fine. What you lose when you lose dedicated QA is: who improves the automation suite? Who checks out new testing tools and POCs them? Who reads QA blogs and goes to QA conferences and generally considers it their profession to find ways to test more effectively?
- runlevel1 5y ago
  This is my take as well. I've worked with various types of QA teams over the years (all in SaaS). Based on my experience, from MOST effective to LEAST effective: 1. QA team works on test harnesses, automation, and works alongside developers to improve how we do testing. This was by far the most effective. 2. No QA team. Developers own every aspect of quality. This worked, but was not as effective as #1. 3. All releases are thrown over the wall to QA who stamp them and do the deploys. This shifted the responsibility for quality to people who weren't as familiar with the code and weren't equipped to fix it. Outages abound. It also slowed releases to a crawl. 4. QAs are non-technical. They mostly write and manage test cases. It's up to the developers to implement tests, but some tests are done manually by QA. I never saw any benefits from tracking test cases outside of the code that validates them. Properly prioritized tickets were fine for defect follow-up. Manual testing might be ok if you're a new startup, but it quickly becomes untenable as you grow.
  - honey-badger 5y ago
    Your comment is super helpful! A few months back, we went from scenario 2 to scenario 1, and I can attest to it being substantially better.
  - cranky908canuck 5y ago
    A case that also arises is: - QA is from a problem domain background - Dev is from a CS/math background In that case, you can have: - QA/Dev riff off each other, and collaborate - The few individuals in the org that have both become the co-ordinators
neriymus 5y ago
Good QAs are so important. There are so many bugs that would have never been found by us. In the past 3 weeks, both our QAs have been off, and the inbox list of problems have grounded to almost zero! Not because we are writing more-than-usual-amazing code, but because no one managed to find them.
criddell 5y ago
We do have a dedicated QA team and it's absolutely essential because developers (like me) aren't necessarily domain experts.
- honey-badger 5y ago
  Does the QA team comprise domain experts? What is your domain?
  - criddell 5y ago
    Yes. They are mechanical engineers.
throwawayninja 5y ago
The QA team brings something significant that developers (myself) will never have: their machines closely represent the customers machines. They use a well-polished methodology to push the models we ship to their limits and provide relatively detailed information about how to reproduce failures. I can't count the number of times I've tested something 5 times, convinced myself that the code handled tons of error cases, shipped it, and in 10 minutes a customer types "five" into a field expecting an integer which causes the next step to use "0" for processing.
- jbjbjbjb 5y ago
  Well their methodology isn’t magic. It’s a skill a developer can learn.
- aynawn 5y ago
  why don't the dev machines also closely represent customer set ups?
lalaithion 5y ago
We don't have a QA team; we have an internal team that uses our product to do their work. It's part QA, part product development, part marketing; they help us prioritize feature work, they write blog posts about the cool stuff they can do with our product, and when bugs surface, they find them, because it's all part of their workflow. This doesn't work for all companies; if you're building a system for children to learn how to type, you can't really hire a bunch of children to learn how to type... but I think for most companies, this can and does work.
- Veuxdo 5y ago
  This sounds like a great setup. Can I ask what kind of product it is?
- martin_drapeau 5y ago
  This is an excellent setup. You're drinking your own champaign!
yodon 5y ago
Having been through the decision process in the past for whether to go when centralized QA, dedicated QA on teams, or automated testing as an engineering responsibility, I came away with the personal opinion that the quality of automated test frameworks has improved so drastically in the past five years or so that we really are in a different place than we were when concept of dedicated QA teams was invented long ago. Large scale software engineering is a human coordination problem (see Conway's Law and the Mythical Man Month), and the time required for engineers to write tests after already understanding the requirements is small compared to the time required for the organization to have someone else try to separately understand the same information and separately write the test. That didn't use to be the case, but today it is. I get not wanting to take time to write tests, just as I get not wanting to take time to document your code, but that's not a good enough reason for introducing the inefficiencies of having someone else try to do something that is best described as your own job (again, the economics used to be different because writing tests used to be slow and inefficient, but that is changing/has changed). I will say however that I think you do generally still need a team (or individual at smaller shops) dedicated to tooling for automated testing to make the process of writing tests efficient for all those who are writing tests.
- ccalloway 5y ago
  The problem with this is that a person who makes a mistake when doing something is much more likely than someone else to make a mistake when evaluating that same piece of work.
  - Veuxdo 5y ago
    Exactly. The problem isn't "writing tests", is writing test cases.
  - yodon 5y ago
    The really big wins happen when you get a framework in place that gives you a formal language like Gherkin for expressing what needs to be tested. That's a win because it starts giving product a way to specify requirements that are both clearer and more explicitly test-like. That starts integrating test design into the spec process as opposed to having test design happen during the implementation process.
ivan_gammel 5y ago
I always setup a dedicated QA team that has right of veto for releases. Developers indeed can do some testing, but quality assurance is more than that. It starts at the problem definition stage, when QA engineers ensure there’s no misunderstanding of the requirements, they make sense and they answer all important questions. It ends long after release, because production support is also part of QA. QA engineers do their part of testing, just like developers do, but they also coordinate delivery and support from quality perspective. Their feedback is often valuable to product managers and designers, they can help a lot your customer support team.
VincentEvans 5y ago
We have a QA team of 4 that supports x4 as many engineers. If they weren’t there - engineers would spend at least 1/4th of their time performing QA duties with likely worse results and at significantly higher cost to the company, since their compensation is higher. So it makes sense to do that only as a cost saving measure in hard times - you are picking losing which employees will be less detrimental in the long term, even if you expect that those that remain will be less productive.
Minor49er 5y ago
I don't at my current job, but my last job did. The benefit of having them is that they maintain domain knowledge about how something works, will ask pertinent questions about new functionality, and are able to dedicate a lot of time to trying to break a component. The drawback of having them is that if they are mismanaged or get backed up, it can take a long time before a feature even gets looked at (my last job was so bad with this that I would often get asked about features that I submitted 4-6 weeks later). The good thing about not having a QA team is that QA becomes a much more active part of the process where something must be tested by the developers before it's completed. You also don't have a single bottleneck if the QA team is overwhelmed. The drawbacks are that developers aren't always very thorough and that their testing time takes away from their development time.
swiftcoder 5y ago
I've always had dedicated QA teams when working with physical devices (android smartphones and VR headsets, specifically). I think a dedicated QA team is absolutely non-negotiable for hardware development, because there is no test harness that can accurately simulate the range of stupid shit that happens with hardware: "smartphone doesn't boot reliably when the ambient temperature is below 40 degrees fahrenheit" (temperature lowers the battery voltage + high power usage during startup = voltage drops below FCC-mandated minimums for the antennas -> automated SoC shutdown) "bluetooth controllers won't pair reliably in building 17" (too much interference to pair if there are more than ~100 other bluetooth devices nearby) The combinatorial explosion of external factors in this kind of bug is insane - you just won't have a good time without dedicated folks whose focus is chasing these down (and I'll take a good QA over a good engineer for this sort of investigation, any day).
justin_oaks 5y ago
QA teams primary benefit shouldn't be running automated tests that the developers could do. It should be to do exploratory testing and trying to break things. There should be a adversarial, yet respectful, relationship between dev and QA where dev tries to get their software released and QA is finding reasons why it shouldn't be released. If there's only developers who test their own code then they're not incentivized for finding the ways their code breaks. They'll downplay the faults or even be blind to them because they're too close to the implementation and aren't "thinking like a user". It seems odd to me when those in charge ignore the problems of disbanding QA. But I assume it's because the management may not be incentivized to promote quality software. Instead the incentives are to crank out features. That said, maybe the market for the software doesn't punish the software makers for having buggy software. In such a case, ignoring QA work is a rational decision.
- timwis 5y ago
  Another solution is to incentivise the developers by lowering the barrier between them and their users, and changing the success measures from outputs to outcomes.
- Clubber 5y ago
  >It seems odd to me when those in charge ignore the problems of disbanding QA. But I assume it's because the management may not be incentivized to promote quality software. Instead the incentives are to crank out features. >That said, maybe the market for the software doesn't punish the software makers for having buggy software. In such a case, ignoring QA work is a rational decision. I think there are a lot of things that promote cost cutting in software quality: 1. Consolidation: There's usually 1 big player that everyone uses. When that's the case there is nowhere else to go. 2. Lock in: Even if there was somewhere to go, once you've committed your business to a software vendor, it's really hard (time + expense) to switch. 3. Executives get bonuses based on income: The easiest way to juice the numbers is to cut employees and departments. The results of cutting QA won't get noticed by the market for a few years. 4. Over reliance on automated unit testing by management: I think unit testing is great, but it's really hard to utilize it on user XP. It was never meant to be a replacement to manual QA testing, but an augmentation. Of course refer to #3. 5. Reduction in autonomous middle management: Executives say cut this and cut that, middle management who actually know the process have little to no say, or won't say because they don't want to lose their jobs.
jasonlotito 5y ago
We have a dedicated QA team. They work side-by-side with devs, and we rely on them. They are absolute lifesavers and let us focus on the dev side of things. Their knowledge of the products, all the various ways it interacts, and their dedication to testing are wonderful. The quality of the product would suffer if they didn't exist, hands down.
- cranky908canuck 5y ago
  I have heard a dev say that 'QA is my conscience'.
gvpmahesh 5y ago
We do have QA team(2 members, 30+ engineers) but they are not the gate keepers for releases. Developers write all sorts of tests(TDD + pair programming helps), and when we need an extra pair of eyes, we ask the product team to have a look at it. But that's about it. QA's are involved more in the creative side of things post releases or have an independent path to see where the things could go wrong, more like doing monkey testing.
jtwigg 5y ago
yes, the customer.
utkarshdix02 5y ago
Problem I faced in past with a dedicated QA team is, the development process is often faster than QA. Which puts QA in the backseat with each new release. Also, human coordination becomes a problem as there will be a small configuration of which QAs team might not be aware of e.g. A/B configurations, hotfixes, etc. On the other hand, with devs writing tests is there is learning curve and most devs have not written tests or think of it as a chore. They also don't have end-user perspectives and have biases towards things working. Ideally in my opinion testing should be shared responsibility, where devs test feature they implemented and the QA team add checks from an end-user perspective. P.S.- I'm co-founder of startup https://crusher.dev https://crusher.dev making QA easy by using a low-code approach.
- dreadlordbone 5y ago
  Your product looks great but your website has a decent amount of grammatical errors, which I think might scare off potential customers. I recommend having an English first person proof read through your site.
  - utkarshdix02 5y ago
    Thanks for the feedback. We built it a while ago and never iterated over it. We will update it soon.
larrik 5y ago
In my experience, devs make terrible QAs, since they know what the code should do and never think of things like "pasting a novel into a textfield" and the like. They also tend to be much more expensive as well. Combine the two, and it's just not at all an efficient use of a company's budget.
yahn00 5y ago
As apps become easier to maintain and keep online, the extreme separation in roles is only important at companies with sufficient scale in some metric to need the specialization. I can deploy a prod EKS/CI/CD setup with Terraform scripts, and be deploying code to the public in a morning now. Like other service jobs, IT is going to contract sooner than later. NoCode infrastructure is where k8s was in 2014. IT workers thinking in IT patterns as usual is a huge blocker to progress. Elder workers domain knowledge is stored in DBs and source control everywhere. It’s trivial to parse and transform into another syntax. Congrats; you trained your Copilot replacements without even knowing it.
honey-badger 5y ago
QA Engineer here. We don't have a centralized QA department, but we do have a dedicated engineer who is responsible for QA that is integrated within the development team. Before I joined my team, it didn't have a dedicated QA member. The quality of the software was fine, but there were other compromises. The team didn't have a good test strategy - every developer made adhoc decisions on how their code was tested. Our E2E tests ran slowly and had tonnes of duplicates, since nobody had gone through the entire list and cleaned it up. A dedicated QA member has the time and the responsibility to solve problems with quality that most devs are likely to treat as secondary to developing features. And the improvements are substantial - our E2E tests now run 18x (!) faster. For the most part, I see my role as being an enabler - I help devs build quality into their work and hold them accountable to it. I don't see it as an adverserial division of responsibility, but more as a collaborative effort, with the dev and QA coming from a different focal point. The problem with strict division of labour and adverserial QA is that it leads to 1) Organizational / team silos 2) Local optimization (software development and QA is intricately interwoven, so it doesn't lend itself well to this strict division) 3) Information overhead and loss A good QA engineer, IMO, requires better coordination and communication skills than a developer because they liaise with multiple developers and inspire them to make quality a priority in their work. Further, QA is involved from the definition of a feature (in sprint refinements) right up to when code is deployed and runs well in production. Therefore, at certain points, the boundaries between QA and the Dev team or QA and the PO blur and disappear entirely. I found this to be a great read that summarizes my thoughts on the topic: https://www.thoughtworks.com/en-de/insights/blog/qa-dead https://www.thoughtworks.com/en-de/insights/blog/qa-dead
- martin_drapeau 5y ago
  I really like this approach. An engineer who's responsiblity and focus is QA.
- cranky908canuck 5y ago
  If QA and developers come from different tracks (eg, devs from CS, QA from problem domain [[ eg. BSc/CS vs BComm/Acct]] ), then QA looks at the things that "weren't in the story requirements" but matter for the product. If it's a collective "we learned from this", that's good. If it's seriously and organizationally adversarial, that's bad.
twblalock 5y ago
At this point I only see value in manual QA testing for front-end apps. It's now possible to test pretty much anything via automation, although front-end testing is still harder than most and there are some intangible UX issues that can only be experienced by human users. But for testing APIs? It is normally possible to cover 100% of what is needed via a combination of unit tests and integration tests. This is especially true now that tooling exists that can bring up ephemeral test environments with dependencies such as databases etc.
aeturnum 5y ago
To me the benefit of a QA team comes from synthesis. If you are providing one service to a few groups you don't get that much out of having a dedicated team. However, once you start having different people using your product in different ways it helps enormously to have a teams whose job is to be familiar with customer pain points and setups and warn you that your change is going to trigger problems.
sidcool 5y ago
Yes. They test by considering the app as a black box. They understand business.
KingOfCoders 5y ago
The biggest benefits a QA team can give is unit/e2e tests code reviews and consult product managers. The worst value they can provide with manuell testing (exc pt exploratory testing)
martin_drapeau 5y ago
Are your product managers/owners involved in testing new features? They can play an important role here. Are you a SaaS practicing continuous delivery? Can customers report bugs rapidly? Can you fix them rapidly? If you can reduce the turn around time of fixing a bug to less than 1 hour, you can use your customers to QA. If so, you should not have a QA department. They only slow things down. Of course, this depends on the type of business you are in.
shmde 5y ago
Was a QA engineer. Yes the company I was working with had a full blown massive 100-150 QA engineers working full time. Manual Testers, Automated Testers, Managers managing them, Test Case documenters. The QA and the dev team worked very very closely.
falcolas 5y ago
No. I wish we did. Well, I guess we do, but we have 1 QA who is shared between 5 logical teams, over 40 developers. AKA, no, we don't.
salawat 5y ago
Built a QA department from scratch. Places are getting tight on the purse strings in general, and you can tell when software comes from somewhere that divested themselves from QA as a business concern. Ratios of 4+ to 1 dev:QA abound, and the depth to which QA departments can even do their primary business integrative functions suffers greatly for it. The most irritating aspect I notice is the tendency for having to do the same project multiple times because requirements, and user/devteam communication isn't what it should be. QA's increasingly are facing an uphill battle against information overload, that management and typical textbook PM processes are very ill-equipped to detect; there's also the fact that your "contract" QA departmemt, does not have incentive alignment. The most horrible devs can be endured by a QA group that cycles out QA's on a 2-3 month basis. The in-house QA group that logs in to the same steaming pile of low-quality crap will soon either find themselves motivated to trim waste or otherwise increase the efficiency and processes where they are. I can only ever get FTE QA's to even deeply learn the business... I can't do that generally with contractors. So if places are doing away with QA... I wouldn't be surprised. And I don't think the upswing in software origin'd suffering from my perspective is just a coincidence either.
HyperRational 5y ago
harel 5y ago
Maintaining test automation is equivalent to maintaining the code base. A dedicated qa team should be doing that as well as manual testing, infra tests, etc. We had a dedicated qa in a project, and when he was taken away twat automation pretty much died. Nobody maintained it.
yabones 5y ago
I'm seeing lots of good reasons for keeping a full QA team around, want to add one more: They're basically the only people that know everything about the product. More than product managers who generally specialize in the Foo feature, more than the engineers, certainly more than sales. When a seriously complex support issue comes in, they are always the most valuable people to take a look. To me, the value of QA is more about product knowledge and expertise than just "testing things".
hogrider 5y ago
I've been doing QA my entire couple of years career and I think most companies can do just fine with devs testing their code. My job is just too alienating to be done effectively IMO.
APhoenixRises 5y ago
In an ideal world, devs would always have the time to test their own code. The unit and integration bits aren't the pain point, but once you reach the API or UI, people's knowledge varies widely. That's where a good QE org, either embedded or a mix of centralized and embedded, can be very helpful. In my past companies we've pushed to be champions or domain experts in test automation who's main purpose is to make testing for everyone else suck less. Sure, you will definitely take the brunt of test automation tasks, but some of those tasks should be targeted towards removing impediments for testing. In one situation I ran into a team who was very frustrated with their automation because it was so slow. After a bit of investigation, I found out they were setting up all of their test data through database migrations, which were 10+ minutes slow. I asked why they weren't using the API to setup test data, to which they replied that none of them had experience with the API. After hacking away it in my spare time for a few weeks, I had a perfectly functional API client, and the tests were converted over time out of using the migrations. I saw this the other day, and while it might sound cheesy, but someone said that QA to them was "quality acceleration", which actually fits fairly well. I have never worked at a company who had enough testers to go around. Keeping a smaller group of centralized testers to handle the really tough technical challenges, and a larger group of embedded testers who evangelize on testing practices was the most successful approach. In this case, I wouldn't say you absolutely need a full quality team, but having some folks whose primary focus is testing and test automation to help skill up your dev teams sounds reasonable.
- honey-badger 5y ago
  Quality Acceleration is brilliant! I love it. >In this case, I wouldn't say you absolutely need a full quality team, but having some folks whose primary focus is testing and test automation to help skill up your dev teams sounds reasonable. This is what we have at our current team, as I have described in a comment above, and it has worked very well for us.
jbjbjbjb 5y ago
You’re not the only ones doing it. The idea is developers do more testing, they think about it upfront and they have assistance from a QA lead who leads and coaches the quality culture across the development team. That QA leads embeds the skills in the dev team and oversees the quality. The QA lead can also be freed up to look at quality from other perspectives like is the product actually delivering value to the customer overall. The devs are already working across the SDLC so have the knowledge and the skin in the game to do a good job. It also works more efficiently with less handover which is better for frequent releases vs a time when releases were once every two years.
osigurdson 5y ago
We have a dedicated QA team. Generally developers write unit tests while QA writes more integration level tests (mostly back-end focused). They do some exploratory manual testing while most of the repetitive work is managed with (brittle) UI automation tests. It is a lot of work to maintain all of the tests that we have. However, we have an incredibly good track record of releasing on time and critical bugs are rarely introduced.
michael_j_x 5y ago
We have 2 dedicated QAs per team of 6 devs. They are not only tasked with finding dev bugs, but also business "bugs". Things that the POs did not consider when writing the stories, requirements that the devs did not understand correctly erc. This has allowed us to move much faster and deploy with confidence, as the QAs also do the UAT part during their QA
throwawayHN378 5y ago
I work for a 10,000 employee organization ( not technically a software company but definitely an who’s customers are engineers and has a strong engineering culture ). We don’t have dedicated QA people by name, but we do have sever BAs who are essentially dedicated QA. Commits need to have test cases, then they get code reviewed, merged into erotic branch, epic branch merged into one of the dev. The. Our technical product managers generally test it ( at my org our technical product managers are highly technical and write code themselves ). Then generally we merge into test and eventually BAs test it. That’s generally how it works where I’m at currently. I’ve only ever worked for one org with a dedicated QA team
nouveaux 5y ago
In general, the people who write and/or design the code are the ones who knows how to best test the code. These are the people who have thought most deeply about the code that is written, not only in the code itself but also how it fits in the larger context. Good QA is hard to find, and in general are underpaid. In my opinion, good QA is like good security researchers. They know how to break the system. These people are invaluable. As a developer, you should want to write tests and encourage people on your team to love it. Good tests let you sleep at night and enjoy your weekends. If you perception is that "writing these tests are a waste of my time", I think that's actually not in your best interest.
coward123 5y ago
Welcome to the cycle... don't worry, they will make you do devops soon enough too... then, maybe after they've burned through enough people or have lost enough money through broken software and ops, they will start hiring QA and ops again. Or, they will send it all offshore because the onshore talent isn't any good anyway - they keep leaving and costing us money with their broken software! Rinse, repeat.
deleted 5y ago
[deleted]
PeterWhittaker 5y ago
For our hardware product, no; for our software product, yes, from time to time. The hardware product is a network security device that essentially works as a black box, with limited inputs and outputs. It requires so much domain expertise that the small group of us that work on it take an incremental approach both to feature development and to bug fixes, with a fair amount of design review, code review, discussion, prototyping and PoC, etc., before moving to actual testing. Most recently, I've tended to do the bulk of the development and one of my colleagues the bulk of the testing. When we find errors, there is a lot of screen sharing to review code and logs, determine the best way forward, etc. Our build process generates ISOs with several layers of security, from completely locked down release instances to a permissive mode with a development account for the privileged access necessary to run onboard diagnostics. The caveat is that one of our main customers is a security agency with complementary expertise, and they tend to beat the bejeesus out of it. Overall, we take a similar approach to our software product (design and code reviews, much screen sharing, discussion between development, marketing, and senior management on what it should do and what it should look like, etc.), but at some point it gets turned over to QA. I wrote above that our QA resource was full time from time to time, because when they are not doing QA on the product, they are doing market research and analysis, developing marketing strategy, etc. From on QA PoV, one major the difference between the two products is audience: The hardware product is used by domain experts to do very specific network segmentation and data transfer; the software product is intended for general use by non-IT corporate users - it's a web application for integrated risk management and compliance reviews. We consider ourselves competent to fully test the hardware product we've updated, because its functionality is essentially binary: either it works or it doesn't, there aren't a lot of variations in use case, and we are best positioned to determine how it should work and whether it is working. We don't consider ourselves competent to fully test the software product, because we are a) too close to it, b) domain experts, c) not typical users, and because d) we have a lot of experience with a lot of different software. Our users are often people who mostly use browsers, and occasionally office products. They may be domain experts, but their domains are far from IT (especially the physical security people). Our QA person is excellent at getting into their shoes, so to speak, imagining how they might approach things, and findings both bugs and ways that our designs and GUIs and UXs lack and don't help users get to where they need. Having someone with that external perspective is invaluable. Having them be full-time when in QA mode helps us move quickly when we need to - they are never the bottleneck, and anything released has been thoroughly beaten up.

nitwit005 5y ago

In my team's current state, there is no point in a QA team as we're already aware things are broken.

melony 5y ago

My customers are the best QA team :)

IceMetalPunk 5y ago

We have a dedicated QA team. Recently they've been "rolled into" the dev team, but they're still a group of people dedicated to testing. The actual developers should be writing their unit tests and functionally validating before opening a PR, then the other devs can validate before approving that PR, but the full E2E, automation, etc. tests should have a dedicated team, because there is a lot involved in doing that properly, and developers need time to... ya know, develop. By putting all those responsibilities on the devs, they have essentially turned a parallel development system into a serial one, which is inefficient and, frankly, dumb.

MattGaiser 5y ago

This seems to be a pretty common setup. I’ve heard opinions on both sides about whether it is a good idea. One of the biggest challenges with it for me is that a dev who is a dev is a very different personality who a dev who manages the entire SDLC. The former placed in the latter environment is going to be lost and annoyed at all the moving parts. They signed up to write code and to trust that the process would deliver good requirements. The latter in the former environment is going to be frustrated by all the process. They signed up to really own the product and realize that they really just own a block of a process. I suspect either can work if you have people suited for each. Switching people between them sounds like a recipe for attrition though.

UglyToad 5y ago
A term I've seen floating around for the latter is a product engineer/developer. This makes the distinction between people who view their jobs as coding to meet requirements vs. solving business problems just happening to use code.
- MattGaiser 5y ago
  Yep. I actually interviewed for a job like that and that was the title.

olvy0 5y ago

In my company we also have dedicated QA teams. I agree with all IcePunkMetal wrote above, getting rid of them is kind of dumb. Like he wrote, they are now rolled up into the dev team instead of being part of the separate "QA division", which is better IMO. But they're still not developers. In my team, they are fully integrated into the decision making process and roadmaps, they are CCd to almost all the team's internal emails, and a format version isn't released without their authorization. Now, there are some problems with being a QA, personal stuff like having a low self-esteem since "you're not smart enough to be a dev", and enmity between devs, PMs and QA due to feelings that QA hold up the teams. As a former QA myself I'm fully aware of them. In our particular team these issues are negligible (well, at least I hope they are). I personally fully respect the QA team and stop whatever I'm doing to help them with whatever they need or ask. I see them as my first line of defense against bugs, especially the kind of UX bugs that are difficult to test for.

etblg 5y ago

We didn't use to, but now we do. I think we did a better job when we didn't have the team TBH. Engineers here used to be the ones testing out the code, verifying their changes on pre-production environments, and keep an eye on the deployed code in production to make sure it's working well. Now I get a sense that engineers do the work up to the point of where QA steps in, and throwing it over to them and moving on. The QA team is new, and so not very well versed in the product, and rely on the engineers a great deal on how to test things, which I get the feeling also ends with the QA team not going out of their way to test things the engineers didn't explicitly say to test. I think it may be playing out in a way where our quality assurance could get worse, and our engineers will end up being less well-versed in the product, and feel less responsibility for delivering quality, while also requiring another team to manage, and spreading work over different people. As a disclaimer I think we may just have a not great setup more-so than saying all QA is bad.

ivan_gammel 5y ago
> the QA team not going out of their way to test things the engineers didn't explicitly say to test This is the exact reason why QA team should have sufficient autonomy to design test cases on their own. If QA team is dev-driven, they will hardly bring any value, but they surely bring a lot of overhead.

bberenberg 5y ago

My feel is that this pattern leads to quantitatively better software that is qualitatively worse. There are two reasons for this: 1. QA teams come up with new ways to break software in the way humans use it. Automated tests etc confirm the software works as designed. 2. Because the org making this choice isn't stupid, they measure bug frequency and impact and find that "only X users have Y impact where Y is ranked as low" per each bug. No one bug is ever worth fixing. But, user X aren't seeing that one bug, they get hit by tens or hundreds of these on a regular basis. As a result, their perceived experience with a given product is poor and it feels buggy. This idea that you can measure bugs in isolation and deliver perceived product quality is faulty. A user experiences a product as a whole, and if you can't evaluate that as well, you will be seen as delivering a poor product.

thelittlenag 5y ago
The most common pattern I've seen, which dovetails with this analysis, is that new features are thus treated as more important as long as there is a path around existing bugs. Bugs gets treated as technical debt, and given as much priority.
sanitythrowaway 5y ago
The best QA setup I've seen was a parallel one. Devs and QA were given the same requirements doc, and independently worked out their assumptions and implementation. QA was often ready before dev work was done so sometimes devs could even run their tests locally to see if they pass. It was really remarkable. It's frustrating its not so clean usually.
- thebean11 5y ago
  That seems like it would only work on very specific types of software teams. I've never been given a requirements doc that prescribed exact interfaces or anything even close.
  - ivan_gammel 5y ago
    True, but documented test cases can help too, because developers can use them along with requirements for better understanding of the problem.

ineptech 5y ago

Devs can add automation tests to a suite just fine. What you lose when you lose dedicated QA is: who improves the automation suite? Who checks out new testing tools and POCs them? Who reads QA blogs and goes to QA conferences and generally considers it their profession to find ways to test more effectively?

runlevel1 5y ago
This is my take as well. I've worked with various types of QA teams over the years (all in SaaS). Based on my experience, from MOST effective to LEAST effective: 1. QA team works on test harnesses, automation, and works alongside developers to improve how we do testing. This was by far the most effective. 2. No QA team. Developers own every aspect of quality. This worked, but was not as effective as #1. 3. All releases are thrown over the wall to QA who stamp them and do the deploys. This shifted the responsibility for quality to people who weren't as familiar with the code and weren't equipped to fix it. Outages abound. It also slowed releases to a crawl. 4. QAs are non-technical. They mostly write and manage test cases. It's up to the developers to implement tests, but some tests are done manually by QA. I never saw any benefits from tracking test cases outside of the code that validates them. Properly prioritized tickets were fine for defect follow-up. Manual testing might be ok if you're a new startup, but it quickly becomes untenable as you grow.
- honey-badger 5y ago
  Your comment is super helpful! A few months back, we went from scenario 2 to scenario 1, and I can attest to it being substantially better.
- cranky908canuck 5y ago
  A case that also arises is: - QA is from a problem domain background - Dev is from a CS/math background In that case, you can have: - QA/Dev riff off each other, and collaborate - The few individuals in the org that have both become the co-ordinators

neriymus 5y ago

Good QAs are so important. There are so many bugs that would have never been found by us. In the past 3 weeks, both our QAs have been off, and the inbox list of problems have grounded to almost zero! Not because we are writing more-than-usual-amazing code, but because no one managed to find them.

criddell 5y ago

We do have a dedicated QA team and it's absolutely essential because developers (like me) aren't necessarily domain experts.

honey-badger 5y ago
Does the QA team comprise domain experts? What is your domain?
- criddell 5y ago
  Yes. They are mechanical engineers.

throwawayninja 5y ago

The QA team brings something significant that developers (myself) will never have: their machines closely represent the customers machines. They use a well-polished methodology to push the models we ship to their limits and provide relatively detailed information about how to reproduce failures. I can't count the number of times I've tested something 5 times, convinced myself that the code handled tons of error cases, shipped it, and in 10 minutes a customer types "five" into a field expecting an integer which causes the next step to use "0" for processing.

jbjbjbjb 5y ago
Well their methodology isn’t magic. It’s a skill a developer can learn.
aynawn 5y ago
why don't the dev machines also closely represent customer set ups?

lalaithion 5y ago

We don't have a QA team; we have an internal team that uses our product to do their work. It's part QA, part product development, part marketing; they help us prioritize feature work, they write blog posts about the cool stuff they can do with our product, and when bugs surface, they find them, because it's all part of their workflow. This doesn't work for all companies; if you're building a system for children to learn how to type, you can't really hire a bunch of children to learn how to type... but I think for most companies, this can and does work.

Veuxdo 5y ago
This sounds like a great setup. Can I ask what kind of product it is?
martin_drapeau 5y ago
This is an excellent setup. You're drinking your own champaign!

yodon 5y ago

Having been through the decision process in the past for whether to go when centralized QA, dedicated QA on teams, or automated testing as an engineering responsibility, I came away with the personal opinion that the quality of automated test frameworks has improved so drastically in the past five years or so that we really are in a different place than we were when concept of dedicated QA teams was invented long ago. Large scale software engineering is a human coordination problem (see Conway's Law and the Mythical Man Month), and the time required for engineers to write tests after already understanding the requirements is small compared to the time required for the organization to have someone else try to separately understand the same information and separately write the test. That didn't use to be the case, but today it is. I get not wanting to take time to write tests, just as I get not wanting to take time to document your code, but that's not a good enough reason for introducing the inefficiencies of having someone else try to do something that is best described as your own job (again, the economics used to be different because writing tests used to be slow and inefficient, but that is changing/has changed). I will say however that I think you do generally still need a team (or individual at smaller shops) dedicated to tooling for automated testing to make the process of writing tests efficient for all those who are writing tests.

ccalloway 5y ago
The problem with this is that a person who makes a mistake when doing something is much more likely than someone else to make a mistake when evaluating that same piece of work.
- Veuxdo 5y ago
  Exactly. The problem isn't "writing tests", is writing test cases.
- yodon 5y ago
  The really big wins happen when you get a framework in place that gives you a formal language like Gherkin for expressing what needs to be tested. That's a win because it starts giving product a way to specify requirements that are both clearer and more explicitly test-like. That starts integrating test design into the spec process as opposed to having test design happen during the implementation process.

ivan_gammel 5y ago

I always setup a dedicated QA team that has right of veto for releases. Developers indeed can do some testing, but quality assurance is more than that. It starts at the problem definition stage, when QA engineers ensure there’s no misunderstanding of the requirements, they make sense and they answer all important questions. It ends long after release, because production support is also part of QA. QA engineers do their part of testing, just like developers do, but they also coordinate delivery and support from quality perspective. Their feedback is often valuable to product managers and designers, they can help a lot your customer support team.

VincentEvans 5y ago

We have a QA team of 4 that supports x4 as many engineers. If they weren’t there - engineers would spend at least 1/4th of their time performing QA duties with likely worse results and at significantly higher cost to the company, since their compensation is higher. So it makes sense to do that only as a cost saving measure in hard times - you are picking losing which employees will be less detrimental in the long term, even if you expect that those that remain will be less productive.

Minor49er 5y ago

I don't at my current job, but my last job did. The benefit of having them is that they maintain domain knowledge about how something works, will ask pertinent questions about new functionality, and are able to dedicate a lot of time to trying to break a component. The drawback of having them is that if they are mismanaged or get backed up, it can take a long time before a feature even gets looked at (my last job was so bad with this that I would often get asked about features that I submitted 4-6 weeks later). The good thing about not having a QA team is that QA becomes a much more active part of the process where something must be tested by the developers before it's completed. You also don't have a single bottleneck if the QA team is overwhelmed. The drawbacks are that developers aren't always very thorough and that their testing time takes away from their development time.

swiftcoder 5y ago

I've always had dedicated QA teams when working with physical devices (android smartphones and VR headsets, specifically). I think a dedicated QA team is absolutely non-negotiable for hardware development, because there is no test harness that can accurately simulate the range of stupid shit that happens with hardware: "smartphone doesn't boot reliably when the ambient temperature is below 40 degrees fahrenheit" (temperature lowers the battery voltage + high power usage during startup = voltage drops below FCC-mandated minimums for the antennas -> automated SoC shutdown) "bluetooth controllers won't pair reliably in building 17" (too much interference to pair if there are more than ~100 other bluetooth devices nearby) The combinatorial explosion of external factors in this kind of bug is insane - you just won't have a good time without dedicated folks whose focus is chasing these down (and I'll take a good QA over a good engineer for this sort of investigation, any day).

justin_oaks 5y ago

QA teams primary benefit shouldn't be running automated tests that the developers could do. It should be to do exploratory testing and trying to break things. There should be a adversarial, yet respectful, relationship between dev and QA where dev tries to get their software released and QA is finding reasons why it shouldn't be released. If there's only developers who test their own code then they're not incentivized for finding the ways their code breaks. They'll downplay the faults or even be blind to them because they're too close to the implementation and aren't "thinking like a user". It seems odd to me when those in charge ignore the problems of disbanding QA. But I assume it's because the management may not be incentivized to promote quality software. Instead the incentives are to crank out features. That said, maybe the market for the software doesn't punish the software makers for having buggy software. In such a case, ignoring QA work is a rational decision.

timwis 5y ago
Another solution is to incentivise the developers by lowering the barrier between them and their users, and changing the success measures from outputs to outcomes.
Clubber 5y ago
>It seems odd to me when those in charge ignore the problems of disbanding QA. But I assume it's because the management may not be incentivized to promote quality software. Instead the incentives are to crank out features. >That said, maybe the market for the software doesn't punish the software makers for having buggy software. In such a case, ignoring QA work is a rational decision. I think there are a lot of things that promote cost cutting in software quality: 1. Consolidation: There's usually 1 big player that everyone uses. When that's the case there is nowhere else to go. 2. Lock in: Even if there was somewhere to go, once you've committed your business to a software vendor, it's really hard (time + expense) to switch. 3. Executives get bonuses based on income: The easiest way to juice the numbers is to cut employees and departments. The results of cutting QA won't get noticed by the market for a few years. 4. Over reliance on automated unit testing by management: I think unit testing is great, but it's really hard to utilize it on user XP. It was never meant to be a replacement to manual QA testing, but an augmentation. Of course refer to #3. 5. Reduction in autonomous middle management: Executives say cut this and cut that, middle management who actually know the process have little to no say, or won't say because they don't want to lose their jobs.

jasonlotito 5y ago

We have a dedicated QA team. They work side-by-side with devs, and we rely on them. They are absolute lifesavers and let us focus on the dev side of things. Their knowledge of the products, all the various ways it interacts, and their dedication to testing are wonderful. The quality of the product would suffer if they didn't exist, hands down.

cranky908canuck 5y ago
I have heard a dev say that 'QA is my conscience'.

gvpmahesh 5y ago

We do have QA team(2 members, 30+ engineers) but they are not the gate keepers for releases. Developers write all sorts of tests(TDD + pair programming helps), and when we need an extra pair of eyes, we ask the product team to have a look at it. But that's about it. QA's are involved more in the creative side of things post releases or have an independent path to see where the things could go wrong, more like doing monkey testing.

jtwigg 5y ago

yes, the customer.

utkarshdix02 5y ago

Problem I faced in past with a dedicated QA team is, the development process is often faster than QA. Which puts QA in the backseat with each new release. Also, human coordination becomes a problem as there will be a small configuration of which QAs team might not be aware of e.g. A/B configurations, hotfixes, etc. On the other hand, with devs writing tests is there is learning curve and most devs have not written tests or think of it as a chore. They also don't have end-user perspectives and have biases towards things working. Ideally in my opinion testing should be shared responsibility, where devs test feature they implemented and the QA team add checks from an end-user perspective. P.S.- I'm co-founder of startup https://crusher.dev https://crusher.dev making QA easy by using a low-code approach.

dreadlordbone 5y ago
Your product looks great but your website has a decent amount of grammatical errors, which I think might scare off potential customers. I recommend having an English first person proof read through your site.
- utkarshdix02 5y ago
  Thanks for the feedback. We built it a while ago and never iterated over it. We will update it soon.

larrik 5y ago

In my experience, devs make terrible QAs, since they know what the code should do and never think of things like "pasting a novel into a textfield" and the like. They also tend to be much more expensive as well. Combine the two, and it's just not at all an efficient use of a company's budget.

yahn00 5y ago

As apps become easier to maintain and keep online, the extreme separation in roles is only important at companies with sufficient scale in some metric to need the specialization. I can deploy a prod EKS/CI/CD setup with Terraform scripts, and be deploying code to the public in a morning now. Like other service jobs, IT is going to contract sooner than later. NoCode infrastructure is where k8s was in 2014. IT workers thinking in IT patterns as usual is a huge blocker to progress. Elder workers domain knowledge is stored in DBs and source control everywhere. It’s trivial to parse and transform into another syntax. Congrats; you trained your Copilot replacements without even knowing it.

honey-badger 5y ago

QA Engineer here. We don't have a centralized QA department, but we do have a dedicated engineer who is responsible for QA that is integrated within the development team. Before I joined my team, it didn't have a dedicated QA member. The quality of the software was fine, but there were other compromises. The team didn't have a good test strategy - every developer made adhoc decisions on how their code was tested. Our E2E tests ran slowly and had tonnes of duplicates, since nobody had gone through the entire list and cleaned it up. A dedicated QA member has the time and the responsibility to solve problems with quality that most devs are likely to treat as secondary to developing features. And the improvements are substantial - our E2E tests now run 18x (!) faster. For the most part, I see my role as being an enabler - I help devs build quality into their work and hold them accountable to it. I don't see it as an adverserial division of responsibility, but more as a collaborative effort, with the dev and QA coming from a different focal point. The problem with strict division of labour and adverserial QA is that it leads to 1) Organizational / team silos 2) Local optimization (software development and QA is intricately interwoven, so it doesn't lend itself well to this strict division) 3) Information overhead and loss A good QA engineer, IMO, requires better coordination and communication skills than a developer because they liaise with multiple developers and inspire them to make quality a priority in their work. Further, QA is involved from the definition of a feature (in sprint refinements) right up to when code is deployed and runs well in production. Therefore, at certain points, the boundaries between QA and the Dev team or QA and the PO blur and disappear entirely. I found this to be a great read that summarizes my thoughts on the topic: https://www.thoughtworks.com/en-de/insights/blog/qa-dead https://www.thoughtworks.com/en-de/insights/blog/qa-dead

martin_drapeau 5y ago
I really like this approach. An engineer who's responsiblity and focus is QA.
cranky908canuck 5y ago
If QA and developers come from different tracks (eg, devs from CS, QA from problem domain [[ eg. BSc/CS vs BComm/Acct]] ), then QA looks at the things that "weren't in the story requirements" but matter for the product. If it's a collective "we learned from this", that's good. If it's seriously and organizationally adversarial, that's bad.

twblalock 5y ago

At this point I only see value in manual QA testing for front-end apps. It's now possible to test pretty much anything via automation, although front-end testing is still harder than most and there are some intangible UX issues that can only be experienced by human users. But for testing APIs? It is normally possible to cover 100% of what is needed via a combination of unit tests and integration tests. This is especially true now that tooling exists that can bring up ephemeral test environments with dependencies such as databases etc.

aeturnum 5y ago

To me the benefit of a QA team comes from synthesis. If you are providing one service to a few groups you don't get that much out of having a dedicated team. However, once you start having different people using your product in different ways it helps enormously to have a teams whose job is to be familiar with customer pain points and setups and warn you that your change is going to trigger problems.

sidcool 5y ago

Yes. They test by considering the app as a black box. They understand business.

KingOfCoders 5y ago

The biggest benefits a QA team can give is unit/e2e tests code reviews and consult product managers. The worst value they can provide with manuell testing (exc pt exploratory testing)

martin_drapeau 5y ago

Are your product managers/owners involved in testing new features? They can play an important role here. Are you a SaaS practicing continuous delivery? Can customers report bugs rapidly? Can you fix them rapidly? If you can reduce the turn around time of fixing a bug to less than 1 hour, you can use your customers to QA. If so, you should not have a QA department. They only slow things down. Of course, this depends on the type of business you are in.

shmde 5y ago

Was a QA engineer. Yes the company I was working with had a full blown massive 100-150 QA engineers working full time. Manual Testers, Automated Testers, Managers managing them, Test Case documenters. The QA and the dev team worked very very closely.

falcolas 5y ago

No. I wish we did. Well, I guess we do, but we have 1 QA who is shared between 5 logical teams, over 40 developers. AKA, no, we don't.

salawat 5y ago

Built a QA department from scratch. Places are getting tight on the purse strings in general, and you can tell when software comes from somewhere that divested themselves from QA as a business concern. Ratios of 4+ to 1 dev:QA abound, and the depth to which QA departments can even do their primary business integrative functions suffers greatly for it. The most irritating aspect I notice is the tendency for having to do the same project multiple times because requirements, and user/devteam communication isn't what it should be. QA's increasingly are facing an uphill battle against information overload, that management and typical textbook PM processes are very ill-equipped to detect; there's also the fact that your "contract" QA departmemt, does not have incentive alignment. The most horrible devs can be endured by a QA group that cycles out QA's on a 2-3 month basis. The in-house QA group that logs in to the same steaming pile of low-quality crap will soon either find themselves motivated to trim waste or otherwise increase the efficiency and processes where they are. I can only ever get FTE QA's to even deeply learn the business... I can't do that generally with contractors. So if places are doing away with QA... I wouldn't be surprised. And I don't think the upswing in software origin'd suffering from my perspective is just a coincidence either.

HyperRational 5y ago

harel 5y ago

Maintaining test automation is equivalent to maintaining the code base. A dedicated qa team should be doing that as well as manual testing, infra tests, etc. We had a dedicated qa in a project, and when he was taken away twat automation pretty much died. Nobody maintained it.

yabones 5y ago

I'm seeing lots of good reasons for keeping a full QA team around, want to add one more: They're basically the only people that know everything about the product. More than product managers who generally specialize in the Foo feature, more than the engineers, certainly more than sales. When a seriously complex support issue comes in, they are always the most valuable people to take a look. To me, the value of QA is more about product knowledge and expertise than just "testing things".

hogrider 5y ago

I've been doing QA my entire couple of years career and I think most companies can do just fine with devs testing their code. My job is just too alienating to be done effectively IMO.

APhoenixRises 5y ago

In an ideal world, devs would always have the time to test their own code. The unit and integration bits aren't the pain point, but once you reach the API or UI, people's knowledge varies widely. That's where a good QE org, either embedded or a mix of centralized and embedded, can be very helpful. In my past companies we've pushed to be champions or domain experts in test automation who's main purpose is to make testing for everyone else suck less. Sure, you will definitely take the brunt of test automation tasks, but some of those tasks should be targeted towards removing impediments for testing. In one situation I ran into a team who was very frustrated with their automation because it was so slow. After a bit of investigation, I found out they were setting up all of their test data through database migrations, which were 10+ minutes slow. I asked why they weren't using the API to setup test data, to which they replied that none of them had experience with the API. After hacking away it in my spare time for a few weeks, I had a perfectly functional API client, and the tests were converted over time out of using the migrations. I saw this the other day, and while it might sound cheesy, but someone said that QA to them was "quality acceleration", which actually fits fairly well. I have never worked at a company who had enough testers to go around. Keeping a smaller group of centralized testers to handle the really tough technical challenges, and a larger group of embedded testers who evangelize on testing practices was the most successful approach. In this case, I wouldn't say you absolutely need a full quality team, but having some folks whose primary focus is testing and test automation to help skill up your dev teams sounds reasonable.

honey-badger 5y ago
Quality Acceleration is brilliant! I love it. >In this case, I wouldn't say you absolutely need a full quality team, but having some folks whose primary focus is testing and test automation to help skill up your dev teams sounds reasonable. This is what we have at our current team, as I have described in a comment above, and it has worked very well for us.

jbjbjbjb 5y ago

You’re not the only ones doing it. The idea is developers do more testing, they think about it upfront and they have assistance from a QA lead who leads and coaches the quality culture across the development team. That QA leads embeds the skills in the dev team and oversees the quality. The QA lead can also be freed up to look at quality from other perspectives like is the product actually delivering value to the customer overall. The devs are already working across the SDLC so have the knowledge and the skin in the game to do a good job. It also works more efficiently with less handover which is better for frequent releases vs a time when releases were once every two years.

osigurdson 5y ago

We have a dedicated QA team. Generally developers write unit tests while QA writes more integration level tests (mostly back-end focused). They do some exploratory manual testing while most of the repetitive work is managed with (brittle) UI automation tests. It is a lot of work to maintain all of the tests that we have. However, we have an incredibly good track record of releasing on time and critical bugs are rarely introduced.

michael_j_x 5y ago

We have 2 dedicated QAs per team of 6 devs. They are not only tasked with finding dev bugs, but also business "bugs". Things that the POs did not consider when writing the stories, requirements that the devs did not understand correctly erc. This has allowed us to move much faster and deploy with confidence, as the QAs also do the UAT part during their QA

throwawayHN378 5y ago

I work for a 10,000 employee organization ( not technically a software company but definitely an who’s customers are engineers and has a strong engineering culture ). We don’t have dedicated QA people by name, but we do have sever BAs who are essentially dedicated QA. Commits need to have test cases, then they get code reviewed, merged into erotic branch, epic branch merged into one of the dev. The. Our technical product managers generally test it ( at my org our technical product managers are highly technical and write code themselves ). Then generally we merge into test and eventually BAs test it. That’s generally how it works where I’m at currently. I’ve only ever worked for one org with a dedicated QA team

nouveaux 5y ago

In general, the people who write and/or design the code are the ones who knows how to best test the code. These are the people who have thought most deeply about the code that is written, not only in the code itself but also how it fits in the larger context. Good QA is hard to find, and in general are underpaid. In my opinion, good QA is like good security researchers. They know how to break the system. These people are invaluable. As a developer, you should want to write tests and encourage people on your team to love it. Good tests let you sleep at night and enjoy your weekends. If you perception is that "writing these tests are a waste of my time", I think that's actually not in your best interest.

coward123 5y ago

Welcome to the cycle... don't worry, they will make you do devops soon enough too... then, maybe after they've burned through enough people or have lost enough money through broken software and ops, they will start hiring QA and ops again. Or, they will send it all offshore because the onshore talent isn't any good anyway - they keep leaving and costing us money with their broken software! Rinse, repeat.

deleted 5y ago

[deleted]

PeterWhittaker 5y ago

For our hardware product, no; for our software product, yes, from time to time. The hardware product is a network security device that essentially works as a black box, with limited inputs and outputs. It requires so much domain expertise that the small group of us that work on it take an incremental approach both to feature development and to bug fixes, with a fair amount of design review, code review, discussion, prototyping and PoC, etc., before moving to actual testing. Most recently, I've tended to do the bulk of the development and one of my colleagues the bulk of the testing. When we find errors, there is a lot of screen sharing to review code and logs, determine the best way forward, etc. Our build process generates ISOs with several layers of security, from completely locked down release instances to a permissive mode with a development account for the privileged access necessary to run onboard diagnostics. The caveat is that one of our main customers is a security agency with complementary expertise, and they tend to beat the bejeesus out of it. Overall, we take a similar approach to our software product (design and code reviews, much screen sharing, discussion between development, marketing, and senior management on what it should do and what it should look like, etc.), but at some point it gets turned over to QA. I wrote above that our QA resource was full time from time to time, because when they are not doing QA on the product, they are doing market research and analysis, developing marketing strategy, etc. From on QA PoV, one major the difference between the two products is audience: The hardware product is used by domain experts to do very specific network segmentation and data transfer; the software product is intended for general use by non-IT corporate users - it's a web application for integrated risk management and compliance reviews. We consider ourselves competent to fully test the hardware product we've updated, because its functionality is essentially binary: either it works or it doesn't, there aren't a lot of variations in use case, and we are best positioned to determine how it should work and whether it is working. We don't consider ourselves competent to fully test the software product, because we are a) too close to it, b) domain experts, c) not typical users, and because d) we have a lot of experience with a lot of different software. Our users are often people who mostly use browsers, and occasionally office products. They may be domain experts, but their domains are far from IT (especially the physical security people). Our QA person is excellent at getting into their shoes, so to speak, imagining how they might approach things, and findings both bugs and ways that our designs and GUIs and UXs lack and don't help users get to where they need. Having someone with that external perspective is invaluable. Having them be full-time when in QA mode helps us move quickly when we need to - they are never the bottleneck, and anything released has been thoroughly beaten up.