9 ms·

Not sure of Mozilla’s financial or organizational structure but it seems to be part of a larger trend of de-emphasizing QA departments at software shops large a

Not sure of Mozilla’s financial or organizational structure but it seems to be part of a larger trend of de-emphasizing QA departments at software shops large and small over the past 10 or so years. In many ways test automation tooling has become much easier to use, develop, and manage. But I suspect the larger driving force is that it’s (arguably) a cost center for an org. The burden of ensuring software quality can be shifted to devs and PMs, though usually with mixed results. For Mozilla, axing quality and security first is a bad look when those are crucial aspects of a privacy-first company value.

rogueresearch 7y ago
>The burden of ensuring software quality can be shifted to devs and PMs Heck, the burden can be shifted to users! But then you might not have so many users in the future. :(
- dgellow 7y ago
  That seems to work for Microsoft so far.
- traskjd 7y ago
  We see this at Raygun with our Crash Reporting and Real User Monitoring products (measuring errors, and end user experience in general). Nearly all power users are more likely to be in product, or very senior/executive developers. Very low engagement out of QA. I'm not sure if they just have a way they like to work, or they think that if they exist then no bugs make it to prod (don't laugh, plenty of our twitter adverts get comments from folks thinking you don't need any monitoring if you have a tester on the team or if a dev writes unit tests). Understand how the end user experiences your software. That's the actual source of the truth.
- scarface74 7y ago
  i.e. Netscape. It was a point of nerd pride on Usenet how well your operating system could handle a Netscape crash back in the day.
- Aperocky 7y ago
  > The burden of ensuring software quality can be shifted to devs and PMs This in itself is a not a problem, coming from an oncall minded company. You take care of code you write and you're given time budget to KTFLO.
jonny_eh 7y ago
Especially just after this happened: https://www.macrumors.com/2020/01/10/mozilla-firefox-update-vulnerability/ https://www.macrumors.com/2020/01/10/mozilla-firefox-update-...
weberc2 7y ago
I wonder if it's improvements in automation or if QA responsibilities are increasingly rolling up into standard developer roles or because the line between QA and dev is blurring (e.g., software testing now requires stronger dev/automation skills so the QA job looks more like standard software engineering)?
- ivan_gammel 7y ago
  I don't think automation or shift of testing responsibilities to developers is harming QA role in any way, it's just the first candidate for layoffs, because it does not have immediate impact on business. Independent SQA is still relevant, even it it becomes a developer specialization.
  - weberc2 7y ago
    That wouldn't explain the perceived increase in QA layoffs; you would need something else, such as worse economic conditions.
- xemdetia 7y ago
  I agree with the concept that it is turning into something else, as I see DevOps/SRE/CI/PM people doing QA work through trying to just make the thing go. It just is leaving a lot of gaps in longevity, complex setup, and building tests either manual or automated to do that better. In the last dozen or so teams I've interfaced with this is just getting worse as I'm seeing lots of situations where things like backup/restore breaks from the last hotfix, upgrade a system a few sequential versions, and only covering X-1/X critical features in ordinary testing. These are things that a traditional QA team would cover after it burns the business/org in one way or another, but since it's not a core focus of some of these people covering parts of the same role it's leaving a lot of not-obvious gaps in a lot of groups.
- thrower123 7y ago
  Developers are just turning into glorified hatracks at this point, and the results are discouraging. It's no wonder that people burn out when they are expected to do ever more and more with less.
crazypython 7y ago
Mozilla uses AFL, which is a genetic algorithm that tests code paths. They are also transitioning to Rust, which will give them a much bigger safety guarantee over most of their code and a much smaller audit surface for the rest.
- Thaxll 7y ago
  Rust has nothing to do with QA things since most bug are business logic / UI / UX related. QA don't test for those kind of errors.
  - mffnbs 7y ago
    In my experience, that’s the exact sort of errors that a QA team our position tests for.
    - newnewpdro 7y ago
      Their comment meant that QA don't test for the kinds of errors (safety) rust protects against.
- zozbot234 7y ago
  They're transitioning quite slowly, a sibling comment mentions https://wiki.mozilla.org/Oxidation#Rust_Components https://wiki.mozilla.org/Oxidation#Rust_Components this page. Even accounting for how they're obviously prioritizing their efforts to get the best bang for the buck, there will be a lot of critical C/C++ code in Mozilla products for a pretty long time.
- SEJeff 7y ago
  American Fuzzy Lop? It's a fuzzer. I've never heard of a fuzzer described as a genetic algorithm that tests code paths, but that is technically correct. A fuzzer is not going to replace unit tests or good SDLC, which often involves QA.
  - craftinator 7y ago
throwaway5752 7y ago
It sounds organizational. The one reference to QA is that the leads were let go, which might mean that the teams were re-orged under a product aligned structure. There is a lot of "shift-left" emphasis out there, but ultimate their is a conflict of interest problem. I think there's no need to go beyond the "we are losing money" explanation and QA is considered more of a cost center.
zelly 7y ago
If there's anything they need to axe, it's the Gecko team. Just replace it with V8. The whole layout engine too--replace it with Blink. It is inevitable, so might as well get over with it now and save the wasted human effort and $$$. I tried to use Firefox recently. It leaked 28 GiB of RAM on x86_64 GNU/Linux with no extensions except uBlock Origin. Happened a few times over the month whenever I visited JS-heavy websites. Never had that happen with Chromium, which runs through megs of JS like butter. Wouldn't it be nice if an experienced browser dev team maintained a privacy-oriented libre version of Chrome (without manifest v3, sync, and all that trash). Or should they keep doing what they've doing and make the best pro-privacy browser that no one ever uses except indirectly through Tor Browser.
- la_fayette 7y ago
  Cpu usage of chrome is higher on chrome on mobile. Cpu affects battery life more than memory... so firefox is a good choice on mobile.
  - The_rationalist 7y ago
    Websites are not tested for Firefox for Android.
    - tomrod 7y ago
      Indeed. I keep Brave around just to handle the 5% of sites that are decidedly not FF friendly. FF -- fix text sizing. As an example, old-style Reddit renders wonderfully on Brave and terribly terribly small on FF mobile. Another tester site is BOFH.
- saagarjha 7y ago
  > If there's anything they need to axe, it's the Gecko team. Just replace it with V8. The whole layout engine too--replace it with Blink. It is inevitable, so might as well get over with it now and save the wasted human effort and $$$. And further contribute to the browser monoculture? > I tried to use Firefox recently. It leaked 28 GiB of RAM on x86_64 GNU/Linux with no extensions except uBlock Origin. Happened a few times over the month whenever I visited JS-heavy websites. And you've reported this, I assume? > Never had that happen with Chromium, which runs through megs of JS like butter. Really? I explicitly avoid Chrome on my computer because it can't handle the web without chewing through my RAM. > Wouldn't it be nice if an experienced browser dev team maintained a privacy-oriented libre version of Chrome (without manifest v3, sync, and all that trash). Or should they keep doing what they've doing and make the best pro-privacy browser that no one ever uses except indirectly through Tor Browser. I would like the experienced Mozilla team to continue to work on their pro-privacy browser than a decent number of people use.
specialist 7y ago
Obligatory: quality assurance is not testing. "...de-emphasizing QA departments..." I feel like we (the industry) have forfeited. I did a stint as a SQA manager. Coming from a dev background, I took it very seriously, totally immersed myself in the domain, transmuted from skeptical to true believer. I honestly don't know what to make of today's state of affairs. For example, today's business analysts often do many of the tasks we used to associate with the QA role. Testing, verification, liaison with customers. Did we just rename the role? Did "Agile" smother QA? Until very recently, I've never heard an Agile explanation for how to do QA/Test. I mean really do it, not just wave your arms. The "Test Into Prod" thesis, strategy, whatever, is the first intellectually honest, actionable, constructive (criticizable) methodology I've seen which is tailored for our new market realties. I've never understood Agile. My teams were way more "nimble" (to use a different adjective) using PMI, critical path, iterative, lightest weight decision making, front loading work, managing risk, and so forth. All the battle hardened time proven stuff people untrained in project management pejoratively call "waterfall". One correct criticism of all failing methodologists, including Agile, is lack of feedback loops. The "throwing over the wall" of work downstream. We designed feedback loops into our processes, some of what today would be called CI/CD. We were definitely not waterfall. (Another is managing transaction costs, something Agile has manifestly failed to do.) Rant over. Sorry. Now get off my lawn.
- celticmusic 7y ago
  > For example, today's business analysts often do many of the tasks we used to associate with the QA role. Testing, verification, liaison with customers. Did we just rename the role? The person asking for a features should absolutely be the one signing off on the feature.

rogueresearch 7y ago

>The burden of ensuring software quality can be shifted to devs and PMs Heck, the burden can be shifted to users! But then you might not have so many users in the future. :(

dgellow 7y ago
That seems to work for Microsoft so far.
traskjd 7y ago
We see this at Raygun with our Crash Reporting and Real User Monitoring products (measuring errors, and end user experience in general). Nearly all power users are more likely to be in product, or very senior/executive developers. Very low engagement out of QA. I'm not sure if they just have a way they like to work, or they think that if they exist then no bugs make it to prod (don't laugh, plenty of our twitter adverts get comments from folks thinking you don't need any monitoring if you have a tester on the team or if a dev writes unit tests). Understand how the end user experiences your software. That's the actual source of the truth.
scarface74 7y ago
i.e. Netscape. It was a point of nerd pride on Usenet how well your operating system could handle a Netscape crash back in the day.
Aperocky 7y ago
> The burden of ensuring software quality can be shifted to devs and PMs This in itself is a not a problem, coming from an oncall minded company. You take care of code you write and you're given time budget to KTFLO.

jonny_eh 7y ago

Especially just after this happened: https://www.macrumors.com/2020/01/10/mozilla-firefox-update-vulnerability/ https://www.macrumors.com/2020/01/10/mozilla-firefox-update-...

weberc2 7y ago

I wonder if it's improvements in automation or if QA responsibilities are increasingly rolling up into standard developer roles or because the line between QA and dev is blurring (e.g., software testing now requires stronger dev/automation skills so the QA job looks more like standard software engineering)?

ivan_gammel 7y ago
I don't think automation or shift of testing responsibilities to developers is harming QA role in any way, it's just the first candidate for layoffs, because it does not have immediate impact on business. Independent SQA is still relevant, even it it becomes a developer specialization.
- weberc2 7y ago
  That wouldn't explain the perceived increase in QA layoffs; you would need something else, such as worse economic conditions.
xemdetia 7y ago
I agree with the concept that it is turning into something else, as I see DevOps/SRE/CI/PM people doing QA work through trying to just make the thing go. It just is leaving a lot of gaps in longevity, complex setup, and building tests either manual or automated to do that better. In the last dozen or so teams I've interfaced with this is just getting worse as I'm seeing lots of situations where things like backup/restore breaks from the last hotfix, upgrade a system a few sequential versions, and only covering X-1/X critical features in ordinary testing. These are things that a traditional QA team would cover after it burns the business/org in one way or another, but since it's not a core focus of some of these people covering parts of the same role it's leaving a lot of not-obvious gaps in a lot of groups.
thrower123 7y ago
Developers are just turning into glorified hatracks at this point, and the results are discouraging. It's no wonder that people burn out when they are expected to do ever more and more with less.

crazypython 7y ago

Mozilla uses AFL, which is a genetic algorithm that tests code paths. They are also transitioning to Rust, which will give them a much bigger safety guarantee over most of their code and a much smaller audit surface for the rest.

Thaxll 7y ago
Rust has nothing to do with QA things since most bug are business logic / UI / UX related. QA don't test for those kind of errors.
- mffnbs 7y ago
  In my experience, that’s the exact sort of errors that a QA team our position tests for.
  - newnewpdro 7y ago
    Their comment meant that QA don't test for the kinds of errors (safety) rust protects against.
zozbot234 7y ago
They're transitioning quite slowly, a sibling comment mentions https://wiki.mozilla.org/Oxidation#Rust_Components https://wiki.mozilla.org/Oxidation#Rust_Components this page. Even accounting for how they're obviously prioritizing their efforts to get the best bang for the buck, there will be a lot of critical C/C++ code in Mozilla products for a pretty long time.
SEJeff 7y ago
American Fuzzy Lop? It's a fuzzer. I've never heard of a fuzzer described as a genetic algorithm that tests code paths, but that is technically correct. A fuzzer is not going to replace unit tests or good SDLC, which often involves QA.
- craftinator 7y ago

throwaway5752 7y ago

It sounds organizational. The one reference to QA is that the leads were let go, which might mean that the teams were re-orged under a product aligned structure. There is a lot of "shift-left" emphasis out there, but ultimate their is a conflict of interest problem. I think there's no need to go beyond the "we are losing money" explanation and QA is considered more of a cost center.

zelly 7y ago

If there's anything they need to axe, it's the Gecko team. Just replace it with V8. The whole layout engine too--replace it with Blink. It is inevitable, so might as well get over with it now and save the wasted human effort and $$$. I tried to use Firefox recently. It leaked 28 GiB of RAM on x86_64 GNU/Linux with no extensions except uBlock Origin. Happened a few times over the month whenever I visited JS-heavy websites. Never had that happen with Chromium, which runs through megs of JS like butter. Wouldn't it be nice if an experienced browser dev team maintained a privacy-oriented libre version of Chrome (without manifest v3, sync, and all that trash). Or should they keep doing what they've doing and make the best pro-privacy browser that no one ever uses except indirectly through Tor Browser.

la_fayette 7y ago
Cpu usage of chrome is higher on chrome on mobile. Cpu affects battery life more than memory... so firefox is a good choice on mobile.
- The_rationalist 7y ago
  Websites are not tested for Firefox for Android.
  - tomrod 7y ago
    Indeed. I keep Brave around just to handle the 5% of sites that are decidedly not FF friendly. FF -- fix text sizing. As an example, old-style Reddit renders wonderfully on Brave and terribly terribly small on FF mobile. Another tester site is BOFH.
saagarjha 7y ago
> If there's anything they need to axe, it's the Gecko team. Just replace it with V8. The whole layout engine too--replace it with Blink. It is inevitable, so might as well get over with it now and save the wasted human effort and $$$. And further contribute to the browser monoculture? > I tried to use Firefox recently. It leaked 28 GiB of RAM on x86_64 GNU/Linux with no extensions except uBlock Origin. Happened a few times over the month whenever I visited JS-heavy websites. And you've reported this, I assume? > Never had that happen with Chromium, which runs through megs of JS like butter. Really? I explicitly avoid Chrome on my computer because it can't handle the web without chewing through my RAM. > Wouldn't it be nice if an experienced browser dev team maintained a privacy-oriented libre version of Chrome (without manifest v3, sync, and all that trash). Or should they keep doing what they've doing and make the best pro-privacy browser that no one ever uses except indirectly through Tor Browser. I would like the experienced Mozilla team to continue to work on their pro-privacy browser than a decent number of people use.

specialist 7y ago

Obligatory: quality assurance is not testing. "...de-emphasizing QA departments..." I feel like we (the industry) have forfeited. I did a stint as a SQA manager. Coming from a dev background, I took it very seriously, totally immersed myself in the domain, transmuted from skeptical to true believer. I honestly don't know what to make of today's state of affairs. For example, today's business analysts often do many of the tasks we used to associate with the QA role. Testing, verification, liaison with customers. Did we just rename the role? Did "Agile" smother QA? Until very recently, I've never heard an Agile explanation for how to do QA/Test. I mean really do it, not just wave your arms. The "Test Into Prod" thesis, strategy, whatever, is the first intellectually honest, actionable, constructive (criticizable) methodology I've seen which is tailored for our new market realties. I've never understood Agile. My teams were way more "nimble" (to use a different adjective) using PMI, critical path, iterative, lightest weight decision making, front loading work, managing risk, and so forth. All the battle hardened time proven stuff people untrained in project management pejoratively call "waterfall". One correct criticism of all failing methodologists, including Agile, is lack of feedback loops. The "throwing over the wall" of work downstream. We designed feedback loops into our processes, some of what today would be called CI/CD. We were definitely not waterfall. (Another is managing transaction costs, something Agile has manifestly failed to do.) Rant over. Sorry. Now get off my lawn.

celticmusic 7y ago
> For example, today's business analysts often do many of the tasks we used to associate with the QA role. Testing, verification, liaison with customers. Did we just rename the role? The person asking for a features should absolutely be the one signing off on the feature.