5 ms·

Yeah if they push too hard, more developers will go the "early access IS QA" route. game devs have already started gamifying bug reporting for in game loot, ski

by elif 3y ago

Yeah if they push too hard, more developers will go the "early access IS QA" route. game devs have already started gamifying bug reporting for in game loot, skins, etc. No one is more eager or qualified to give feedback about games but the gamers themselves. The downside in this industry is minimal. No one can sue a game dev if a bug slips out, assets can be rolled back or whole systems changed to negate their impact with no recourse available to consumers.

MichaelZuo 3y ago
All the widely anticipated games have more than enough folks nowadays willing to endure through janky private alphas and betas. So much so that it's common for a lot of bugs identified in later beta stages to not even be fixed. So a large QA team is only needed nowadays for games that nobody is really that hyped about, or games that cannot risk any chance of plot-lines/art leaking before release.
- BehindBlueEyes 3y ago
  Yes and no, guess who has to repro bugs so they can be assigned for fixing based off what players report during alpha/beta tests?
  - MichaelZuo 3y ago
    In this case it’s actually detrimental to have a large QA team. A smaller, more competent team is invariably better.
ryandrake 3y ago
QA is not just about running the software, observing problems, and filing them. As you say, eager beta testers can do that! QA is also about: 1. Exploring every corner of the software, comparing it to the written spec, and measuring any discrepancies, even those not visible to the user. 2. Exploring beyond the corners of the spec and using that exploration to improve the documents. 3. Writing software to automate this exploration and to automate defect-finding in general. 4. Fuzzing and other “red team” penetration testing. 5. Participating in design reviews and contributing towards the testability of the product. In other words working side by side with the developer to ensure the hooks are there to make the software testable and automatable. 6. Monitoring metrics and telemetry to find trends that are emerging post-deployment and getting them documented and reported before they are widely noticed. And so on. QA should be a highly skilled, highly technical job, a level equal to developers. If QA at your company is just waiting for a new build to get thrown over the wall so they can manually test it in a vacuum, your company might be doing it wrong.
- bdowling 3y ago
  I really like this idea, but what you’re describing sounds more like “Software Engineer, Testing Department” than QA. In my experience, QA functions more like an extension of automated unit tests for the things an engineer hasn’t gotten around to automating yet.
  - SOLAR_FIELDS 3y ago
    SDET (Software Developer Engineer in Test) as a title/career path seems to have gone mostly out of vogue in the past decade or so unfortunately. Most companies I've worked for seem to hold the philosophy that the engineers developing the features themselves should do the testing. Which is a legitimate viewpoint, but there are of course pros and cons to everything and not every use case is really suited for that. I think part of the problem for SDET is that pay bands are often different (read: lower) for SDET, and if you are doing SDET you already probably have the skills to be a developer and are going to opt for the higher pay band/career path. A lot of companies seem to hold a weird perspective that SDET is "lesser" or requires less skills than just a typical feature developer.
- ThrowawayR2 3y ago
  General software QA does, or at least should, work like that but games QA isn't like general software QA. Since games software is basically a one-shot project (i.e. there's no expectation of long term support beyond a few patches) and not really amenable to automation, games QA are usually non-technical manual testers given fixed scripts to execute.
  - PsylentKnight 3y ago
    > Since games software is basically a one-shot project (i.e. there's no expectation of long term support beyond a few patches) That used to be the norm but it’s not always now. Just look at Minecraft or Terraria, they’ve been getting updates for over a decade now
    - serf 3y ago
jayd16 3y ago
Actual QA and the fan feedback are just not the same. For one, you won't get hate mail from QA when something is nerfed.
- elif 3y ago
  i would argue that's a negative for a QA department and a positive for player feedback. generally a good game developer wants to know how they've affected balance, which is something an acceptance test won't tell you.
  - jayd16 3y ago
    The QA team will tell you how the game plays but the difference is they're professionals.
    - antisthenes 3y ago
      They're most definitely not professionals in the sense that their profession isn't playing the game competitively. They might be better skilled than the average players, but their actual profession is QA, e.g. making sure the game runs, not that it's perfectly balanced. As far as balance feedback, you can't do better than actual top players.
      - BehindBlueEyes 3y ago
        Actually pro players may have the deepest understanding of the balance but they only represent a tiny fraction of players and while useful for esports, implementing their expert feedback may alienate the broader audience. QA may provide good balance feedback but are rarely valued for it/do it on their own time with career progression in mind. They often are a rare combination of players who also understand the dev side. For actual tuning before release, playtesting and other user research involving players representative of the target audience is the gold standard.
      - jayd16 3y ago
        Just becauae you can drive a car it doesn't mean you can make a car. Player feedback is not useless but it's not the same at all.
scrps 3y ago
Even early access fails for QA. I was randomly selected for a private beta of a game, played it, sent feedback, about 20+ bugs none of which were fully game breaking but I'd say some were pretty severe... Every bug was present on release. Every. Bug. Corporate thinking being what it is, I don't hold out hope they will ever learn you can't lean and bean-count a creative industry, it isn't a widget factory. Upside is when they lean so much their financials get shakey because gamers are staying away you can pick up their entire catalog for 75%+ off, even better if you pick a game with a solid mod community that patches and adds functionality.

by elif 3y ago

MichaelZuo 3y ago

All the widely anticipated games have more than enough folks nowadays willing to endure through janky private alphas and betas. So much so that it's common for a lot of bugs identified in later beta stages to not even be fixed. So a large QA team is only needed nowadays for games that nobody is really that hyped about, or games that cannot risk any chance of plot-lines/art leaking before release.

BehindBlueEyes 3y ago
Yes and no, guess who has to repro bugs so they can be assigned for fixing based off what players report during alpha/beta tests?
- MichaelZuo 3y ago
  In this case it’s actually detrimental to have a large QA team. A smaller, more competent team is invariably better.

ryandrake 3y ago

QA is not just about running the software, observing problems, and filing them. As you say, eager beta testers can do that! QA is also about: 1. Exploring every corner of the software, comparing it to the written spec, and measuring any discrepancies, even those not visible to the user. 2. Exploring beyond the corners of the spec and using that exploration to improve the documents. 3. Writing software to automate this exploration and to automate defect-finding in general. 4. Fuzzing and other “red team” penetration testing. 5. Participating in design reviews and contributing towards the testability of the product. In other words working side by side with the developer to ensure the hooks are there to make the software testable and automatable. 6. Monitoring metrics and telemetry to find trends that are emerging post-deployment and getting them documented and reported before they are widely noticed. And so on. QA should be a highly skilled, highly technical job, a level equal to developers. If QA at your company is just waiting for a new build to get thrown over the wall so they can manually test it in a vacuum, your company might be doing it wrong.

bdowling 3y ago
I really like this idea, but what you’re describing sounds more like “Software Engineer, Testing Department” than QA. In my experience, QA functions more like an extension of automated unit tests for the things an engineer hasn’t gotten around to automating yet.
- SOLAR_FIELDS 3y ago
  SDET (Software Developer Engineer in Test) as a title/career path seems to have gone mostly out of vogue in the past decade or so unfortunately. Most companies I've worked for seem to hold the philosophy that the engineers developing the features themselves should do the testing. Which is a legitimate viewpoint, but there are of course pros and cons to everything and not every use case is really suited for that. I think part of the problem for SDET is that pay bands are often different (read: lower) for SDET, and if you are doing SDET you already probably have the skills to be a developer and are going to opt for the higher pay band/career path. A lot of companies seem to hold a weird perspective that SDET is "lesser" or requires less skills than just a typical feature developer.
ThrowawayR2 3y ago
General software QA does, or at least should, work like that but games QA isn't like general software QA. Since games software is basically a one-shot project (i.e. there's no expectation of long term support beyond a few patches) and not really amenable to automation, games QA are usually non-technical manual testers given fixed scripts to execute.
- PsylentKnight 3y ago
  > Since games software is basically a one-shot project (i.e. there's no expectation of long term support beyond a few patches) That used to be the norm but it’s not always now. Just look at Minecraft or Terraria, they’ve been getting updates for over a decade now
  - serf 3y ago

jayd16 3y ago

Actual QA and the fan feedback are just not the same. For one, you won't get hate mail from QA when something is nerfed.

elif 3y ago
i would argue that's a negative for a QA department and a positive for player feedback. generally a good game developer wants to know how they've affected balance, which is something an acceptance test won't tell you.
- jayd16 3y ago
  The QA team will tell you how the game plays but the difference is they're professionals.
  - antisthenes 3y ago
    They're most definitely not professionals in the sense that their profession isn't playing the game competitively. They might be better skilled than the average players, but their actual profession is QA, e.g. making sure the game runs, not that it's perfectly balanced. As far as balance feedback, you can't do better than actual top players.
    - BehindBlueEyes 3y ago
      Actually pro players may have the deepest understanding of the balance but they only represent a tiny fraction of players and while useful for esports, implementing their expert feedback may alienate the broader audience. QA may provide good balance feedback but are rarely valued for it/do it on their own time with career progression in mind. They often are a rare combination of players who also understand the dev side. For actual tuning before release, playtesting and other user research involving players representative of the target audience is the gold standard.
    - jayd16 3y ago
      Just becauae you can drive a car it doesn't mean you can make a car. Player feedback is not useless but it's not the same at all.

scrps 3y ago

Even early access fails for QA. I was randomly selected for a private beta of a game, played it, sent feedback, about 20+ bugs none of which were fully game breaking but I'd say some were pretty severe... Every bug was present on release. Every. Bug. Corporate thinking being what it is, I don't hold out hope they will ever learn you can't lean and bean-count a creative industry, it isn't a widget factory. Upside is when they lean so much their financials get shakey because gamers are staying away you can pick up their entire catalog for 75%+ off, even better if you pick a game with a solid mod community that patches and adds functionality.