4 ms·

I felt like I knew the scale of game testing, but 200 still shocks me.

bombcar 4y ago
Depending on the game it would seem a small number. I’m not sure how you could test something on the WoW scale or even Hogwarts Legacy with only 200.
- Waterluvian 4y ago
  Public test realms do the heavy lifting I think. But indeed, I lack context. I don’t know how many is correct for a game of a given type and size. 200 just feels like a lot.
  - gabythenerd 4y ago
    QA normally do everything manually, with 0 automation I can see some games taking more than 500 testers to go through all possible paths. Especially in live service games like Apex. I worked briefly on a AAA game and my team had 2 dedicated american QAs for the feature that knew everything there was to know and tested happy paths/made docs, and there also was a building of testers in Ukraine that tried to break the game in a myriad of ways and filled general bugs. At some point in alpha we had around 500 bugs filled in jira, and this was only one game mode.
    - eddsh1994 4y ago
      > to go through all possible paths This isn't even possible, I'd be amazed at 25% of paths!
  - clnq 4y ago
    Laying off 200 is definitely a lot. That’s many more than most AAA studios have internally and closer to what AAA publishers usually have. Although EA is known in the industry to have relatively many QA people and good QA. Their games stand head and shoulders above many other publishers’ in terms of polish at release. I’m not sure about the exact QA figures, but I expect maybe 500? It would be a shame to see EA ruin this well functioning component of their publishing business to chase short term profit with cheapest outsourcers. But it would not be beyond the “smarts” of some game industry executives.
jack1243star 4y ago
Too few, or too many? Seems like a small number.
midoridensha 4y ago
Given the quality of today's software, 200 shocks me too: I'm surprised it's more than a handful of people, let alone 200. I thought the normal way of doing game QA was to simply release the game full of obvious bugs, and then wait for users to complain.
- Gigachad 4y ago
  Perhaps the level of complexity in modern games is such that even 200 testers is not sufficient to spot all issues. Things have moved a long way beyond 2D platformer games
shagie 4y ago
The game supports 60 players in a battle Royale at once. To do testing, you're going to need at least a multiple of 60 at once. If you want to have 3 sets concurrent, you're going to need 180 testers. As an aside, consider this story of testing and donuts - http://trenchescomic.com/tales/post/whats-up-with-donuts http://trenchescomic.com/tales/post/whats-up-with-donuts It was to have 64 simultaneous players in a game and they had 80 testers... at the start.

Depending on the game it would seem a small number. I’m not sure how you could test something on the WoW scale or even Hogwarts Legacy with only 200.

Waterluvian 4y ago
Public test realms do the heavy lifting I think. But indeed, I lack context. I don’t know how many is correct for a game of a given type and size. 200 just feels like a lot.
- gabythenerd 4y ago
  QA normally do everything manually, with 0 automation I can see some games taking more than 500 testers to go through all possible paths. Especially in live service games like Apex. I worked briefly on a AAA game and my team had 2 dedicated american QAs for the feature that knew everything there was to know and tested happy paths/made docs, and there also was a building of testers in Ukraine that tried to break the game in a myriad of ways and filled general bugs. At some point in alpha we had around 500 bugs filled in jira, and this was only one game mode.
  - eddsh1994 4y ago
    > to go through all possible paths This isn't even possible, I'd be amazed at 25% of paths!
- clnq 4y ago
  Laying off 200 is definitely a lot. That’s many more than most AAA studios have internally and closer to what AAA publishers usually have. Although EA is known in the industry to have relatively many QA people and good QA. Their games stand head and shoulders above many other publishers’ in terms of polish at release. I’m not sure about the exact QA figures, but I expect maybe 500? It would be a shame to see EA ruin this well functioning component of their publishing business to chase short term profit with cheapest outsourcers. But it would not be beyond the “smarts” of some game industry executives.

jack1243star 4y ago

Too few, or too many? Seems like a small number.

midoridensha 4y ago

Given the quality of today's software, 200 shocks me too: I'm surprised it's more than a handful of people, let alone 200. I thought the normal way of doing game QA was to simply release the game full of obvious bugs, and then wait for users to complain.

Gigachad 4y ago
Perhaps the level of complexity in modern games is such that even 200 testers is not sufficient to spot all issues. Things have moved a long way beyond 2D platformer games

shagie 4y ago

The game supports 60 players in a battle Royale at once. To do testing, you're going to need at least a multiple of 60 at once. If you want to have 3 sets concurrent, you're going to need 180 testers. As an aside, consider this story of testing and donuts - http://trenchescomic.com/tales/post/whats-up-with-donuts http://trenchescomic.com/tales/post/whats-up-with-donuts It was to have 64 simultaneous players in a game and they had 80 testers... at the start.