4 ms·

A game the scale of GW will have a dedicated QA department with at least a half-dozen testers, who are constantly playing the game anyway.

kevingadd 14y ago
This is pretty much exactly how it worked when I was there. I imagine when the team was smaller, they leveraged a combination of developer recordings + recordings from volunteer testers (the game had a group of volunteer testers in the high hundreds by the time I started working on it). In particular QA recordings got used a lot to reproduce particularly tricky bugs and race conditions, since players usually weren't savvy enough to spot them. The volunteer tester group was also useful for load testing, since it gave us a way to throw a hundred or so players at one of our development servers and record all of them simultaneously. It gave a reasonable approximation of what the load might look like on a single production server and it meant we could also capture recordings of how the servers (and clients) behaved under load instead of just having 4-8 developers playing on them.
- BryantD 14y ago
  Any comments on the infrastructure for capturing the replays? Did everything get recorded? I'm used to recording all actions, ability uses, etc. but recording movement isn't something I've seen done all that often and it seems like it'd be another order of magnitude of data.
  - endianswap 14y ago
    Think of a web server with a fresh SQL database with no data (other than whatever is pre-populated there by the installation.) Simply record all incoming messages to the server over a period of time and save it to disk. Then, when you want to replay that recording, start with a fresh database again and replay all the messages. Assuming your system is deterministic (no race conditions, generally not relying on a wall clock [there are tricks for this, since it's a harder problem[1]]) and you're good to go. The key is designing everything to support this from the beginning. [1] Done right, you can play a minutes long recording back in seconds, which is very useful for trying to fix a bug and verifying it against a known-bad replay.
- endianswap 14y ago
  QA testers could write recordings out, sure, but taking a recording from a crash and then playing it back locally to 1. inspect the crash with full debugging powers (stepping through code, seeing the full heap, etc.) and 2. being able to fix the crash and re-run the recording to verify the fix corrected the problem, were both a more common and valuable use of the recordings.

kevingadd 14y ago

This is pretty much exactly how it worked when I was there. I imagine when the team was smaller, they leveraged a combination of developer recordings + recordings from volunteer testers (the game had a group of volunteer testers in the high hundreds by the time I started working on it). In particular QA recordings got used a lot to reproduce particularly tricky bugs and race conditions, since players usually weren't savvy enough to spot them. The volunteer tester group was also useful for load testing, since it gave us a way to throw a hundred or so players at one of our development servers and record all of them simultaneously. It gave a reasonable approximation of what the load might look like on a single production server and it meant we could also capture recordings of how the servers (and clients) behaved under load instead of just having 4-8 developers playing on them.

BryantD 14y ago
Any comments on the infrastructure for capturing the replays? Did everything get recorded? I'm used to recording all actions, ability uses, etc. but recording movement isn't something I've seen done all that often and it seems like it'd be another order of magnitude of data.
- endianswap 14y ago
  Think of a web server with a fresh SQL database with no data (other than whatever is pre-populated there by the installation.) Simply record all incoming messages to the server over a period of time and save it to disk. Then, when you want to replay that recording, start with a fresh database again and replay all the messages. Assuming your system is deterministic (no race conditions, generally not relying on a wall clock [there are tricks for this, since it's a harder problem[1]]) and you're good to go. The key is designing everything to support this from the beginning. [1] Done right, you can play a minutes long recording back in seconds, which is very useful for trying to fix a bug and verifying it against a known-bad replay.
endianswap 14y ago
QA testers could write recordings out, sure, but taking a recording from a crash and then playing it back locally to 1. inspect the crash with full debugging powers (stepping through code, seeing the full heap, etc.) and 2. being able to fix the crash and re-run the recording to verify the fix corrected the problem, were both a more common and valuable use of the recordings.