3 ms·

Riot Games also has an automated testing framework: https://technology.riotgames.com/news/automated-testing-league-legends https://technology.riotgames.com/news

Riot Games also has an automated testing framework: https://technology.riotgames.com/news/automated-testing-league-legends https://technology.riotgames.com/news/automated-testing-leag... I still think the best combo is a mix of automated and QA. Software in general went through a large decline in quality after QA was laid off across the industry, and in my experience, automated testing just tends to find different kinds of issues. Also, playtesting is also not the same thing as QA, but I digress :)

tmpz22 4y ago
What a lot of people don’t realize about QA is that it’s also a point of Empathy to your end users. Some might think it’s the PMs job to think as an end user but in my experience PMs are so bogged down with other sales adjacent duties that they rarely if ever can provide that pillar. This is a core issue with for example Google products where MANUAL PROCESS BAD seems to be a religious bastion.
bongobingo1 4y ago
Wonder what their coverage is in this - or if they even still use it. There are many occurrences of "how does this ship" bugs in League where damage numbers don't add up or status's have no actual effect, etc - sometimes hanging around for many patches. Not that I don't understand how hard it is to test the big ball of mud that (10+ years old!) game engines end up being.
treeman79 4y ago
Recently Inherited a code base with almost 20000 tests. Many very detailed. Most of them were so mocked out that at end of day it was just ensuring the mock library was well tested.
- KronisLV 4y ago
  > Most of them were so mocked out that at end of day it was just ensuring the mock library was well tested. But what's the alternative here, when your code depends on other services and other logic that might return complex data structures, or need to do dozens of database calls? Should you go for primarily integration tests, which might not only slow everything down, but also force you to think about the possibility of either needing to mirror your current environment per test run (e.g. launch a database, run all the migrations, generate needed test data, launch your app against the database instance, run tests against it, clean everything up), or being limited to a single test run at a time with no parallelism?
  - ehnto 4y ago
    It depends on the codebase I think, some codebases are spaghetti mess that really benefit from being well integration tested. Other codebases are really well organized, atomic and have really clearly separated concerns. I think you can afford to run smaller less integrated unit tests. I don't think there's a right or wrong answer, just a balance of the two depending on your project.
towawy 4y ago
I strongly agree with this. You need someone around that actually uses what you develop, be it dogfooding or structured QA. I wonder if there is any books on game/software QA akin to Masters of Doom or Stay Awhile and Listen? Those pretty much skip over QA, it's usually just the devs playing their own game (which anecdotally is usually biased and of limited use).

tmpz22 4y ago

What a lot of people don’t realize about QA is that it’s also a point of Empathy to your end users. Some might think it’s the PMs job to think as an end user but in my experience PMs are so bogged down with other sales adjacent duties that they rarely if ever can provide that pillar. This is a core issue with for example Google products where MANUAL PROCESS BAD seems to be a religious bastion.

bongobingo1 4y ago

Wonder what their coverage is in this - or if they even still use it. There are many occurrences of "how does this ship" bugs in League where damage numbers don't add up or status's have no actual effect, etc - sometimes hanging around for many patches. Not that I don't understand how hard it is to test the big ball of mud that (10+ years old!) game engines end up being.

treeman79 4y ago

Recently Inherited a code base with almost 20000 tests. Many very detailed. Most of them were so mocked out that at end of day it was just ensuring the mock library was well tested.

KronisLV 4y ago
> Most of them were so mocked out that at end of day it was just ensuring the mock library was well tested. But what's the alternative here, when your code depends on other services and other logic that might return complex data structures, or need to do dozens of database calls? Should you go for primarily integration tests, which might not only slow everything down, but also force you to think about the possibility of either needing to mirror your current environment per test run (e.g. launch a database, run all the migrations, generate needed test data, launch your app against the database instance, run tests against it, clean everything up), or being limited to a single test run at a time with no parallelism?
- ehnto 4y ago
  It depends on the codebase I think, some codebases are spaghetti mess that really benefit from being well integration tested. Other codebases are really well organized, atomic and have really clearly separated concerns. I think you can afford to run smaller less integrated unit tests. I don't think there's a right or wrong answer, just a balance of the two depending on your project.

towawy 4y ago

I strongly agree with this. You need someone around that actually uses what you develop, be it dogfooding or structured QA. I wonder if there is any books on game/software QA akin to Masters of Doom or Stay Awhile and Listen? Those pretty much skip over QA, it's usually just the devs playing their own game (which anecdotally is usually biased and of limited use).