4 ms·
This is an interesting idea. Let's say there's a new feature where electric damage and cold damage ignores metal armor. But suddenly there's a bug where if the
by muzani 5y ago
This is an interesting idea. Let's say there's a new feature where electric damage and cold damage ignores metal armor. But suddenly there's a bug where if the weapon does both electric and cold damage, the physical damage also ignores armor.
How would someone design tests that work well for roguelikes? It's not simply Human strikes Orc for 10 damage. Damage range may be a bell curve of 5-15. There's misses and hits. There's little bonuses that increase the miss probability, probably to ridiculous levels. There's armor, calculated from things like race, constitution, magic, divine bonuses, class mastery bonus.
And then you have procedurally generated... stuff. Equipment are the easiest to control. Some games have procedurally generated monsters, some have procedurally generated deities. To be able to reduce this stuff to tests kind of defeats the purpose of doing it, which is to be unpredictable.
Sometimes you have a combo of things that add up to 40% damage resistance. Is 40% too much or too little?
After all this, how do you detect that the bugged spear is doing an unreasonable amount of damage?
- personjerry 5y agoIn cases like this we use mock functions to replace the rng dependent functions during testing. https://en.m.wikipedia.org/wiki/Mock_object https://en.m.wikipedia.org/wiki/Mock_object
- deleted 5y ago[deleted]
- muzani 5y agoThis won't catch the procedurally generated edge case though unless it mocks every possible combination. It could well run into millions of scenarios just to test damage.
- a_e_k 5y ago(a) Set a deterministic RNG seed before each test. and/or (b) Have each test loop enough to get a good estimate of the probability distribution.
- muzani 5y agoB seems fine. But wouldn't a RNG seed give different results as you add more randomly generated features?
- camtarn 5y agoMonte Carlo simulation? Have your CI server run millions of games in fast forward, and collect as many statistics as you can. When you make a change, check that the stats that you expected to change did change, and that others stayed the same. Wouldn't detect extremely improbable things, though, like having a spear with a super-rare effect + having armour with another super-rare effect, since those would show up so infrequently that they wouldn't skew the overall stats much.