8 ms·

I'm in this research area, it's facinating, but also (like many things in AI) easy to get wrong. The trickiest bit is deciding exactly what you want your AI to

I'm in this research area, it's facinating, but also (like many things in AI) easy to get wrong. The trickiest bit is deciding exactly what you want your AI to optimise for -- If you want a game where "anyone can win", you end up with coin toss. If you want a game where "skill is everything", you can accidentally end up with "who can click fastest". On a project I recently worked on (and I'm writing up), we ended up on our 6th or 7th "optimisation function" before we found something where the "AI optimal" seemed actually enjoyable to play. Of course, if you already have a fairly fixed game, it can be easier to optimise constants, as you already know your target -- but people have also been doing that for a long time previously.

ericjang 6y ago
Interesting! Is there some kind of meta-objective for "AI optimal" that could have replaced the 6 or 7 iterations you did with human R&D? For instance, if you had real human playtesters interacting with the prototypes, is there some signal you could extract to measure that it's "good"?
- CJefferson 6y ago
  The problem is AIs are very good at optimising what you asked them for, rather than what you meant to ask for, and figuring out what you want is super hard :) As a simple example: * Start by optimising "players can always do something on their turn" -- but that just ends up with everyone always having exactly one thing they can do (no choice). * So then say "give players more things to do each turn" -- but then they end up being able to do everything every turn (the game gives them too much 'money' (still not really a choice) * OK, so we want to force players to make a choice -- so we say "No, give players as much choice as possible, but make sure if they choose an option it blocks off others (in practice, make as many sets of maximal tasks as possible)" -- but then the AI will make sure every turn every player can do (for example) exactly 3 out of 6 things (any 3), and make sure no matter how well or badly they play they still always get to choose 3 from 6, so the game doesn't really progress, or vary. So, what we want is choice, but also variability, and progress, and players to feel like they are effecting the game, but also don't let one player run away too early, but also don't make it just "feel random who wins", etc.
  - ericjang 6y ago
    Got it. I do research in reinforcement learning and I can sympathize with the difficulty here - in my experience, even something as simple as "I want to balance two objectives: 1) the agent should get a high score and 2) the agent should try to make as few decisions as possible" tends to result in the agent doing neither of those things well"
adkadskhj 6y ago
Personally, i'd like to see AI used for "simple" things, not often tied to the core gameplay loop itself. Ie conceptually if you could tie the right gameplay components to an RNG function then the user could get new/unexpected experiences for a longer time. The problem is RNG often makes crap outputs. I'd be curious to see ML take this problem. I imagine the examples are limitless, but one that comes to mind might be procedural worlds. Rather than tuning procedural worlds around seed values, what would it look like if we try to get an ML to craft the worlds? Procedurally generated worlds often have that feeling. Things don't make sense, etc. I imagine ML could make some far more diverse patterns that are both interesting and fun. I'd like to see this pattern of "informed RNG" in a lot of things. Skyrim with NPC patterns to feel a bit more authentic. Etc. I'm less interested in the classic AI dreams inside games, as it seems far out and less gamey. But our current games with informed RNG? That sounds.. neat.
- Jasper_ 6y ago
  ML is a system that takes a large data set, and an error function, and finds a generated output that minimizes the loss. What data set and error function are you proposing for "RNG"?
  - matsemann 6y ago
    That's a wrong simplification of ML. Take RL for instance. And the parent already explained a concept: generating maps. It could still have an RNG as the base (noise function over something), but then use ML to place elements based on existing human-made maps.
  - malka 6y ago
    You feed it rng created world that were curated by humans.
- jonbaer 6y ago
  I think you are right. There are SO many small mini games inside of nearly all RTS games which AI has yet to actually conquer, I will give another simple example, given a random map (and resources) identify chokepoints. As a human this is a huge clue on how to go about a game strategy (policy) but I have yet to find an AI do this (or optimize that part). From what I gather most time is spent on actually optimizing random build orders and modifying them along the way.
Quarrelsome 6y ago
I feel like some of these questions are a bit backwards. Isn't it better to give the model an arena to play within and try to make all of the options _viable_? I.E. Balance is maintained when the average of the weapons the gladiators choose to take into the arena are diverse and have less bearing on the win outcome? This means that rather focusing on what is merely broken one is focusing on both ends, what is broken and what is useless, attempting to push both towards an average of usability. Personally it astonishes me that groups like WOTC (who produce MTG) don't appear to be using such models given the quantity of mistakes they seem to continue to make in game design.
- meheleventyone 6y ago
  For something like a deck building game, if all decks are viable (competitive) then there may as well be no deck building component. In fact a game like that is probably a pretty muddy brown color in terms of card variety. That would adversely affect the business model of selling new cards as well. Desirable qualities in a game in terms of how it is balanced are way more complex than just every option being viable and leading to an even win rate against a similarly skilled opponent.
- vladTheInhaler 6y ago
  Attempting to optimize the card generation process, or deck building, or just play itself seem like unbelievably difficult problems. Some effort at least has been made in the first part (see RoboRosewater) but the other two seem almost intractable, given the combinatorial explosion of possibilities in deckbuilding, the incredibly nonlinear interactions between cards (Splinter Twin is a 1/10 card, Pestermite is maybe 3/10, Splinter Twin + Pestermite literally wins the game on the spot).
YeGoblynQueenne 6y ago
>> The trickiest bit is deciding exactly what you want your AI to optimise for -- If you want a game where "anyone can win", you end up with coin toss. If you want a game where "skill is everything", you can accidentally end up with "who can click fastest". In Magic: the Gathering this balancing act is achieved by a set of constantly changing "environments" (sets of cards) that are updated every few months, when a new set of cards is published. Sets "rotate" in and out of the various environments and there's a few where it's legal to play with all cards ever printed (excluding some that are banned or restricted for being OP- Over Powered). When a new set rotates (its cards become legal) in a given environment, there will be a reshuffle of the balance of power between already-established decks, that now acquire new cards or lose cards they needed, and some new strategies become available resulting in new decks being designed. Eventually the dust settles and the "tier one" decks (the ones that win the most) are found. Then tournament play in particular becomes a rock-paper-scissors affair, until the next update. In some of the environments were all cards from all sets are legal (again, minus the ones in the banned or restricted lists) games indeed often come down to a coin toss: the player who goes first wins (in one turn, through some OP combo, typically). I suspect that a perfectly balanced game is impossible to have and would probably be boring even if it was possible to have it. An element of creativity with constantly updated design parts, like in M:tG is probably the best one can do.
- vvanders 6y ago
  Perfectly balanced games are plentiful, take rock, paper, scissors as an example. What makes those games interesting is the concept of Yomi[1] and learning to read an opponent. There's a decent amount of literature in the fighting game space and a bunch of other genres overlap as well. I remain pretty skeptical of ML being a primary tool here since a lot of Yomi is psychological and not necessarily an optimization problem. [1] https://www.gamasutra.com/blogs/CalebCompton/20190211/336365/Know_Thy_Enemy_Designing_for_Yomi_in_Games.php https://www.gamasutra.com/blogs/CalebCompton/20190211/336365...
  - philipov 6y ago
    "Nerf rock" - Scissors

ericjang 6y ago

Interesting! Is there some kind of meta-objective for "AI optimal" that could have replaced the 6 or 7 iterations you did with human R&D? For instance, if you had real human playtesters interacting with the prototypes, is there some signal you could extract to measure that it's "good"?

CJefferson 6y ago
The problem is AIs are very good at optimising what you asked them for, rather than what you meant to ask for, and figuring out what you want is super hard :) As a simple example: * Start by optimising "players can always do something on their turn" -- but that just ends up with everyone always having exactly one thing they can do (no choice). * So then say "give players more things to do each turn" -- but then they end up being able to do everything every turn (the game gives them too much 'money' (still not really a choice) * OK, so we want to force players to make a choice -- so we say "No, give players as much choice as possible, but make sure if they choose an option it blocks off others (in practice, make as many sets of maximal tasks as possible)" -- but then the AI will make sure every turn every player can do (for example) exactly 3 out of 6 things (any 3), and make sure no matter how well or badly they play they still always get to choose 3 from 6, so the game doesn't really progress, or vary. So, what we want is choice, but also variability, and progress, and players to feel like they are effecting the game, but also don't let one player run away too early, but also don't make it just "feel random who wins", etc.
- ericjang 6y ago
  Got it. I do research in reinforcement learning and I can sympathize with the difficulty here - in my experience, even something as simple as "I want to balance two objectives: 1) the agent should get a high score and 2) the agent should try to make as few decisions as possible" tends to result in the agent doing neither of those things well"

adkadskhj 6y ago

Personally, i'd like to see AI used for "simple" things, not often tied to the core gameplay loop itself. Ie conceptually if you could tie the right gameplay components to an RNG function then the user could get new/unexpected experiences for a longer time. The problem is RNG often makes crap outputs. I'd be curious to see ML take this problem. I imagine the examples are limitless, but one that comes to mind might be procedural worlds. Rather than tuning procedural worlds around seed values, what would it look like if we try to get an ML to craft the worlds? Procedurally generated worlds often have that feeling. Things don't make sense, etc. I imagine ML could make some far more diverse patterns that are both interesting and fun. I'd like to see this pattern of "informed RNG" in a lot of things. Skyrim with NPC patterns to feel a bit more authentic. Etc. I'm less interested in the classic AI dreams inside games, as it seems far out and less gamey. But our current games with informed RNG? That sounds.. neat.

Jasper_ 6y ago
ML is a system that takes a large data set, and an error function, and finds a generated output that minimizes the loss. What data set and error function are you proposing for "RNG"?
- matsemann 6y ago
  That's a wrong simplification of ML. Take RL for instance. And the parent already explained a concept: generating maps. It could still have an RNG as the base (noise function over something), but then use ML to place elements based on existing human-made maps.
- malka 6y ago
  You feed it rng created world that were curated by humans.
jonbaer 6y ago
I think you are right. There are SO many small mini games inside of nearly all RTS games which AI has yet to actually conquer, I will give another simple example, given a random map (and resources) identify chokepoints. As a human this is a huge clue on how to go about a game strategy (policy) but I have yet to find an AI do this (or optimize that part). From what I gather most time is spent on actually optimizing random build orders and modifying them along the way.

Quarrelsome 6y ago

I feel like some of these questions are a bit backwards. Isn't it better to give the model an arena to play within and try to make all of the options _viable_? I.E. Balance is maintained when the average of the weapons the gladiators choose to take into the arena are diverse and have less bearing on the win outcome? This means that rather focusing on what is merely broken one is focusing on both ends, what is broken and what is useless, attempting to push both towards an average of usability. Personally it astonishes me that groups like WOTC (who produce MTG) don't appear to be using such models given the quantity of mistakes they seem to continue to make in game design.

meheleventyone 6y ago
For something like a deck building game, if all decks are viable (competitive) then there may as well be no deck building component. In fact a game like that is probably a pretty muddy brown color in terms of card variety. That would adversely affect the business model of selling new cards as well. Desirable qualities in a game in terms of how it is balanced are way more complex than just every option being viable and leading to an even win rate against a similarly skilled opponent.
vladTheInhaler 6y ago
Attempting to optimize the card generation process, or deck building, or just play itself seem like unbelievably difficult problems. Some effort at least has been made in the first part (see RoboRosewater) but the other two seem almost intractable, given the combinatorial explosion of possibilities in deckbuilding, the incredibly nonlinear interactions between cards (Splinter Twin is a 1/10 card, Pestermite is maybe 3/10, Splinter Twin + Pestermite literally wins the game on the spot).

YeGoblynQueenne 6y ago

>> The trickiest bit is deciding exactly what you want your AI to optimise for -- If you want a game where "anyone can win", you end up with coin toss. If you want a game where "skill is everything", you can accidentally end up with "who can click fastest". In Magic: the Gathering this balancing act is achieved by a set of constantly changing "environments" (sets of cards) that are updated every few months, when a new set of cards is published. Sets "rotate" in and out of the various environments and there's a few where it's legal to play with all cards ever printed (excluding some that are banned or restricted for being OP- Over Powered). When a new set rotates (its cards become legal) in a given environment, there will be a reshuffle of the balance of power between already-established decks, that now acquire new cards or lose cards they needed, and some new strategies become available resulting in new decks being designed. Eventually the dust settles and the "tier one" decks (the ones that win the most) are found. Then tournament play in particular becomes a rock-paper-scissors affair, until the next update. In some of the environments were all cards from all sets are legal (again, minus the ones in the banned or restricted lists) games indeed often come down to a coin toss: the player who goes first wins (in one turn, through some OP combo, typically). I suspect that a perfectly balanced game is impossible to have and would probably be boring even if it was possible to have it. An element of creativity with constantly updated design parts, like in M:tG is probably the best one can do.

vvanders 6y ago
Perfectly balanced games are plentiful, take rock, paper, scissors as an example. What makes those games interesting is the concept of Yomi[1] and learning to read an opponent. There's a decent amount of literature in the fighting game space and a bunch of other genres overlap as well. I remain pretty skeptical of ML being a primary tool here since a lot of Yomi is psychological and not necessarily an optimization problem. [1] https://www.gamasutra.com/blogs/CalebCompton/20190211/336365/Know_Thy_Enemy_Designing_for_Yomi_in_Games.php https://www.gamasutra.com/blogs/CalebCompton/20190211/336365...
- philipov 6y ago
  "Nerf rock" - Scissors