20 ms·

I’ve been thinking about how traditional game AI can be improved by generative models. One of the biggest problems with games like Civ is that the AI strategy i

I’ve been thinking about how traditional game AI can be improved by generative models. One of the biggest problems with games like Civ is that the AI strategy is predictable - especially if you’ve played a few dozen hours. LLMs with some decent harnesses could build up unpredictable - but internally consistent - strategies per each new game you play. This is close to a proof of concept for those improvements.

peytonshields 1y ago
Absolutely! Max and I were huge Civ fans and always tried to make the game AI deviate from its programmed strategy. We also believe you can get some really interesting story arcs by adjusting parameters like temperature and how context is presented. Some of the things you'll notice in the game is we have a no-holds barred approach -- you can fully modify system prompts and adjust how the LLM interprets the state of the world.
tatjam 1y ago
I wonder how could you keep the LLM from going bonkers as the game progresses? I have a feeling it's possibly better to re-create the prompts after some time, and have the LLM work more like one of those "reasoning models" with the game as something it can interact with. Otherwise you run into the risk of "TOTAL NUCLEAR FINANCIAL LEGAL DESTRUCTION" ;)
- peytonshields 1y ago
  This is something we've been working on and are planning to release a "decision" update to the game which should allow for multi-step, configurable options to choose if the LLM actually gets to contribute to the current world / chat. There's a lot of trial and error involved and we're all ears, if you have ideas we'd love to hear them! We actively monitor our discord https://discord.com/invite/theinterface https://discord.com/invite/theinterface
- Aerroon 1y ago
  Probably still performs more rationally than the lategame AI in Civ.
yawnxyz 1y ago
even being able to scheme with / backstab leaders, and they would "understand" all that's happened (and acts accordingly) would be so fun
bob1029 1y ago
From a player perspective, oftentimes the best AI systems are the most trivial ones. You can get really far with an agent that is allowed to cheat. It's a hell of a lot easier to build and troubleshoot a model that manipulates the amount of in-game resources received per unit time than it is to implement actual strategic intelligence.
- mvdtnz 1y ago
  I play strategy games a lot and cheating AI can be fun to play against at first, but the more you learn a game the more cheating AI sucks. When you're new to the game it just feels like you're playing against a good player, but you soon learn that what they are achieving isn't possible with the resources available. Once you hit that realisation it can be fun to beat them as a challenge but it never feels like a fair game.
  - mh- 1y ago
    Agreed, this is an instant turn-off for me when I realize this in e.g. an RTS game. Red Alert or C&C come to mind on higher difficulty, can't remember which.
    - snerbles 1y ago
      IIRC the RA1 skirmish mode AIs always had perfect information and resource multipliers based on difficulty. RA2 did it a little differently with "virtual ore purifiers" added for the high difficulty AIs. I'm sure a similar thing was done for the Tiberian Dawn campaign and the Tiberian Sun multiplayer/skirmish AIs. OpenRA's bots are a bit more clever, and also don't need to magically see into fog-of-war.
      - mh- 1y ago
        I never played much of RA2, but played hundreds of hours of RA1 skirmish. Must have been that. Thanks for the insight! Skirmish was a blast- I'd turtle until I had the enormous battleships (cruisers?) that could fire onto land. Loads of fun when I was like 12.
ralusek 1y ago
Definitely the case. That being said, I think it would be hard, at least in the immediate future, to translate the concept of difficulty to a universal LLM for a bespoke/specific game. I assume most game AIs are tuned by hand to feel fair for a given difficulty level...but if you just give an LLM some new game, explain the rules and what resources/abilities it has available to it, you're stuck with adding some addendum to the tune of "and you're meant to represent an entity of 'medium' difficulty." For very well established games, it might have a sense of how given actions might fall into a skill-level hierarchy, but not for anything new. Fine tuned LLMs though with actual experience with the game, maybe?
- AlecSchueler 1y ago
  You can just have the old AI doing stuff like resource management while the LLM handles the diplomacy and it wouldy be a lot better already.
dawnerd 1y ago
As someone that plays those games pretty heavily: I’d rather not have LLMs take over game AI like that. If I want different gameplay I’d play online. We don’t need to bog down already heavy games with LLMs.
- thrown-0825 1y ago
  Odd take, the stale dialogue and static quests of most rpgs could certainly benefit from llm enhancements
  - krainboltgreene 1y ago
    Nah, they’ll still be stale. Many people play RPGs that haven’t changed in 30 years, so static isn’t an issue either.
    - thrown-0825 1y ago
      And many people don’t, there are already skyrim mods for this so your point doesnt really hold water.
      - nottorp 1y ago
        However, no one has ever praised the Elder Scrolls storyline. They win by the sheer quantity and by giving you a lot of subsystems to play with. So LLM generated quest text probably feels it belongs here. It wouldn't, for example, in something with the Witcher 3 story quality.
        thrown-0825 1y ago
        except the hundreds of hours of youtube videos discussing the story of the franchise
      - krainboltgreene 1y ago
        It’s so funny to reference a game that has like 12 editions and is on every platform including a refrigerator and think “this game is missing something” By the way there are LLM dialog mods for Skyrim and everyone thinks they’re a joke because they suck.
dahauns 1y ago
I can't help it, the first thought that came to mind was "Huh...talk about sheer senseless brute force." Why use a Large Language Model on something as clearly defined in scope as a game instead of a model designed and trained for the task/ruleset? Sure, there's the argument of not having to train that model, but OTOH, "decent harnesses" does some very heavy lifting there...
- throwanem 1y ago
  I think it's a compelling argument. You would need a large dataset of completed games on which to train, which may have something to do with why the games considered solved by AI are also among those where exist a very rich and heavily annotated corpus of completed games in algebraic notation.
  - dahauns 1y ago
    Of course - but in practice you won't be aiming towards fully a "solved" game or that kind of player skill for something like Civ - and even so, I severely doubt an LLM realistically can hope to even get in the vicinity unless the aforementioned "harness" does something similar anywayas part of its heavy lifting I mentioned.
viccis 1y ago
I'd love to see LLM based versions procedural tasks like "radiant quests" which are generally disappointing, though I've heard it discussed before and the real challenge is keeping it from going way off the rails. The other challenge I think you'll run into in general is that there's a huge knee jerk reaction against any use of LLMs or other popular types of gen AI in games in places like Reddit or Bluesky.

peytonshields 1y ago

Absolutely! Max and I were huge Civ fans and always tried to make the game AI deviate from its programmed strategy. We also believe you can get some really interesting story arcs by adjusting parameters like temperature and how context is presented. Some of the things you'll notice in the game is we have a no-holds barred approach -- you can fully modify system prompts and adjust how the LLM interprets the state of the world.

tatjam 1y ago

I wonder how could you keep the LLM from going bonkers as the game progresses? I have a feeling it's possibly better to re-create the prompts after some time, and have the LLM work more like one of those "reasoning models" with the game as something it can interact with. Otherwise you run into the risk of "TOTAL NUCLEAR FINANCIAL LEGAL DESTRUCTION" ;)

peytonshields 1y ago
This is something we've been working on and are planning to release a "decision" update to the game which should allow for multi-step, configurable options to choose if the LLM actually gets to contribute to the current world / chat. There's a lot of trial and error involved and we're all ears, if you have ideas we'd love to hear them! We actively monitor our discord https://discord.com/invite/theinterface https://discord.com/invite/theinterface
Aerroon 1y ago
Probably still performs more rationally than the lategame AI in Civ.

yawnxyz 1y ago

even being able to scheme with / backstab leaders, and they would "understand" all that's happened (and acts accordingly) would be so fun

bob1029 1y ago

From a player perspective, oftentimes the best AI systems are the most trivial ones. You can get really far with an agent that is allowed to cheat. It's a hell of a lot easier to build and troubleshoot a model that manipulates the amount of in-game resources received per unit time than it is to implement actual strategic intelligence.

mvdtnz 1y ago
I play strategy games a lot and cheating AI can be fun to play against at first, but the more you learn a game the more cheating AI sucks. When you're new to the game it just feels like you're playing against a good player, but you soon learn that what they are achieving isn't possible with the resources available. Once you hit that realisation it can be fun to beat them as a challenge but it never feels like a fair game.
- mh- 1y ago
  Agreed, this is an instant turn-off for me when I realize this in e.g. an RTS game. Red Alert or C&C come to mind on higher difficulty, can't remember which.
  - snerbles 1y ago
    IIRC the RA1 skirmish mode AIs always had perfect information and resource multipliers based on difficulty. RA2 did it a little differently with "virtual ore purifiers" added for the high difficulty AIs. I'm sure a similar thing was done for the Tiberian Dawn campaign and the Tiberian Sun multiplayer/skirmish AIs. OpenRA's bots are a bit more clever, and also don't need to magically see into fog-of-war.
    - mh- 1y ago
      I never played much of RA2, but played hundreds of hours of RA1 skirmish. Must have been that. Thanks for the insight! Skirmish was a blast- I'd turtle until I had the enormous battleships (cruisers?) that could fire onto land. Loads of fun when I was like 12.

ralusek 1y ago

Definitely the case. That being said, I think it would be hard, at least in the immediate future, to translate the concept of difficulty to a universal LLM for a bespoke/specific game. I assume most game AIs are tuned by hand to feel fair for a given difficulty level...but if you just give an LLM some new game, explain the rules and what resources/abilities it has available to it, you're stuck with adding some addendum to the tune of "and you're meant to represent an entity of 'medium' difficulty." For very well established games, it might have a sense of how given actions might fall into a skill-level hierarchy, but not for anything new. Fine tuned LLMs though with actual experience with the game, maybe?

AlecSchueler 1y ago
You can just have the old AI doing stuff like resource management while the LLM handles the diplomacy and it wouldy be a lot better already.

dawnerd 1y ago

As someone that plays those games pretty heavily: I’d rather not have LLMs take over game AI like that. If I want different gameplay I’d play online. We don’t need to bog down already heavy games with LLMs.

thrown-0825 1y ago
Odd take, the stale dialogue and static quests of most rpgs could certainly benefit from llm enhancements
- krainboltgreene 1y ago
  Nah, they’ll still be stale. Many people play RPGs that haven’t changed in 30 years, so static isn’t an issue either.
  - thrown-0825 1y ago
    And many people don’t, there are already skyrim mods for this so your point doesnt really hold water.
    - nottorp 1y ago
      However, no one has ever praised the Elder Scrolls storyline. They win by the sheer quantity and by giving you a lot of subsystems to play with. So LLM generated quest text probably feels it belongs here. It wouldn't, for example, in something with the Witcher 3 story quality.
      - thrown-0825 1y ago
        except the hundreds of hours of youtube videos discussing the story of the franchise
    - krainboltgreene 1y ago
      It’s so funny to reference a game that has like 12 editions and is on every platform including a refrigerator and think “this game is missing something” By the way there are LLM dialog mods for Skyrim and everyone thinks they’re a joke because they suck.

dahauns 1y ago

I can't help it, the first thought that came to mind was "Huh...talk about sheer senseless brute force." Why use a Large Language Model on something as clearly defined in scope as a game instead of a model designed and trained for the task/ruleset? Sure, there's the argument of not having to train that model, but OTOH, "decent harnesses" does some very heavy lifting there...

throwanem 1y ago
I think it's a compelling argument. You would need a large dataset of completed games on which to train, which may have something to do with why the games considered solved by AI are also among those where exist a very rich and heavily annotated corpus of completed games in algebraic notation.
- dahauns 1y ago
  Of course - but in practice you won't be aiming towards fully a "solved" game or that kind of player skill for something like Civ - and even so, I severely doubt an LLM realistically can hope to even get in the vicinity unless the aforementioned "harness" does something similar anywayas part of its heavy lifting I mentioned.

viccis 1y ago

I'd love to see LLM based versions procedural tasks like "radiant quests" which are generally disappointing, though I've heard it discussed before and the real challenge is keeping it from going way off the rails. The other challenge I think you'll run into in general is that there's a huge knee jerk reaction against any use of LLMs or other popular types of gen AI in games in places like Reddit or Bluesky.