4 ms·

Honestly I'm really excited about this. I've always dreamed of full blown sandbox games with extremely advanced NPCs (which the current LLMs can already kinda e

Honestly I'm really excited about this. I've always dreamed of full blown sandbox games with extremely advanced NPCs (which the current LLMs can already kinda emulate), but on the bigger scale. In just a few decades this will finally be made into proper games. I can't wait.

aleph_minus_one 2y ago
> Honestly I'm really excited about this. I've always dreamed of full blown sandbox games with extremely advanced NPCs (which the current LLMs can already kinda emulate), but on the bigger scale. I don't believe that you want this. Even really good players don't have a chance against super-advanced NPCs (think how chess grandmasters have barely any chance against modern chess programs running on a fast computer). You will rather get crushed. What you likely want is NPC that "behave more human-like (or animal-like)" - whatever this means.
- Tiberium 2y ago
  Oh, I should've clarified - I don't want to fight against them, I just want to watch and sometimes interfere to see how the agents react ;) A god game like WorldBox/Galimulator, if you will. Or observer mode in tons of games like almost all Paradox ones.
  - aleph_minus_one 2y ago
    > I just want to watch and sometimes interfere to see how the agents react ;) Even there, I am not sure whether if the AI bcomes too advanced, it will be of interest for many players (you might of course nevertheless be interested): Here, the relevant comparison is to watching (the past) games of AlphaGo against Go grandmasters, where even the highly qualified commentators had insane difficulties explaining AlphaGo's moves because many of the moves were so different from the strategy of any Go game before. The commentors could just accept and grasp that these highly advanced moves did crush the Go grandmaster opponents. In my opinion, the "typical" sandbox game player wants to watch something that he still can "somewhat" grasp.
  - com2kid 2y ago
    I'm working on something similar, https://www.generativestorytelling.ai/tinyllmtown/index.html https://www.generativestorytelling.ai/tinyllmtown/index.html a small town where all NPCs are simulated using a small LLM. They react to everything the hero does, which means no more killing a dragon and having no one even mention it. Once I release it, I'll have it simulate 4 hours every 2 hours or so of real time, and visitors can vote on what quest the hero undertakes next. The simulation is simpler, I am aiming to keep everything to a size that can run on a local GPU with a small model. Right now you can just watch the NPCs try to figure out love triangles, hide their drinking problems, complain about carrots, and celebrate when the hero saves the town yet again.
    - aspenmayer 2y ago
      This description reminded me of Dwarf Fortress. You might look into how the AI in it works to see if it gives you any ideas about how emergent behaviors can interact?
  - int_19h 2y ago
    What I want is for someone to remake this, but with modern AI and a vast interactive environment typical of a modern open world game: https://en.wikipedia.org/wiki/Majesty:_The_Fantasy_Kingdom_Sim https://en.wikipedia.org/wiki/Majesty:_The_Fantasy_Kingdom_S...
- kgeist 2y ago
  >Even really good players don't have a chance against super-advanced NPCs I guess you can make them dumber by randomly switching to hardcoded behavioral trees (without modern AI) once in a while so that they made mistakes (while feeling pretty intelligent overall), and the player would then have a chance to outsmart them.
ted_bunny 2y ago
Game designers have barely scratched the surface of NPC modeling even as it is. Rimworld is considered deep but it's nothing close to it.
- aleksiy123 2y ago
  Yeah I think there is a lot of potential here. Especially in city building games etc.
- ileonichwiesz 2y ago
  Rimworld is heavily inspired by Dwarf Fortress, so if you’re looking for more complex examples you don’t have to look far. DF is pretty granular with the physical and mental states of its characters - to the point that a character might lose a specific toe or get depressed about their situation - but of course it’s still a video game, not a scientific simulation of an AI society.
jsemrau 2y ago
I think it can be quite interesting especially if you consider different character types (in Anthropic lingo this "personality"). The only problem right now is that using a proprietary LLM is incredibly expensive. Therefore having a local LLM might be the best option. Unfortunately, these are still not on the same level as their larger brethren. [1] https://jdsemrau.substack.com/p/evaluating-consciousness-and-reasoning https://jdsemrau.substack.com/p/evaluating-consciousness-and...
drusepth 2y ago
> I've always dreamed of full blown sandbox games with extremely advanced NPCs (which the current LLMs can already kinda emulate) The future of gaming is going to get weird fast with all this new tech, and there are a lot of new mechanics emerging that just weren't possible before LLMs, generative AI, etc. At our game studio we're already building medium-scale sandbox games where NPCs form memories, opinions, problems (that translate to quests), and have a continuous "internal monologue" that uses all of this context plus sensory input from their place in a 3D world to constantly decide what actions they should be performing in the game world. A player can decide to chat with an NPC about their time at a lake nearby and then see that NPC deciding to go visit the lake the next day. A paper last year ("Generative Agents: Interactive Simulacra of Human Behavior", [0]) is a really good sneak-peek into the kind of evolving sandboxes LLMs (with memory and decisionmaking) enable. There's a lot of cool stuff that happens in that "game", but one anecdote I always think back to is this: in a conversation between two NPCs, one happens to mention they have a birthday coming up to the other; and that other NPC then goes around town talking to other NPCs about a birthday party, and _those_ NPCs mention the party to other NPCs, and so on until the party happened and most of the NPCs in town arrived on time. None of it was scripted, but you very quickly start to see emergent behavior from these sorts of "flocks" of agents as soon as you add persistence and decision-making. And there are other interesting layers games can add for even more kinds of emergent behavior; that's what we're exploring at our studio [1], and I've seen lots of other studios pop up this last year to try their hand at it too. I'm optimistic and excited about the future of gaming (or, at least some new genres). It should be fun. :) [0] https://arxiv.org/abs/2304.03442 https://arxiv.org/abs/2304.03442 [1] https://www.chromagolem.com/games https://www.chromagolem.com/games

aleph_minus_one 2y ago

> Honestly I'm really excited about this. I've always dreamed of full blown sandbox games with extremely advanced NPCs (which the current LLMs can already kinda emulate), but on the bigger scale. I don't believe that you want this. Even really good players don't have a chance against super-advanced NPCs (think how chess grandmasters have barely any chance against modern chess programs running on a fast computer). You will rather get crushed. What you likely want is NPC that "behave more human-like (or animal-like)" - whatever this means.

Tiberium 2y ago
Oh, I should've clarified - I don't want to fight against them, I just want to watch and sometimes interfere to see how the agents react ;) A god game like WorldBox/Galimulator, if you will. Or observer mode in tons of games like almost all Paradox ones.
- aleph_minus_one 2y ago
  > I just want to watch and sometimes interfere to see how the agents react ;) Even there, I am not sure whether if the AI bcomes too advanced, it will be of interest for many players (you might of course nevertheless be interested): Here, the relevant comparison is to watching (the past) games of AlphaGo against Go grandmasters, where even the highly qualified commentators had insane difficulties explaining AlphaGo's moves because many of the moves were so different from the strategy of any Go game before. The commentors could just accept and grasp that these highly advanced moves did crush the Go grandmaster opponents. In my opinion, the "typical" sandbox game player wants to watch something that he still can "somewhat" grasp.
- com2kid 2y ago
  I'm working on something similar, https://www.generativestorytelling.ai/tinyllmtown/index.html https://www.generativestorytelling.ai/tinyllmtown/index.html a small town where all NPCs are simulated using a small LLM. They react to everything the hero does, which means no more killing a dragon and having no one even mention it. Once I release it, I'll have it simulate 4 hours every 2 hours or so of real time, and visitors can vote on what quest the hero undertakes next. The simulation is simpler, I am aiming to keep everything to a size that can run on a local GPU with a small model. Right now you can just watch the NPCs try to figure out love triangles, hide their drinking problems, complain about carrots, and celebrate when the hero saves the town yet again.
  - aspenmayer 2y ago
    This description reminded me of Dwarf Fortress. You might look into how the AI in it works to see if it gives you any ideas about how emergent behaviors can interact?
- int_19h 2y ago
  What I want is for someone to remake this, but with modern AI and a vast interactive environment typical of a modern open world game: https://en.wikipedia.org/wiki/Majesty:_The_Fantasy_Kingdom_Sim https://en.wikipedia.org/wiki/Majesty:_The_Fantasy_Kingdom_S...
kgeist 2y ago
>Even really good players don't have a chance against super-advanced NPCs I guess you can make them dumber by randomly switching to hardcoded behavioral trees (without modern AI) once in a while so that they made mistakes (while feeling pretty intelligent overall), and the player would then have a chance to outsmart them.

ted_bunny 2y ago

Game designers have barely scratched the surface of NPC modeling even as it is. Rimworld is considered deep but it's nothing close to it.

aleksiy123 2y ago
Yeah I think there is a lot of potential here. Especially in city building games etc.
ileonichwiesz 2y ago
Rimworld is heavily inspired by Dwarf Fortress, so if you’re looking for more complex examples you don’t have to look far. DF is pretty granular with the physical and mental states of its characters - to the point that a character might lose a specific toe or get depressed about their situation - but of course it’s still a video game, not a scientific simulation of an AI society.

jsemrau 2y ago

I think it can be quite interesting especially if you consider different character types (in Anthropic lingo this "personality"). The only problem right now is that using a proprietary LLM is incredibly expensive. Therefore having a local LLM might be the best option. Unfortunately, these are still not on the same level as their larger brethren. [1] https://jdsemrau.substack.com/p/evaluating-consciousness-and-reasoning https://jdsemrau.substack.com/p/evaluating-consciousness-and...

drusepth 2y ago

> I've always dreamed of full blown sandbox games with extremely advanced NPCs (which the current LLMs can already kinda emulate) The future of gaming is going to get weird fast with all this new tech, and there are a lot of new mechanics emerging that just weren't possible before LLMs, generative AI, etc. At our game studio we're already building medium-scale sandbox games where NPCs form memories, opinions, problems (that translate to quests), and have a continuous "internal monologue" that uses all of this context plus sensory input from their place in a 3D world to constantly decide what actions they should be performing in the game world. A player can decide to chat with an NPC about their time at a lake nearby and then see that NPC deciding to go visit the lake the next day. A paper last year ("Generative Agents: Interactive Simulacra of Human Behavior", [0]) is a really good sneak-peek into the kind of evolving sandboxes LLMs (with memory and decisionmaking) enable. There's a lot of cool stuff that happens in that "game", but one anecdote I always think back to is this: in a conversation between two NPCs, one happens to mention they have a birthday coming up to the other; and that other NPC then goes around town talking to other NPCs about a birthday party, and _those_ NPCs mention the party to other NPCs, and so on until the party happened and most of the NPCs in town arrived on time. None of it was scripted, but you very quickly start to see emergent behavior from these sorts of "flocks" of agents as soon as you add persistence and decision-making. And there are other interesting layers games can add for even more kinds of emergent behavior; that's what we're exploring at our studio [1], and I've seen lots of other studios pop up this last year to try their hand at it too. I'm optimistic and excited about the future of gaming (or, at least some new genres). It should be fun. :) [0] https://arxiv.org/abs/2304.03442 https://arxiv.org/abs/2304.03442 [1] https://www.chromagolem.com/games https://www.chromagolem.com/games