14 ms·

LLMs in games would absolutely be amazing. Weird and bizarre conversations and quests would make the games more interesting not less. Imagine a generative world

LLMs in games would absolutely be amazing. Weird and bizarre conversations and quests would make the games more interesting not less. Imagine a generative world like Morrowind with actually generative dialogue too.

fnordpiglet 4y ago
I think some interesting stuff can happen using a generative AI to create content then using semantic parsers on the generated text to convert it into the structure of the story and visa versa. I think it might make single player games less hollow for sure, and make rogue like games as rich as human written.
- fisjy 4y ago
  This is my thinking also. LLMs as chatbots are the least interesting possibility imho. I can foresee LLMs being used as an 'OS' for NPCs, where context from the environment/player is updated and appropriate actions are made by the NPCs according to the 'persona'. I expect this would lead to much richer emergent behaviour, such as the player donning beggar clothes and walking into a city eliciting a very different response than walking around with a sword in one hand. As a gamedev, this is an area I'm actively investigating.
  - fnordpiglet 4y ago
    I think it can serve for more than that though. If you construct a prompt that provides a structured enough output you can then ask it to “tell me a story about X” then ask if clarifying details about objects in X, and from that generate an entire story arc with characters etc and load it into a game runtime using the structured format of the responses (I.e., create places and things and npcs). As the player interacts you can have it continue to expand the story assembled. Further you can integrate the visual AIs to make characters and places based on the language AIs descriptions. Then of course you make characters chat bots - but I think the whole thing generalizes to a lot more - the statefulness of the chat can also be used as the statefulness of an unfolding story.
orbital-decay 4y ago
People are trying to build games based on LLM generation since 2019. Problem is, the models are too flexible for this purpose - you can easily break the world rules and make the game look like nonsense. And games are all about comprehensible rules and properly placed restrictions. So these attempts are currently concentrated on two things: 1. How to make the model play by the rules. 2. How to translate the model output into something concrete in terms of game mechanics.
- imtringued 4y ago
  ChatGPT is an instruction fine tuned version of GPT3 with RLHF. You could train an AI game master that tries to maintain coherence within a story or game world. The only problem is fine tuning the GM for each individual game and making it reproducible for debugging.

fnordpiglet 4y ago

I think some interesting stuff can happen using a generative AI to create content then using semantic parsers on the generated text to convert it into the structure of the story and visa versa. I think it might make single player games less hollow for sure, and make rogue like games as rich as human written.

fisjy 4y ago
This is my thinking also. LLMs as chatbots are the least interesting possibility imho. I can foresee LLMs being used as an 'OS' for NPCs, where context from the environment/player is updated and appropriate actions are made by the NPCs according to the 'persona'. I expect this would lead to much richer emergent behaviour, such as the player donning beggar clothes and walking into a city eliciting a very different response than walking around with a sword in one hand. As a gamedev, this is an area I'm actively investigating.
- fnordpiglet 4y ago
  I think it can serve for more than that though. If you construct a prompt that provides a structured enough output you can then ask it to “tell me a story about X” then ask if clarifying details about objects in X, and from that generate an entire story arc with characters etc and load it into a game runtime using the structured format of the responses (I.e., create places and things and npcs). As the player interacts you can have it continue to expand the story assembled. Further you can integrate the visual AIs to make characters and places based on the language AIs descriptions. Then of course you make characters chat bots - but I think the whole thing generalizes to a lot more - the statefulness of the chat can also be used as the statefulness of an unfolding story.

orbital-decay 4y ago

People are trying to build games based on LLM generation since 2019. Problem is, the models are too flexible for this purpose - you can easily break the world rules and make the game look like nonsense. And games are all about comprehensible rules and properly placed restrictions. So these attempts are currently concentrated on two things: 1. How to make the model play by the rules. 2. How to translate the model output into something concrete in terms of game mechanics.

imtringued 4y ago
ChatGPT is an instruction fine tuned version of GPT3 with RLHF. You could train an AI game master that tries to maintain coherence within a story or game world. The only problem is fine tuning the GM for each individual game and making it reproducible for debugging.