7 ms·

TBH I haven't seen a single use of LLMs in games that wasn't better served by traditional algorithms beyond less repetitive NPC interactions. Maybe once they ge

TBH I haven't seen a single use of LLMs in games that wasn't better served by traditional algorithms beyond less repetitive NPC interactions. Maybe once they get good enough to create usable rigged and textured meshes with enough control to work in-game? They can't create a story on the fly that's reliable enough to be a compelling accompaniment to a coherent game plot. Maps and such don't seem to need anything beyond what current procedural algorithms provide, and they're still working with premade assets— the implementations I've seen can't even reliably place static meshes on the ground in believable positions. And as far as NPCs go— how far does that actually go? It's pure novelty worth far less than an hour of time. Let's even say you get a guided plot progression worded on the fly using an LLM, is that even as good, let alone better, than a dialog tree put together by a professional writer? This Civ idea at least seems like a new approach to some extent, but it still seems to conceptually not add much. Even if not, learning that it doesn't it's still worthwhile. But almost universally these ideas seem to be either buzzwordy solutions in search of problems, or a cheaper-than-people source of creativity with some serious quality tradeoffs and still require far too much developer wrangling to actually save money. I'm a tech artist so I'm a bit biased towards the value of human creativity, but also likely the primary demographic for LLM tools in game dev. I am, so far, not compelled.

MichaelZuo 2y ago
Nobody will know for sure until a big budget game is actually released with a serious effort behind its NPCs.
- chefandy 2y ago
  I can't see anything that Gen AI NPCs would add unless maybe you're talking about a Sims kind of game where the interactions are the point, and they don't have to adhere to a defined progression. Other than that, it's a chat bot. We already have chatbots and having them in the context of a video game doesn't seem like it would add anything revolutionary to that product. And would that fundamentally stand a chance of being as compelling to socially-focused role-playing gamers as online games? This is my field so I'm always looking for the angle that new tech will take. I still rank this lower than VR— with all of its problems— for potential to significantly change player interactions. Tooling to make games is a different story, but for actual use in games? I don't see it yet.
  - mywittyname 2y ago
    Sandbox games are probably where they will shine. Imagine being able to play Minecraft, and tell a prompt to generate a world that resembles Tatooine, or a vampire-themed mansion. Expectations are lower with sandbox games, so there's no risk of breaking immersion like would happen with an LLM Elder Scrolls game when someone tricks in NPC into solving problems in python. Granted, I'm certain there will be copyrights issues associated with this capability, which is why I don't think it will be established game companies who first take a crack at this approach.
    - chefandy 2y ago
      The problem is what it takes to implement that. I've seen companies currently trying to do exactly that, and their demos go like this "ok, give me a prompt for the environment" and if they're lucky, they can cherry pick some stuff the crowd says and if they're not, they sheepishly ask for a prompt that would visit indicate one of 5 environment types they've worked on and include several of the dozen premade textured meshes they've made, and in reality you've got a really really expensive procedural map with asset placement that's worse than if it was done using traditional semi-pre-baked approaches. A deceptive amount of work goes into the nitty gritty of making environments, and even with all of the incredible tooling that's around now, we are not even close to automating that. It's worth noting that my alma mater has game environment art degree programs. Unless you're making these things, you can't easily see how much finesse and artistic sensibility it takes to make beautiful compositions with complementary lighting and nice atmospheric progression. It's not just that nobody has really given it a go— it's really difficult. When you have tooling that uses AI controlled by an artist that knows these things, that's one thing. When they need to make great results every time so players keep coming back? That's a very different task. I've never met anyone that thought it was remotely currently feasible without lacking knowledge of generative AI, game development, or both. Automating the tools so a smaller workforce can make more worlds and more possibilities? We're already there— but it's a very large leap to remove the human creative and technical intermediaries.
      - MichaelZuo 2y ago
JohnMakin 2y ago
It's been posted in-depth a few times across this forum to varying degrees by game developers - I was initially very excited about the implementation of LLM's in NPC interactions, until I read some of these posts. The gist of it was - the thing that makes a game fundamentally a game is its constraints. LLM-based NPC's fundamentally break these constraints in a way that is not testable or predictable by the developer and will inevitably destroy the gameplay experience (at least with current technology).
- chefandy 2y ago
  Yeah, same. Epic's Matrix demo implemented it and even without a plot, the interactions were so heavily guided that the distinction was pointless. So you can find out what that NPCs spous's name is and their favorite color. It's that neat? Sure it's neat. It's it going to make it a better game? Probably less than hiring another good writer to make NPC dialog. To be truly useful, I think they would have to be able to affect the world in meaningful ways that worked with the game plot, and again, when you clamp that down as much as you'd need to to still have a plot, you're looking at a fancy decision tree.

MichaelZuo 2y ago

Nobody will know for sure until a big budget game is actually released with a serious effort behind its NPCs.

chefandy 2y ago
I can't see anything that Gen AI NPCs would add unless maybe you're talking about a Sims kind of game where the interactions are the point, and they don't have to adhere to a defined progression. Other than that, it's a chat bot. We already have chatbots and having them in the context of a video game doesn't seem like it would add anything revolutionary to that product. And would that fundamentally stand a chance of being as compelling to socially-focused role-playing gamers as online games? This is my field so I'm always looking for the angle that new tech will take. I still rank this lower than VR— with all of its problems— for potential to significantly change player interactions. Tooling to make games is a different story, but for actual use in games? I don't see it yet.
- mywittyname 2y ago
  Sandbox games are probably where they will shine. Imagine being able to play Minecraft, and tell a prompt to generate a world that resembles Tatooine, or a vampire-themed mansion. Expectations are lower with sandbox games, so there's no risk of breaking immersion like would happen with an LLM Elder Scrolls game when someone tricks in NPC into solving problems in python. Granted, I'm certain there will be copyrights issues associated with this capability, which is why I don't think it will be established game companies who first take a crack at this approach.
  - chefandy 2y ago
    The problem is what it takes to implement that. I've seen companies currently trying to do exactly that, and their demos go like this "ok, give me a prompt for the environment" and if they're lucky, they can cherry pick some stuff the crowd says and if they're not, they sheepishly ask for a prompt that would visit indicate one of 5 environment types they've worked on and include several of the dozen premade textured meshes they've made, and in reality you've got a really really expensive procedural map with asset placement that's worse than if it was done using traditional semi-pre-baked approaches. A deceptive amount of work goes into the nitty gritty of making environments, and even with all of the incredible tooling that's around now, we are not even close to automating that. It's worth noting that my alma mater has game environment art degree programs. Unless you're making these things, you can't easily see how much finesse and artistic sensibility it takes to make beautiful compositions with complementary lighting and nice atmospheric progression. It's not just that nobody has really given it a go— it's really difficult. When you have tooling that uses AI controlled by an artist that knows these things, that's one thing. When they need to make great results every time so players keep coming back? That's a very different task. I've never met anyone that thought it was remotely currently feasible without lacking knowledge of generative AI, game development, or both. Automating the tools so a smaller workforce can make more worlds and more possibilities? We're already there— but it's a very large leap to remove the human creative and technical intermediaries.
    - MichaelZuo 2y ago

JohnMakin 2y ago

It's been posted in-depth a few times across this forum to varying degrees by game developers - I was initially very excited about the implementation of LLM's in NPC interactions, until I read some of these posts. The gist of it was - the thing that makes a game fundamentally a game is its constraints. LLM-based NPC's fundamentally break these constraints in a way that is not testable or predictable by the developer and will inevitably destroy the gameplay experience (at least with current technology).

chefandy 2y ago
Yeah, same. Epic's Matrix demo implemented it and even without a plot, the interactions were so heavily guided that the distinction was pointless. So you can find out what that NPCs spous's name is and their favorite color. It's that neat? Sure it's neat. It's it going to make it a better game? Probably less than hiring another good writer to make NPC dialog. To be truly useful, I think they would have to be able to affect the world in meaningful ways that worked with the game plot, and again, when you clamp that down as much as you'd need to to still have a plot, you're looking at a fancy decision tree.