3 ms·

How much framerate are you willing to give up so that NPCs don't repeat themselves? The elephant in the room with LLMs in games is that the users GPU is usually

How much framerate are you willing to give up so that NPCs don't repeat themselves? The elephant in the room with LLMs in games is that the users GPU is usually already being maxed out by pushing pixels, so any amount of compute allocated to running AI models instead is going to have a very tangible cost. They could be offloaded to the cloud instead but that brings its own set of issues (ongoing upkeep costs, finite capacity, servers will inevitably shut down...)

squigz 3y ago
> The elephant in the room ... For now. LLMs are still relatively new. The technology will continue to be optimized, and we'll be shipping tiny LLMs along with games soon enough. (Aren't companies already running LLMs on phones?)
- krainboltgreene 3y ago
  The housing market will only go up, it's always gone up.
Workaccount2 3y ago
I can only imagine nvidia/amd pouncing on the chance to sell companion "AI Cards" that enable LLM NPCs in games. If the tech is good (and insane progress in tiny local models points to "yes"), then I can easily imagine a large contingent of gamers forking over another $600 for a second GPU. It might not even be necessary though, similar to how physixs (sp?) cards died out quick in the early 00's, the models might become slim enough and the GPU's fast enough that the extra load is easily manageable.
- jsheard 3y ago
  The vast majority of games need to run on consoles to be financially viable, anything that requires an entire second GPU is a complete non-starter. The only way it's going to work is if the models get small enough to run on a negligible slice of GPU or CPU time, with a reasonable memory footprint (<1GB), on whatever the current console hardware happens to be.
  - Workaccount2 3y ago
    I see your point, but it hasn't really stopped novel features from coming to PC in the past. The PC market is about 50% larger than the console market too.
  - gaganyaan 3y ago
    Console manufacturers would salivate at the thought of new hardware that makes people want to buy a new console instead of playing the ones they've already got. Once someone gets it working reasonably on PC, it will quickly spread to a new generation of consoles with new hardware that makes people want to upgrade.

squigz 3y ago

> The elephant in the room ... For now. LLMs are still relatively new. The technology will continue to be optimized, and we'll be shipping tiny LLMs along with games soon enough. (Aren't companies already running LLMs on phones?)

krainboltgreene 3y ago
The housing market will only go up, it's always gone up.

Workaccount2 3y ago

I can only imagine nvidia/amd pouncing on the chance to sell companion "AI Cards" that enable LLM NPCs in games. If the tech is good (and insane progress in tiny local models points to "yes"), then I can easily imagine a large contingent of gamers forking over another $600 for a second GPU. It might not even be necessary though, similar to how physixs (sp?) cards died out quick in the early 00's, the models might become slim enough and the GPU's fast enough that the extra load is easily manageable.

jsheard 3y ago
The vast majority of games need to run on consoles to be financially viable, anything that requires an entire second GPU is a complete non-starter. The only way it's going to work is if the models get small enough to run on a negligible slice of GPU or CPU time, with a reasonable memory footprint (<1GB), on whatever the current console hardware happens to be.
- Workaccount2 3y ago
  I see your point, but it hasn't really stopped novel features from coming to PC in the past. The PC market is about 50% larger than the console market too.
- gaganyaan 3y ago
  Console manufacturers would salivate at the thought of new hardware that makes people want to buy a new console instead of playing the ones they've already got. Once someone gets it working reasonably on PC, it will quickly spread to a new generation of consoles with new hardware that makes people want to upgrade.