5 ms·

With how much NVidia is developing AI-workload accelerating hardware, I expect this will cost maybe few dozen dollars and train in few hours within next few yea

With how much NVidia is developing AI-workload accelerating hardware, I expect this will cost maybe few dozen dollars and train in few hours within next few years. What I think will be interesting is when commodity hardware can run cheap inference from very capable, specialized models. Pretty sure it will spawn a new golden age of AI-powered desktop applications. For example, video game space has already been trying to create AI-powered NPCs, world generation and story-telling (e.g. Inworld AI).

up2isomorphism 2y ago
> For example, video game space has already been trying to create AI-powered NPCs, world generation and story-telling (e.g. Inworld AI). To me this is a downside compared to the NPC generated by humans, since that’s the only reason I would like to read them.
- MisterBastahrd 2y ago
  You don't even necessarily need to have them coming up with valid speech. Simply giving out random quests and rewards would keep people running on a loot treadmill for most open world multiplayer games.
  - LeanderK 2y ago
    I think at first even background NPCs that don't give quests and rewards would be nice. Sort of give everyone a character and let them just babble. It breaks the immersion to hear the same phrases repeatedly. You can still handcraft every important quest and interaction to achieve high fidelity, but I would like random NPCs you bump into to not just repeat things all the time.
    - swatcoder 2y ago
      You're more likely to see studios (who want lots of more varied content) use generative AI in the studio, where they might generate and review it before release. Letting generators run free on the client sets up different kinds of immersion-breaking, where NPC's hallucinate misleading details about the story/world, can be tricked into reciting off-topic absurdities or age-rating violations, etc. AAA studios can't afford the embarrassment of that and smaller designers with pride of craft won't see their signature come through the art in it. Surely, some designers will figure out ways to make it work great for some specific idea, but it's not the best way to use the technology in most cases.
      - fragmede 2y ago
        Sounds like there's a ton of money to be had to someone that can solve those problems though. A version of fallout where I can go up to people/things and just start talking to them, instead of picking from a list of things to say? :shutupandtakemymoney:
  - swatcoder 2y ago
    To make this work, you need your LLM-based AI to outperform any other form of generating quests and rewards -- and that performance is measured on things like player enjoyment, game/story progression, exploitability, client system requirements or server operating costs, etc and most of those things are very hard to constrain or optimize for with an LLM right now. While the costs are hidden from end users and are going down quickly, good LLM's remain very expensive to run and very hard to keep on track compared to other options.
    - MisterBastahrd 2y ago
      You're trying to big brain something that isn't that complicated or special. I never said that every interaction with an NPC must necessarily require AI to create a new quest and reward from scratch. Players aren't that savvy. You just generate a pool of quests and assign them to NPCs in the instance. Players who are addicted to loot treadmills will stay in the game and pay for boosts and other rewards as long as there is content to engage them. This more than pays for the AI service.
  - techjamie 2y ago
    You could maybe pull this off in a game like Borderlands where the loot is basically just the same dozen guns but with different numbers and effects. But as is, the LLM text isn't going to be much different than a sufficiently large AdLib system. I think there is value to be gained in having LLMs as part of the development process, maybe even the game itself, but I think conventional methods are about as sufficient for quests.
- bick_nyers 2y ago
  What if the LLM that a specific NPC utilizes was handcrafted/fine-tuned by a human?
- chongli 2y ago
  Yeah. I have played a bunch of the roguelike Caves of Qud [1] and it has both hand-written text and procedurally generated text. The former is quite interesting and relevant to both gameplay and plot. The latter is mostly uninteresting and irrelevant, though it does work as "filler." This is similar to how procedurally-generated grass can give a more natural look to a hill than you'd get with tiles (which are incredibly easy to spot unless a ton of work is put into hiding the seams and repeating patterns). I still long for the day when we can have procedurally-generated stories and quests that are actually interesting to play through. I have no idea how that is going to work though! [1] https://www.cavesofqud.com https://www.cavesofqud.com
  - FooBarBizBazz 2y ago
    > I still long for the day when we can have procedurally-generated stories and quests that are actually interesting to play through. It's an interesting artistic and technical challenge to investigate, absolutely. I hope people work on it, and I'm sure they already are. However, let me also offer a counterpoint with something much simpler: coffee. It is possible to fully automate the process of making an espresso drink. You can buy cheap versions of these machines that will sit on your kitchen counter; the quality of the drinks so produced is not the highest, but I suspect it's entirely possible to build a machine that would actually make high-quality drinks, perhaps with even more precision than a human can. Yet the opposite trend has prevailed over the last few decades. Not long ago, Americans drank drip coffee (hence the approximation thereof as the Americano). This is an easy process to make in large batches with minimal labor. It is almost fully automatic, without any "automation". And certainly this still exists all over the place. But we now also have coffee shops everywhere -- witness the explosion of Starbucks out of its home in Seattle -- where people, baristas, individually make espresso drinks for customers. This is immensely popular. Why are tons of people now employed in this way, and why hasn't it been automated? Is it really just a matter of cost? I don't think so. I think the knowledge that another person took the trouble to do something for you is in fact part of the product. Consider the practice of drawing designs, like leaves or hearts, in the milk foam. This is entirely unnecessary, from a taste perspective. If you put a lid on it, as is commonly done, you will not even see it. But it carries a message -- that somebody gave enough of a damn to do it. (We could then analyze how this phenomenon gets watered down and eventually destroys itself when it tries to turn itself into a mass-produced fast-food franchise staffed by underpaid/exploited proletarians, but that would take this post in another direction.) My point is, I think something similar happens in video games. The very knowledge that another person was involved is important. You are receiving communication from this person. It does something to synchronize, partially, your mind with theirs. And this is a thing we appreciate. If there's no person on the other end, why should we care? This general phenomenon of paradoxiciality can be a bad thing. It can "feel like" socializing, without producing the actual, real, thick social networks that give us rich lives. There can be an exploitative and even druglike dynamic. It isn't entirely a good class of phenomena. But in small doses I think it's good, useful, important. I think it's a core part of what makes art valuable. And of course, it applies to games. This is why, I believe, we will not fully automate the telling of stories, or the making of lattes. The person doing it is important.
forrestthewoods 2y ago
> For example, video game space has already been trying to create AI-powered NPCs, world generation and story-telling (e.g. Inworld AI). Current AI isn't even close to good enough for video game NPCs and related. We're several breakthroughs away from that being possible at any cost. Those breakthroughs might happen in 3 years, or they might not happen in 10. Hard to predict.
- Tiberium 2y ago
  Are you sure? Models like Claude 3.5 Sonnet are both good at writing and instructions, as long as you set some guardrails for the model, they can be great NPCs.
  - forrestthewoods 2y ago
    > Are you sure? Absolutely. Current LLMs have insufficient world state. Imagine a game like Stardew Valley. It's got a town with 30 NPCs or some such. They all have personalities and the player builds a relationship with them over time. Current LLMs can't do that. They hallucinate waaaaaaay too much. You can't reliably define and evolve relationships. Amongst many other short comings. I'm super pro AI and use ChatGPT all the time for programming. So I'm not being an AI hater. But I am a gamedev and I can say that what exists today simply isn't good enough.
    - Tiberium 2y ago
      But you're saying that you want a single model to handle all NPCs and the whole world. Of course this isn't possible currently. But using a separate model with separate context for each character is. Also, if you use ChatGPT for programming, try Claude 3.5 Sonnet - it's really better than GPT-4o for programming.
      - forrestthewoods 2y ago
        > But you're saying that you want a single model to handle all NPCs and the whole world. No, I did not say that at all. I didn't specify how the LLMs may or may not be structure. I'm saying that current LLMs - and yes I've used Claude 3.5 Sonnet - are insufficient. There is no existence proof that they're sufficient. LLMs are great. They aren't great enough for video game NPC. Not yet. Further innovation is needed. You're free to disagree. I can't prove a dispositive. But there is no working example.
talldayo 2y ago
> I expect this will cost maybe few dozen dollars and train in few hours within next few years. I wouldn't count on it. Nvidia's been cleaning up shop, but their best option for expanding right now is through parallelization (bigger clusters, basically). Now that Blackwell is on TSMC, Nvidia is alongside Apple waiting for new and denser nodes to upgrade to. A real "generational leap" in training cost is going to require some form of efficiency gain that we're not seeing right now. It's possible that Nvidia has something up their sleeve, but I'm not holding my breath. > What I think will be interesting is when commodity hardware can run cheap inference from very capable, specialized models. What's funny is, you basically already can. The problem is becoming integration, and in the case of video games, giving the AI a meaningful role to fill. With today's finest technology, you can enjoy an AI-generated roguelike that is nigh-incomprehensible: https://store.steampowered.com/app/1889620/AI_Roguelite/ https://store.steampowered.com/app/1889620/AI_Roguelite/ As time goes on, I really think developers are just going to not use AI for video games. Maybe I'm missing the "minecraft moment" for procedurally-generated stories here, but the sort of constraints needed to tell a story of create an interactive experience don't exist within LLMs. It's a stochastic nightmare of potential softlocks, contradictions or outright offensive requests. The majority of places I've seen AI applied today isn't for content creation, but instead automated moderation.
swatcoder 2y ago
> For example, video game space has already been trying to create AI-powered NPCs, world generation and story-telling (e.g. Inworld AI). This'll be a niche for a long, long time. Games are generally carefully crafted to deliver a specific mechanical and/or narrative experience. A world populated by LLM/etc bots or content is one choice of what that experience might be, but it's not going to be a very satisfying one for many game designers -- especially given the current/near state of the technology. There will be games and experiments that explore it, for sure, but the vast majority of games just don't have any need for it.
- ImHereToVote 2y ago
  I don't have needs. I have wants. You see, I don't need. I want.
- headcanon 2y ago
  I do think there is a big opportunity for widely supported hardware-accelerated matrix algebra in games. Currently most of that is geared towards graphics (naturally) but being able to easily encode arbitrary models and have them run on-device would open up a lot of opportunities for games (like deep simulation) that weren't possible before. Its currently possible of course but requires custom tooling and (relatively) niche hardware like a high-end graphics card. I see the development energy around LLMs as a way to open up support for that.
- jetrink 2y ago
  Those don't have to be mutually exclusive though. To take AI out of it, think of those murder mystery parties where actors interact with attendees. Actors have roles to play and things they must do to move the story forward, but they improvise their dialog when talking with the players and sometimes each other. Or if you've ever played D&D, you have experienced talking with NPCs that are controlled by your DM. I think video game AI could be a lot like that, where NPCs use natural language instead of rigid dialog trees, but otherwise, they behave a lot like they do today.
  - thatguymike 2y ago
robbomacrae 2y ago
Whilst I agree with the reservations of the other replies I think you were implying in the future and I'm sure the LLM's will be more trustworthy and up to the task at some point. What I would really like to see now is all the new TTS models being used more widespread. There are still so many games that have text only output. My kid love Alba: A wildlife Adventure but the eldest still isn't quite ready to read all the text so I have to sit with them reading out all the lines. If anyone has a way of applying universal mods / accessibility features to existing games I'd love to see someone solve this and happy to help with the TTS!
charlescurt123 2y ago
I imagine we could do this now but not the way you think. have a human created story and text as a guideline. With that have genAI make the text per stage, you would get different statements every time and would stay on track. Would be interesting to play a game where all players say the same information in slightly different ways every single playthrough.
HPsquared 2y ago
Similar scaling to genome sequencing. First genome was a huge undertaking, now routine after a few Moore-esque cycles.

up2isomorphism 2y ago

> For example, video game space has already been trying to create AI-powered NPCs, world generation and story-telling (e.g. Inworld AI). To me this is a downside compared to the NPC generated by humans, since that’s the only reason I would like to read them.

MisterBastahrd 2y ago
You don't even necessarily need to have them coming up with valid speech. Simply giving out random quests and rewards would keep people running on a loot treadmill for most open world multiplayer games.
- LeanderK 2y ago
  I think at first even background NPCs that don't give quests and rewards would be nice. Sort of give everyone a character and let them just babble. It breaks the immersion to hear the same phrases repeatedly. You can still handcraft every important quest and interaction to achieve high fidelity, but I would like random NPCs you bump into to not just repeat things all the time.
  - swatcoder 2y ago
    You're more likely to see studios (who want lots of more varied content) use generative AI in the studio, where they might generate and review it before release. Letting generators run free on the client sets up different kinds of immersion-breaking, where NPC's hallucinate misleading details about the story/world, can be tricked into reciting off-topic absurdities or age-rating violations, etc. AAA studios can't afford the embarrassment of that and smaller designers with pride of craft won't see their signature come through the art in it. Surely, some designers will figure out ways to make it work great for some specific idea, but it's not the best way to use the technology in most cases.
    - fragmede 2y ago
      Sounds like there's a ton of money to be had to someone that can solve those problems though. A version of fallout where I can go up to people/things and just start talking to them, instead of picking from a list of things to say? :shutupandtakemymoney:
- swatcoder 2y ago
  To make this work, you need your LLM-based AI to outperform any other form of generating quests and rewards -- and that performance is measured on things like player enjoyment, game/story progression, exploitability, client system requirements or server operating costs, etc and most of those things are very hard to constrain or optimize for with an LLM right now. While the costs are hidden from end users and are going down quickly, good LLM's remain very expensive to run and very hard to keep on track compared to other options.
  - MisterBastahrd 2y ago
    You're trying to big brain something that isn't that complicated or special. I never said that every interaction with an NPC must necessarily require AI to create a new quest and reward from scratch. Players aren't that savvy. You just generate a pool of quests and assign them to NPCs in the instance. Players who are addicted to loot treadmills will stay in the game and pay for boosts and other rewards as long as there is content to engage them. This more than pays for the AI service.
- techjamie 2y ago
  You could maybe pull this off in a game like Borderlands where the loot is basically just the same dozen guns but with different numbers and effects. But as is, the LLM text isn't going to be much different than a sufficiently large AdLib system. I think there is value to be gained in having LLMs as part of the development process, maybe even the game itself, but I think conventional methods are about as sufficient for quests.
bick_nyers 2y ago
What if the LLM that a specific NPC utilizes was handcrafted/fine-tuned by a human?
chongli 2y ago
Yeah. I have played a bunch of the roguelike Caves of Qud [1] and it has both hand-written text and procedurally generated text. The former is quite interesting and relevant to both gameplay and plot. The latter is mostly uninteresting and irrelevant, though it does work as "filler." This is similar to how procedurally-generated grass can give a more natural look to a hill than you'd get with tiles (which are incredibly easy to spot unless a ton of work is put into hiding the seams and repeating patterns). I still long for the day when we can have procedurally-generated stories and quests that are actually interesting to play through. I have no idea how that is going to work though! [1] https://www.cavesofqud.com https://www.cavesofqud.com
- FooBarBizBazz 2y ago
  > I still long for the day when we can have procedurally-generated stories and quests that are actually interesting to play through. It's an interesting artistic and technical challenge to investigate, absolutely. I hope people work on it, and I'm sure they already are. However, let me also offer a counterpoint with something much simpler: coffee. It is possible to fully automate the process of making an espresso drink. You can buy cheap versions of these machines that will sit on your kitchen counter; the quality of the drinks so produced is not the highest, but I suspect it's entirely possible to build a machine that would actually make high-quality drinks, perhaps with even more precision than a human can. Yet the opposite trend has prevailed over the last few decades. Not long ago, Americans drank drip coffee (hence the approximation thereof as the Americano). This is an easy process to make in large batches with minimal labor. It is almost fully automatic, without any "automation". And certainly this still exists all over the place. But we now also have coffee shops everywhere -- witness the explosion of Starbucks out of its home in Seattle -- where people, baristas, individually make espresso drinks for customers. This is immensely popular. Why are tons of people now employed in this way, and why hasn't it been automated? Is it really just a matter of cost? I don't think so. I think the knowledge that another person took the trouble to do something for you is in fact part of the product. Consider the practice of drawing designs, like leaves or hearts, in the milk foam. This is entirely unnecessary, from a taste perspective. If you put a lid on it, as is commonly done, you will not even see it. But it carries a message -- that somebody gave enough of a damn to do it. (We could then analyze how this phenomenon gets watered down and eventually destroys itself when it tries to turn itself into a mass-produced fast-food franchise staffed by underpaid/exploited proletarians, but that would take this post in another direction.) My point is, I think something similar happens in video games. The very knowledge that another person was involved is important. You are receiving communication from this person. It does something to synchronize, partially, your mind with theirs. And this is a thing we appreciate. If there's no person on the other end, why should we care? This general phenomenon of paradoxiciality can be a bad thing. It can "feel like" socializing, without producing the actual, real, thick social networks that give us rich lives. There can be an exploitative and even druglike dynamic. It isn't entirely a good class of phenomena. But in small doses I think it's good, useful, important. I think it's a core part of what makes art valuable. And of course, it applies to games. This is why, I believe, we will not fully automate the telling of stories, or the making of lattes. The person doing it is important.

forrestthewoods 2y ago

> For example, video game space has already been trying to create AI-powered NPCs, world generation and story-telling (e.g. Inworld AI). Current AI isn't even close to good enough for video game NPCs and related. We're several breakthroughs away from that being possible at any cost. Those breakthroughs might happen in 3 years, or they might not happen in 10. Hard to predict.

Tiberium 2y ago
Are you sure? Models like Claude 3.5 Sonnet are both good at writing and instructions, as long as you set some guardrails for the model, they can be great NPCs.
- forrestthewoods 2y ago
  > Are you sure? Absolutely. Current LLMs have insufficient world state. Imagine a game like Stardew Valley. It's got a town with 30 NPCs or some such. They all have personalities and the player builds a relationship with them over time. Current LLMs can't do that. They hallucinate waaaaaaay too much. You can't reliably define and evolve relationships. Amongst many other short comings. I'm super pro AI and use ChatGPT all the time for programming. So I'm not being an AI hater. But I am a gamedev and I can say that what exists today simply isn't good enough.
  - Tiberium 2y ago
    But you're saying that you want a single model to handle all NPCs and the whole world. Of course this isn't possible currently. But using a separate model with separate context for each character is. Also, if you use ChatGPT for programming, try Claude 3.5 Sonnet - it's really better than GPT-4o for programming.
    - forrestthewoods 2y ago
      > But you're saying that you want a single model to handle all NPCs and the whole world. No, I did not say that at all. I didn't specify how the LLMs may or may not be structure. I'm saying that current LLMs - and yes I've used Claude 3.5 Sonnet - are insufficient. There is no existence proof that they're sufficient. LLMs are great. They aren't great enough for video game NPC. Not yet. Further innovation is needed. You're free to disagree. I can't prove a dispositive. But there is no working example.

talldayo 2y ago

> I expect this will cost maybe few dozen dollars and train in few hours within next few years. I wouldn't count on it. Nvidia's been cleaning up shop, but their best option for expanding right now is through parallelization (bigger clusters, basically). Now that Blackwell is on TSMC, Nvidia is alongside Apple waiting for new and denser nodes to upgrade to. A real "generational leap" in training cost is going to require some form of efficiency gain that we're not seeing right now. It's possible that Nvidia has something up their sleeve, but I'm not holding my breath. > What I think will be interesting is when commodity hardware can run cheap inference from very capable, specialized models. What's funny is, you basically already can. The problem is becoming integration, and in the case of video games, giving the AI a meaningful role to fill. With today's finest technology, you can enjoy an AI-generated roguelike that is nigh-incomprehensible: https://store.steampowered.com/app/1889620/AI_Roguelite/ https://store.steampowered.com/app/1889620/AI_Roguelite/ As time goes on, I really think developers are just going to not use AI for video games. Maybe I'm missing the "minecraft moment" for procedurally-generated stories here, but the sort of constraints needed to tell a story of create an interactive experience don't exist within LLMs. It's a stochastic nightmare of potential softlocks, contradictions or outright offensive requests. The majority of places I've seen AI applied today isn't for content creation, but instead automated moderation.

swatcoder 2y ago

> For example, video game space has already been trying to create AI-powered NPCs, world generation and story-telling (e.g. Inworld AI). This'll be a niche for a long, long time. Games are generally carefully crafted to deliver a specific mechanical and/or narrative experience. A world populated by LLM/etc bots or content is one choice of what that experience might be, but it's not going to be a very satisfying one for many game designers -- especially given the current/near state of the technology. There will be games and experiments that explore it, for sure, but the vast majority of games just don't have any need for it.

ImHereToVote 2y ago
I don't have needs. I have wants. You see, I don't need. I want.
headcanon 2y ago
I do think there is a big opportunity for widely supported hardware-accelerated matrix algebra in games. Currently most of that is geared towards graphics (naturally) but being able to easily encode arbitrary models and have them run on-device would open up a lot of opportunities for games (like deep simulation) that weren't possible before. Its currently possible of course but requires custom tooling and (relatively) niche hardware like a high-end graphics card. I see the development energy around LLMs as a way to open up support for that.
jetrink 2y ago
Those don't have to be mutually exclusive though. To take AI out of it, think of those murder mystery parties where actors interact with attendees. Actors have roles to play and things they must do to move the story forward, but they improvise their dialog when talking with the players and sometimes each other. Or if you've ever played D&D, you have experienced talking with NPCs that are controlled by your DM. I think video game AI could be a lot like that, where NPCs use natural language instead of rigid dialog trees, but otherwise, they behave a lot like they do today.
- thatguymike 2y ago

robbomacrae 2y ago

Whilst I agree with the reservations of the other replies I think you were implying in the future and I'm sure the LLM's will be more trustworthy and up to the task at some point. What I would really like to see now is all the new TTS models being used more widespread. There are still so many games that have text only output. My kid love Alba: A wildlife Adventure but the eldest still isn't quite ready to read all the text so I have to sit with them reading out all the lines. If anyone has a way of applying universal mods / accessibility features to existing games I'd love to see someone solve this and happy to help with the TTS!

charlescurt123 2y ago

I imagine we could do this now but not the way you think. have a human created story and text as a guideline. With that have genAI make the text per stage, you would get different statements every time and would stay on track. Would be interesting to play a game where all players say the same information in slightly different ways every single playthrough.

HPsquared 2y ago

Similar scaling to genome sequencing. First genome was a huge undertaking, now routine after a few Moore-esque cycles.