4 ms·

Has anyone tried having them DM text games? Seems like they could create a dungeon and DM a game pretty well. It should be easier than playing, I'd think. Thoug

Has anyone tried having them DM text games? Seems like they could create a dungeon and DM a game pretty well. It should be easier than playing, I'd think. Though I'd be curious how good they are at making fun games or whether they struggle with that.

thrance 1y ago
Had a friend try to build an AI-driven text RPG. Told me it was rather bland and unimaginative, and gets boring fast.
- deleted 1y ago
  [deleted]
duskwuff 1y ago
Yes, many people have tried this. The inherent deference of the language models makes them ineffective DMs - they're unable to effectively enforce rules or create meaningful challenges for the player. Even if that were somehow avoided, though, they're still inherently unable to do any serious planning or worldbuilding, or to conceal information from the player - at best, you might get the experience of playing with a frazzled, unprepared DM who's making everything up on the fly.
- daxfohl 1y ago
  Makes sense. In my experience, LLMs are not very good at maintaining deterministic state machines internally either. Our team did an experiment trying use an LLM to guide users through a four step signup flow, and gave it very explicit instructions what order to collect information, rules for validating it, under what conditions a step could be skipped, how many times each step could be retried. The rules were highly simplified, just as a proof of concept. Still, it would get lost half the time. So yeah, I have a hard time imagining it would do any better managing the full state of some RPG it made up
  - duskwuff 1y ago
    I'm reminded of the infamous vending machine benchmark: https://arxiv.org/abs/2502.15840 https://arxiv.org/abs/2502.15840 in which some LLMs simply failed to manage the basic task of keeping a vending machine stocked with products to sell, and some went completely off the rails, e.g. by threatening their (simulated) supplier with "ABSOLUTE FINAL ULTIMATE TOTAL NUCLEAR LEGAL INTERVENTION".
ryan-c 1y ago
I've actually have a pretty good experience with gemini-2.5-pro doing this, but I was focused primarily on role-play and collaborative story telling. Definitely some issues, but I enjoyed it.

thrance 1y ago

Had a friend try to build an AI-driven text RPG. Told me it was rather bland and unimaginative, and gets boring fast.

deleted 1y ago
[deleted]

duskwuff 1y ago

Yes, many people have tried this. The inherent deference of the language models makes them ineffective DMs - they're unable to effectively enforce rules or create meaningful challenges for the player. Even if that were somehow avoided, though, they're still inherently unable to do any serious planning or worldbuilding, or to conceal information from the player - at best, you might get the experience of playing with a frazzled, unprepared DM who's making everything up on the fly.

daxfohl 1y ago
Makes sense. In my experience, LLMs are not very good at maintaining deterministic state machines internally either. Our team did an experiment trying use an LLM to guide users through a four step signup flow, and gave it very explicit instructions what order to collect information, rules for validating it, under what conditions a step could be skipped, how many times each step could be retried. The rules were highly simplified, just as a proof of concept. Still, it would get lost half the time. So yeah, I have a hard time imagining it would do any better managing the full state of some RPG it made up
- duskwuff 1y ago
  I'm reminded of the infamous vending machine benchmark: https://arxiv.org/abs/2502.15840 https://arxiv.org/abs/2502.15840 in which some LLMs simply failed to manage the basic task of keeping a vending machine stocked with products to sell, and some went completely off the rails, e.g. by threatening their (simulated) supplier with "ABSOLUTE FINAL ULTIMATE TOTAL NUCLEAR LEGAL INTERVENTION".

ryan-c 1y ago

I've actually have a pretty good experience with gemini-2.5-pro doing this, but I was focused primarily on role-play and collaborative story telling. Definitely some issues, but I enjoyed it.