2 ms·
Yes, many people have tried this. The inherent deference of the language models makes them ineffective DMs - they're unable to effectively enforce rules or crea
by duskwuff 1y ago
Yes, many people have tried this. The inherent deference of the language models makes them ineffective DMs - they're unable to effectively enforce rules or create meaningful challenges for the player. Even if that were somehow avoided, though, they're still inherently unable to do any serious planning or worldbuilding, or to conceal information from the player - at best, you might get the experience of playing with a frazzled, unprepared DM who's making everything up on the fly.
- daxfohl 1y agoMakes sense. In my experience, LLMs are not very good at maintaining deterministic state machines internally either. Our team did an experiment trying use an LLM to guide users through a four step signup flow, and gave it very explicit instructions what order to collect information, rules for validating it, under what conditions a step could be skipped, how many times each step could be retried. The rules were highly simplified, just as a proof of concept. Still, it would get lost half the time. So yeah, I have a hard time imagining it would do any better managing the full state of some RPG it made up
- duskwuff 1y agoI'm reminded of the infamous vending machine benchmark: https://arxiv.org/abs/2502.15840 https://arxiv.org/abs/2502.15840 in which some LLMs simply failed to manage the basic task of keeping a vending machine stocked with products to sell, and some went completely off the rails, e.g. by threatening their (simulated) supplier with "ABSOLUTE FINAL ULTIMATE TOTAL NUCLEAR LEGAL INTERVENTION".