11 ms·

LLM based agents as Dungeon Masters

140 points by utiiiD 2y ago

Aeolun 2y ago
Based on my experiences with this. Keeping context relevant is the hardest part. Your human GM will remember that funny remark from the session 3 years ago, but the LLM?
- worthless-trash 2y ago
  I'd love to see some human & machine co-op for DM. That'd be super helpful.
  - 2099miles 2y ago
    Now this is solid, like an ai enhanced Evernote, but for organization, not generation/creation.
    - Sanzig 2y ago
      I've been meaning to take a crack at doing this - I'd like to use Whisper to transcribe session recordings and then have an LLM build out a campaign wiki. I enjoy being a DM but note taking and organization is my least favourite part.
  - cdiamand 2y ago
    I've used LLM's in this capacity, and it's awesome. It quickly becomes a crutch.
- ratedgene 2y ago
  I think we just haven't built complex enough architectures to allow this. I have a few ideas boiling that would help facilitate long term context understanding and recall.
- cloudking 2y ago
  Maybe this is a solution https://www.letta.com/ https://www.letta.com/
  - potatoman22 2y ago
    A context-based retrieval/memory system would be cool, too. Pull in relevant events based on the group's location and NPCs they're with.
- mirages 2y ago
  Also for that same module, is it able to integrate it in a broader campaign. I expanded the lore and why the players got here with extra quests, fluff and mystery and they only reached said dungeon at 2nd session only. Also the way a PC died due to a rat bite that could only be cured by magic (5e of this module is ironically harsher than the 3.5) as he was losing max HP every 24H. As the players expressed dislike with that character the owning player was willing to go for a new character and I had planned a big drama scene where the players exit in emergency the dungeon and try to rally the nearet village where there's a priestessd and happens that I could match the rolls so he would die in the priestess hands. But I made the mistake to allow them to spend the night to get their 3rd level which would have allowed the druid to get his cure illnesses spell botching the set up. Happens that I decided to exploit the fact that all other character the previous days made an offering to Beshaba to curse that character and force rolls during night (and try to find a way to make the new character come into play). How it unfolded : - they were sleeping and it was the bard's turn for watch - the bard was a follower of Tharizdun and was about to sacrifice the dieing character as said character was agonizing - the new character was a Kara-turian enforcer that was chasing a yokai (he didn't knew) that used other people as disguise, made it happen that the yokai choose the dieing character "by coincidence" appearance - the yokai being chased opens a portal while hiding to the dungeon - the new character litteraly pops through a Wild Magic portal that pushed away the bard and tries to bash the old character as the old character runes on his skin glow a last time as he dies and repels everybody - made the new character pop naked as I wasn't paying attention to what the player was saying when he was listing his inventory - I let them handle the big mess
- Kuinox 2y ago
  Neuro-sama, an AI VTuber streamer, manages to do this. There are topics brought along multiple streams and manages to remember details of previous streams.
  - CuriousCosmic 2y ago
    With soemthing like Neuro-sama I wouldn't doubt that Vedal is doing a lot of liftwork behind the scenes to keep sufficient context available to Neuro-sama's LLM.
    - Philpax 2y ago
      And with the right kind of engineering, that could be automated too. I don't think the results would be as good as they are now - Vedal is bringing his own perspective into it - but once you have a proof of concept, you can improve it.
- lemoncookiechip 2y ago
  It's that: Spacial awareness, struggling to maintain the personas (and often talking for the player/s), difficulty improvising, pacing issues, lack of initiative, and just an over reliance on prompting to keep going correctly.
2099miles 2y ago
This is so interesting that people want this. This is beyond the guy saying “people don’t like making music so we made ai so it’s easier and faster to just prompt it to make music” . Like in my experience the best part of d&d is the dungeon master who knows their friends making a story and details specifically for fun and the fun of the group. And the interactions between players and the dm is where I derived the most enjoyment, automating that sounds insane and peak confusion/self defeating.
- airstrike 2y ago
  Some people don't have friends they can play with :sadge:
  - foobarian 2y ago
    This. At home it's me and my one kid, and having an additional party as a DM would be priceless.
- Melomomololo 2y ago
  Just that finding a good DM is hard
- dingnuts 2y ago
  they think they want it -- that doesn't mean they will actually like it if they get it
- prettyblocks 2y ago
  I think it's more about the challenge/curiosity of creating the implementation. I'm not sure these AI driven text-RPGs will find long term audiences.
  - pmichaud 2y ago
    They are a step on the road to fully responsive rpg worlds though, which don’t have to be text based.
noodletheworld 2y ago
Alas, I was desperately keen to read transcripts of actual gameplay and all we got was user surveys? Was there a structured combat system as sort of this or was it a talking only kind of thing? Who knows? Trying to get empirical evidence that an LLM can even remotely plausibly work as a DM with no social aspect and (as far as I can tell, reading the paper) no programmed system of stats, rolls, combat etc. seems impressive. …but was it that? Or just a DnD themed chatGPT conversation? Or a text adventure with an LLM describing each “room”? Did the LLM actually have agency to make decisions and perform tasks, or was it not an agent at all, just a conversation? Without a transcript, all you can really say for sure is yeah, I agree; more research is needed.
empath75 2y ago
The main problem with doing this on chatgpt is that it is _relentlessly_ cheerful and it's almost impossible for it to produce any kind of interpersonal drama or threat or danger. I tried to get it to run space opera about a diplomatic mission to a newly recontacted planet, full of Game of Thrones-style intrigue and plotting and scheming, and it was like playing with a cheerily optimistic new UN intern that completely believes in the power of compromise and diplomacy to solve all problems. It was always like: "And so the two of you hashed out your differences over a glass of Dubranian Forblik, and peace reigned forever and ever after." That's true of how it does role playing games and any kind of fiction at all -- it always wants to tie a neat bow on it at the end of every few paragraphs with a moral and everything. You also have to ask it to use python to dice roll for anything you want to potentially fail on, or it will always make everything you do succeed wonderfully. I will say this -- the advanced audio mode for D&D is amazing and it really does act out the characters and it will occasionally slip into different voices or even add sound effects, even though it's not "allowed" to. It will also sometimes copy your voice and act out what it thinks you should do instead of letting you talk, though.
- erehweb 2y ago
  Reminiscent of the Twilight Zone episode where a criminal ends up in the afterlife. Spoilers in link https://en.wikipedia.org/wiki/A_Nice_Place_to_Visit https://en.wikipedia.org/wiki/A_Nice_Place_to_Visit
  - empath75 2y ago
    The old gpt2 engine that used to drive AI Dungeon had the opposite problem, it was obviously trained on a ton of really bad fan fiction and would frequently either shift into filthy erotica or some kind of hyper violent serial killer story, no matter how innocently the story started.
    - MiddleEndian 2y ago
      I loved AI Dungeon for generating funny/stupid fanfiction style adventure/action/fantasy/scifi stories. It could keep track of characters and settings, have them return with motivations. And then something unexpected would happen. Provided me with a lot of entertainment. But my problem with using it as a DM is that you have to police yourself. I could just declare that I did something and it would automatically succeed. There were no rules or antagonism in the system. It was more of a collaborative storytelling tool than a game. To be fair I haven't used it in years so I dunno how it works nowadays.
cmpalmer52 2y ago
Yeah, completely automatic DMing just makes it like an open-ended computer game. On the other hand, the AI generated NPC responses are better than I could have come up with quickly, which makes me think co-DMing with an AI partner might be fun, particularly if they get better at remembering the details. That would let the human DM control the flow of the story without worrying that Thug #4 has taken 3hp of damage and is Dazzled. Of course, I most often use Foundry or Roll20 these days and they do a lot of that for you. I wonder how well it could do for game summaries and recaps?
short_sells_poo 2y ago
I find this idea fascinating, and I think a lot of very interesting emergent behavior can be added to video games by having something like an LLM act as a Dungeon Master. Rather than having to pre-program a bunch of choices that the player is allowed to pick between, have an AI model evolve the story and just move the pieces on the story board based on what the player actually does. Story driven games these days are still very much railroaded. They are like interactive movies. The player has very limited agency. They can't really pick any choice which the designers haven't thought to implement upfront. At the same time, I'm personally not super enthusiastic about using LLMs for a pen and paper game. Let's assume for a moment that we have free will, then the most fun aspect of PnP dungeon mastering is the fact that the DM has agency. It adds an (IMO) very fun mini-game where the players try to get away with as much shenanigans as they can, and the DMs job is to try to thwart these shenanigans in the most entertaining ways possible. The keyword is on "try" - the GM has to put entertaining obstacles in front of the players to keep the game fun, but ultimately the players should have an avenue of success. Even if they don't always succeed, they need to succeed enough and when they fail they need to feel that it was fair. I feel that with an AI DM, this goes out the window. At least with the current crop of AI, I would never feel that I'm facing an agent with it's own free will, rather just a machine. As such, it cannot replace a human DM for the PnP purposes.
cdiamand 2y ago
Last year, I built a DM for myself using the OpenAI api and an elevenLabs voice generator. I asked it set my character in Baldur's Gate, so it could pull upon the huge amount of DnD source material it had been trained on. A few takeaways: 1. An LLM based DM can give the player essentially infinite richness and description on anything they ask for. 2. There is difficulty in setting the rules for the LLM to follow that match the DnD rulebook. But this is possible to solve for. Also, I found the LLM to be too pliable as a DM. I kept getting my way, or getting my hand held through scenarios. Maybe this is a feature? 3. My conversation quickly began to approach the context window for the LLM and some RAG engineering is very necessary to keep the LLM informed about the key parts of your history. 4. Most importantly, I found that I most enjoy the human connection that I get through DnD and an LLM with a voice doesn't really satisfy that.
- coder543 2y ago
  > My conversation quickly began to approach the context window for the LLM and some RAG engineering is very necessary to keep the LLM informed about the key parts of your history Assuming we're talking about GPT-4o, that 128k context window theoretically corresponds to somewhere around 73,000 words. People talk at around 100 words per minute in conversation, so that would be about 730 minutes of context, or about 12 hours. The Gemini models can do up to 2 million tokens of context... which we could extrapolate to 11,400 minutes of context (190 hours), which might be enough? I would say GPT-4o was only good up to about 64k tokens the last time I really tested large context stuff, so let's call that 6 hours of context. In my experience, Gemini's massive context windows are actually able to retain a lot of information... it's not like there's only 64k usable or something. Google has some kind of secret sauce there. One could imagine architecting the app to use Gemini's Context Caching[0] to keep response times low, since it wouldn't need to re-process the entire session for every response. The application would just spin up a new context cache in the background every 10 minutes or so and delete the old one, reducing the amount of recent conversation that would have to be re-processed each time to generate a response. I've just never seen RAG work particularly well... and fitting everything into the context is very nice by comparison. But, one alternative to RAG would be a form of context compression... you could give the LLM several tools/functions for managing the context. The LLM would be instructed to use these tools to record (and update) the names and information of different characters, places, and items that the players encounter, important events that have occurred during the game, as well as information about who the current players are and what items and abilities those players have, and then the LLM would be provided with this "memory" in the context in place of a complete conversational record. The LLM would then just receive (for example) the most recent 15 or 30 minutes of conversation, in addition to that memory. > I found the LLM to be too pliable as a DM. I haven't tried using an LLM as a DM, but in my experience, GPT-4o is happy to hold its ground on things. This isn't like the GPT-3.5 days where it was a total pushover for anything and everything. I believe the big Gemini models are also stronger than the old models used to be in this regard. Maybe you just need a stricter prompt for the LLM that tells it how to behave? I also think the new trend of "reasoning" models could be very interesting for use cases like this. The model could try to (privately) develop a more cohesive picture of the situation before responding to new developments. You could already do this to some extent by making multiple calls to the LLM, one for the LLM to "think", and then another for the LLM to provide a response that would actually go to the players. One could also imagine giving the LLM access to other functions that it could call, such as the ability to play music and sound effects from a pre-defined library of sounds, or to roll the dice using an external random number generator. > 4. Most importantly, I found that I most enjoy the human connection that I get through DnD and an LLM with a voice doesn't really satisfy that. Sure, maybe it's not something people actually want... who knows. But, I think it looks pretty fun.[1] One of the harder things with this would be helping the LLM learn when to speak and when to just let the players talk amongst themselves. A simple solution could just be to have a button that the players can press when they want, which will then trigger the LLM to respond to what's been recently said, but it would be cool to just have a natural flow. [0]: https://ai.google.dev/gemini-api/docs/caching https://ai.google.dev/gemini-api/docs/caching [1]: https://www.youtube.com/watch?v=9oBdLUEayGI https://www.youtube.com/watch?v=9oBdLUEayGI
Deukhoofd 2y ago
Using a fine-tuned GPT-3.5 model is definitely a choice, why not pick one of the newer models? The methodology only states they use "ChatGPT" as LLM, but doesn't clarify the model until way later, nor why they picked that model over other existing models. The methodology also barely explains how they did their baseline experiments with human DMs. Did they do them face-to-face, or via text? Did they have different DMs in the 3 games they used as baseline, or was it the same person? As it stands the research is barely reproducible.
bnetd 2y ago
[dead]
noufalibrahim 2y ago
This is fascinating. I don't have any experience with D&D but wanted to learn it to play it with my kids. I used claude to learn the basic rules and then had it help me create the scenario. Then I played as the dm while it simulated being the players and then vice versa. Really nice experience
wuliwong 2y ago
A while ago I was playing with chatgpt and had it create me a text based game. It took some coaxing but eventually it actually created a pretty compelling beginning to a game at least. This wasn't D&D, I had chatgpt just make the rules but the interesting part was creating a compelling story. Maybe it would have fallen apart after a little bit of game play but it was neat. I could see the fun of having completely unique and instantly created modules to run through.
bsenftner 2y ago
I've made several DMs for a variety of entertainments. It's as much fun to create them as it is to play a scenario with them. I've experimented with a few methods, such as providing them with an ASCII map that looks like a hack/rogue map, giving them a famous book as the scenario, giving them multiple books and or famous fictional characters and then just letting them loose in a DM/AI guided narrative the user is making decisions. Of course, I have a "classic" DM that happens to be visiting here on vacation from the dungeons & dragon's multiverse, where his real job is being a DM for visiting tourists. There's a "Shaggy" and his talking dog one can take LSD with when going to Woodstock in '68. There's a StarTrek strategic tactical simulations instructor. I quite like a Underground railroad "conductor" that guides escaped slaves to freedom before the US Civil War. I think these DM / interactive narrative applications have a huge future in education and training.
SubiculumCode 2y ago
I try to variance of this where I was the DM and the llm played the player characters. It went okay but it was also a bit lame.
- drak0n1c 2y ago
  An ideal entry role for LLM in tabletop may be NPC companions/followers in the party that players/DM don't care for roleplaying or find tedious. Set it to listen in on conversations or VTT chat, and interject dialog on a rate-limited basis. Could lead to unusual situational ideas or at least humorous moments.
kelseyfrog 2y ago
Last year, during a period of recovery, I hosted a ChatGPT game with my nieces and newphew. I played the role of being a bride between them and the LLM. I read aloud the text, and translated their actions into queries. With slight nudges, GPT performed well. It excelled at NPC dialog, location description, and to some extent improvisation. However, eventually the campaign fell apart on session four or five. There was a disagreement between the players over exactly how many maps we had acquired. As you know, answering this question is a needle in a haystack problem and asking the LLM proved to illicit conflicting responses. It ultimately led to me concluding that pausing the campaign was the right choice and accepting that we all still had an amazing time up to that point. Which leads me back to the questions in the paper: > We believe that with the above evaluation methods we can try to answer the following questions: • How consistent is a LLM Agent in its generative dialogue? • How well can a LLM Agent keep a user engaged in a narrative plot? • How creative is a LLM Agent when generating complex story driven narratives? None of these are exactly relevant to my personal experience running a TTRPG game using ChatGPT and what makes it succeed or fail. What would make such a system an improvement is being able to store and retrieve facts about players (HP, stats, inventory), NPCs(names, location, history), and Places(items, descriptions, inhabitants) in order to solve the needs-haystack problem. Creating a world ontology and being able to CRUD facts against such an ontology would be a game changer(literally). Despite the ultimate demise of the game, the experiment was a success. I'd highly encourage anyone with a laptop, a handful of dice, and a group of interested friends to give a shot. It's fun, memorable, and a unique way to pass the time and bond.
- TeMPOraL 2y ago
  > What would make such a system an improvement is being able to store and retrieve facts about players (HP, stats, inventory), NPCs(names, location, history), and Places(items, descriptions, inhabitants) in order to solve the needs-haystack problem. Creating a world ontology and being able to CRUD facts against such an ontology would be a game changer(literally). Curiously, those are the exact things LLMs can't do well, and are not really supposed to. This is all stuff you (or an external system) should keep track of and supply to LLM in context "just in time". This is to say, LLMs are an important component, but alone they won't work. They need their tools :). In short, LLMs are pure "system 1", the complete solution still needs "system 2", which fortunately is more amenable to classical computing approaches.
jrpt 2y ago
I made something like this to let people explore the lore of my game. It has an AI act like a dungeon master as you play through a story. You can try it for free at: https://orbsccg.com/adventures https://orbsccg.com/adventures The main game is a strategic collectible card game, but the “adventure mode” is just for fun AI-powered stories.
karaterobot 2y ago
This is a student thesis, which doesn't mean it's bad, but is worth keeping in mind. I see that their control group size was three people (in three different sessions), but I don't see how many people they ran through the AI part of the experiment. I was wondering the other day whether there were good services for running tabletop games over AI. I don't know about good, because I haven't tried them, but there are certainly a few products like this today. Given that ChatGPT plus some fine-tuning was comparable to a human GM in this experiment, I'd be interested in another experiment that looked at whether these products (which charge more money) offer any benefit over just using ChatGPT. For them to be better than ChatGPT, it seems like they'd almost have to be better than a human GM to be worth the money.
devmor 2y ago
Like others, I don't really understand why anyone would desire this. D&D is a social activity, why would I want to remove the most core human component of a social activity. It'd be like someone who enjoys going to a sports bar with friends deciding they'd rather sit next to a vending machine.
- bongodongobob 2y ago
  Do you take issue with Baldur's Gate 3? Video games in general? Why not?
  - devmor 2y ago
    Baldur's Gate 3 (and other games of its nature) are narrative experiences crafted by writers to convey a story. I don't play BG3 when I want to play D&D, and when I do want to play something like BG3 I certainly wouldn't be interested in a machine generated amalgamation of other people's ideas with no thought or emotion behind it - that would be fundamentally pointless and a waste of my time - like reading Lorem Ipsum instead of a novel.
lowsong 2y ago
I am baffled as to why anyone would want this, or anything like it. Sure, it's a cool technical demo. But... the point of D&D is a social game played with others. Even if you play a D&D-like video game such as Baldur's Gate 3 (which I'd argue is a fundamentally different experience to playing a tabletop RPG anyway), you're experiencing a world and a story that someone else has crafted. What's the point of replacing that social interaction, or that connection to another person's creative vision through their art, with an LLM? What value can it ever provide?
- mog_dev 2y ago
  I imagine for people like me who do not have anyone nearby willing to play D&D
  - samwiseg 2y ago
    Right, but why not play BG3? You know that an LLM is not going to provide any meaningful narrative, or craft a thought provoking story.
    - standardly 2y ago
      > You know that an LLM is not going to provide any meaningful narrative, or craft a thought provoking story The title of the paper is "Exploring the POTENTIAL of LLM-based agents..." Right now, yeah, just play BG3. But the context of the discussion is its potential, which is certainly worth discussing imo.
  - lowsong 2y ago
    I've been in a weekly D&D game for over a decade. I don't keep playing it because of the game of D&D (certainly not, I could rant for a long time about the game's myriad shortcomings across many editions). I play it because of the friends I play it with, because of the social connection that exists in and around the game. To play a tabletop RPG without that isn't the same thing, and to play with LLM is bankrupt of meaning. You'd be better off picking up one of those old-school choose your own adventure books or a video game like BG3. At least those were written by someone with intent, at least there's meaning in the story and setting. Or, finding an online group.
drak0n1c 2y ago
For those averse to LLM defeating the social aspect of tabletop roleplaying, perhaps a better entry role for an LLM agent is as NPC party companions/followers. Set up the agent with a detailed character sheet and hidden backstory/motivations and set it to listen in on conversations or VTT chat, and interject dialog on a non-spammy basis. Could lead to unusual situational ideas or at least humorous moments.
- bongodongobob 2y ago
  This is what our DM does. It works really well for NPCs and saves him a shit ton of time and effort.
HanClinto 2y ago
This is extremely interesting to me. I understand why so many people are surprised that people would want this, but for whatever reason, I'm very much in the target audience. I'm a "perpetual DM" in most of my roleplaying groups, having led campaigns spanning years, but very rarely do I get to participate as a player. My kids love me running adventures and one-shots, and while I enjoy it, people don't always recognize how much work it is -- both the prep, as well as the mental effort of being "on" for the entire duration of the adventure. I like afternoon one-shot adventures, but I dislike reading through long sourcebooks. I enjoy crafting puzzles and presenting challenges to the players, but also recognize the mental effort that this takes, and sometimes I just want a copilot to handle the lore and the setting. I already rely on the rules-nerds in my group to act as copilots for the specifics of game mechanics -- I don't have time to read through that many tables and commit them to memory. Re: datasets, the paper authors used a dataset that they created by scraping Critical Role transcripts -- I doubt they had permission for that. I like contributing to open datasets for training open-source LLMs, and I think it would be really sweet for a playgroup to be able to contribute their play transcripts for such a thing. I would love to see a standard open-source dataset be collected for this, and perhaps even a standardized benchmark for quantifying the performance of particular LLMs in RPG contexts -- rules compliance, needle-in-haystack fact finding, consistency of roleplaying, understanding user intent, challenge creation / resolution, etc.
c6o 2y ago
My experience with D&D and LLM is that is a cheat sheet and a great helper, specially for newbies like me, I'm a Dad of 4 that plays DnD to teach some lessons and to get my kids out of the screens
alasdairking 2y ago
A good use for this is people wanting to try an RPG, but not having a bunch of people to do that with. A parent wanting to try out this D&D thing so he or she can run it with the kids, for example.
- rexpop 2y ago
  Not having a bunch of people to do that with is solved by going out and creating relationships and communities. This is the basis of democracy, of society. It's a tremendous loss for our species if we lose the impetus to connect. Do your kids a REAL favor, and build them a community—a SOCIETY—that they can join.
gunalx 2y ago
I found it to be a pretty OK first draft character creator, scene described and occasional lore/world builder. Not really anything more. Kinda a DM assistant more than a DM.
sentimentscan 2y ago
I've created two AI LLM D&D-style games. The second one will soon be released as a D&D-style game with Visual Novel elements (AI-generated images_. Here's how I can address some common concerns from comments above: Lack of Human and Social Interactions: This is a significant drawback when you look it as the social thing, however, they require scheduling. For me I wanted to venture beyond the typical fantasy/sci-fi settings and good bored of those D&D/warhammer world. LLM Versatility: LLMs can take you on wild (now non-NSFW) adventures. I wanted to play a D&D-style game set in the Venetian Renaissance and GPT delivered with accurate locations, families, and artists. It can do the same with medieval settings from my country, ancient Rome, surrealist landscapes, and whatever your favorite books, movies, or games involve. Customization: Want a game master who replies in poetry, slang, in style of you favourite author? No problem. Context: This is a tough one, but there are two approaches: Retrieval-Augmented Generation (RAG) Summarization, and then summarization of the summary. I've used this approach in my previous startup and in this game. Characters: I utilised the randomness of different character styles. You might meet smart/beautiful/evil/good characters with predefined styles, and the LLM quite effectively supports roleplay. Cons: Lack of Continuity: The biggest downside is the lack of continuity. Even if I generate a world and scenario, after a few rooms, it doesn't come together like a game would, where previous events feed into current ones, even when I provide again LLM with previous details. I kinda lack big picture design of story. I am working on that I'm currently running a private beta and looking for co-founders (game/design/coding) in the EU, preferably Poland. Please email me at contact@sentimentscanner.com for more information for beta membership and so one.
monkeycantype 2y ago
I’ve used an llm to help my niece, who is a performer, and has a flair for story, but no rpg experience dm as a human - llm team up. It was very successful
constantcrying 2y ago
The single greatest appeal of Pen and Paper RPGs is human interaction, where obviously the DM has the most important role. We have a solution for playing RPGs when you have no DM. It is called a "Video game", you can even play it with your friends. What I have used LLMs for is helping me prepare for certain things, giving ideas for organizations, quests, characters, etc. . They are pretty good at that, although I would avoid relying on them during actual gameplay.
standardly 2y ago
Yay, can't wait to read this. I spent a lot of time tuning prompts for gpt4 DM sessions, and I had a lot of fun. It has SO much potential, and I was pleasantly surprised! I am not really considering mainstream commercialization, but I just found it really fun to interact with. We just need a customized model full of fantasy text and a little DM RLHF. It's essentially just a story-telling llm, which obviously will need some work before consistent narratives can be delivered to the user. Not an AI expert or anything, just wanted to share my excitement about this. I urge you to try it out, if you're into this kind of thing. Tell ChaptGPT it's a DM, explain the rules, give it some context on how deliver the narrative and player choices. You can give some story or character exposition to give it a baseline. Its not too bad.
odedbendov 2y ago
[dead]
chaara-dev 2y ago
LLDM (Large Language Dungeon Master)

140 points by utiiiD 2y ago

Based on my experiences with this. Keeping context relevant is the hardest part. Your human GM will remember that funny remark from the session 3 years ago, but the LLM?

worthless-trash 2y ago
I'd love to see some human & machine co-op for DM. That'd be super helpful.
- 2099miles 2y ago
  Now this is solid, like an ai enhanced Evernote, but for organization, not generation/creation.
  - Sanzig 2y ago
    I've been meaning to take a crack at doing this - I'd like to use Whisper to transcribe session recordings and then have an LLM build out a campaign wiki. I enjoy being a DM but note taking and organization is my least favourite part.
- cdiamand 2y ago
  I've used LLM's in this capacity, and it's awesome. It quickly becomes a crutch.
ratedgene 2y ago
I think we just haven't built complex enough architectures to allow this. I have a few ideas boiling that would help facilitate long term context understanding and recall.
cloudking 2y ago
Maybe this is a solution https://www.letta.com/ https://www.letta.com/
- potatoman22 2y ago
  A context-based retrieval/memory system would be cool, too. Pull in relevant events based on the group's location and NPCs they're with.
mirages 2y ago
Also for that same module, is it able to integrate it in a broader campaign. I expanded the lore and why the players got here with extra quests, fluff and mystery and they only reached said dungeon at 2nd session only. Also the way a PC died due to a rat bite that could only be cured by magic (5e of this module is ironically harsher than the 3.5) as he was losing max HP every 24H. As the players expressed dislike with that character the owning player was willing to go for a new character and I had planned a big drama scene where the players exit in emergency the dungeon and try to rally the nearet village where there's a priestessd and happens that I could match the rolls so he would die in the priestess hands. But I made the mistake to allow them to spend the night to get their 3rd level which would have allowed the druid to get his cure illnesses spell botching the set up. Happens that I decided to exploit the fact that all other character the previous days made an offering to Beshaba to curse that character and force rolls during night (and try to find a way to make the new character come into play). How it unfolded : - they were sleeping and it was the bard's turn for watch - the bard was a follower of Tharizdun and was about to sacrifice the dieing character as said character was agonizing - the new character was a Kara-turian enforcer that was chasing a yokai (he didn't knew) that used other people as disguise, made it happen that the yokai choose the dieing character "by coincidence" appearance - the yokai being chased opens a portal while hiding to the dungeon - the new character litteraly pops through a Wild Magic portal that pushed away the bard and tries to bash the old character as the old character runes on his skin glow a last time as he dies and repels everybody - made the new character pop naked as I wasn't paying attention to what the player was saying when he was listing his inventory - I let them handle the big mess
Kuinox 2y ago
Neuro-sama, an AI VTuber streamer, manages to do this. There are topics brought along multiple streams and manages to remember details of previous streams.
- CuriousCosmic 2y ago
  With soemthing like Neuro-sama I wouldn't doubt that Vedal is doing a lot of liftwork behind the scenes to keep sufficient context available to Neuro-sama's LLM.
  - Philpax 2y ago
    And with the right kind of engineering, that could be automated too. I don't think the results would be as good as they are now - Vedal is bringing his own perspective into it - but once you have a proof of concept, you can improve it.
lemoncookiechip 2y ago
It's that: Spacial awareness, struggling to maintain the personas (and often talking for the player/s), difficulty improvising, pacing issues, lack of initiative, and just an over reliance on prompting to keep going correctly.

2099miles 2y ago

This is so interesting that people want this. This is beyond the guy saying “people don’t like making music so we made ai so it’s easier and faster to just prompt it to make music” . Like in my experience the best part of d&d is the dungeon master who knows their friends making a story and details specifically for fun and the fun of the group. And the interactions between players and the dm is where I derived the most enjoyment, automating that sounds insane and peak confusion/self defeating.

airstrike 2y ago
Some people don't have friends they can play with :sadge:
- foobarian 2y ago
  This. At home it's me and my one kid, and having an additional party as a DM would be priceless.
Melomomololo 2y ago
Just that finding a good DM is hard
dingnuts 2y ago
they think they want it -- that doesn't mean they will actually like it if they get it
prettyblocks 2y ago
I think it's more about the challenge/curiosity of creating the implementation. I'm not sure these AI driven text-RPGs will find long term audiences.
- pmichaud 2y ago
  They are a step on the road to fully responsive rpg worlds though, which don’t have to be text based.

noodletheworld 2y ago

Alas, I was desperately keen to read transcripts of actual gameplay and all we got was user surveys? Was there a structured combat system as sort of this or was it a talking only kind of thing? Who knows? Trying to get empirical evidence that an LLM can even remotely plausibly work as a DM with no social aspect and (as far as I can tell, reading the paper) no programmed system of stats, rolls, combat etc. seems impressive. …but was it that? Or just a DnD themed chatGPT conversation? Or a text adventure with an LLM describing each “room”? Did the LLM actually have agency to make decisions and perform tasks, or was it not an agent at all, just a conversation? Without a transcript, all you can really say for sure is yeah, I agree; more research is needed.

empath75 2y ago

The main problem with doing this on chatgpt is that it is _relentlessly_ cheerful and it's almost impossible for it to produce any kind of interpersonal drama or threat or danger. I tried to get it to run space opera about a diplomatic mission to a newly recontacted planet, full of Game of Thrones-style intrigue and plotting and scheming, and it was like playing with a cheerily optimistic new UN intern that completely believes in the power of compromise and diplomacy to solve all problems. It was always like: "And so the two of you hashed out your differences over a glass of Dubranian Forblik, and peace reigned forever and ever after." That's true of how it does role playing games and any kind of fiction at all -- it always wants to tie a neat bow on it at the end of every few paragraphs with a moral and everything. You also have to ask it to use python to dice roll for anything you want to potentially fail on, or it will always make everything you do succeed wonderfully. I will say this -- the advanced audio mode for D&D is amazing and it really does act out the characters and it will occasionally slip into different voices or even add sound effects, even though it's not "allowed" to. It will also sometimes copy your voice and act out what it thinks you should do instead of letting you talk, though.

erehweb 2y ago
Reminiscent of the Twilight Zone episode where a criminal ends up in the afterlife. Spoilers in link https://en.wikipedia.org/wiki/A_Nice_Place_to_Visit https://en.wikipedia.org/wiki/A_Nice_Place_to_Visit
- empath75 2y ago
  The old gpt2 engine that used to drive AI Dungeon had the opposite problem, it was obviously trained on a ton of really bad fan fiction and would frequently either shift into filthy erotica or some kind of hyper violent serial killer story, no matter how innocently the story started.
  - MiddleEndian 2y ago
    I loved AI Dungeon for generating funny/stupid fanfiction style adventure/action/fantasy/scifi stories. It could keep track of characters and settings, have them return with motivations. And then something unexpected would happen. Provided me with a lot of entertainment. But my problem with using it as a DM is that you have to police yourself. I could just declare that I did something and it would automatically succeed. There were no rules or antagonism in the system. It was more of a collaborative storytelling tool than a game. To be fair I haven't used it in years so I dunno how it works nowadays.

cmpalmer52 2y ago

Yeah, completely automatic DMing just makes it like an open-ended computer game. On the other hand, the AI generated NPC responses are better than I could have come up with quickly, which makes me think co-DMing with an AI partner might be fun, particularly if they get better at remembering the details. That would let the human DM control the flow of the story without worrying that Thug #4 has taken 3hp of damage and is Dazzled. Of course, I most often use Foundry or Roll20 these days and they do a lot of that for you. I wonder how well it could do for game summaries and recaps?

short_sells_poo 2y ago

I find this idea fascinating, and I think a lot of very interesting emergent behavior can be added to video games by having something like an LLM act as a Dungeon Master. Rather than having to pre-program a bunch of choices that the player is allowed to pick between, have an AI model evolve the story and just move the pieces on the story board based on what the player actually does. Story driven games these days are still very much railroaded. They are like interactive movies. The player has very limited agency. They can't really pick any choice which the designers haven't thought to implement upfront. At the same time, I'm personally not super enthusiastic about using LLMs for a pen and paper game. Let's assume for a moment that we have free will, then the most fun aspect of PnP dungeon mastering is the fact that the DM has agency. It adds an (IMO) very fun mini-game where the players try to get away with as much shenanigans as they can, and the DMs job is to try to thwart these shenanigans in the most entertaining ways possible. The keyword is on "try" - the GM has to put entertaining obstacles in front of the players to keep the game fun, but ultimately the players should have an avenue of success. Even if they don't always succeed, they need to succeed enough and when they fail they need to feel that it was fair. I feel that with an AI DM, this goes out the window. At least with the current crop of AI, I would never feel that I'm facing an agent with it's own free will, rather just a machine. As such, it cannot replace a human DM for the PnP purposes.

cdiamand 2y ago

Last year, I built a DM for myself using the OpenAI api and an elevenLabs voice generator. I asked it set my character in Baldur's Gate, so it could pull upon the huge amount of DnD source material it had been trained on. A few takeaways: 1. An LLM based DM can give the player essentially infinite richness and description on anything they ask for. 2. There is difficulty in setting the rules for the LLM to follow that match the DnD rulebook. But this is possible to solve for. Also, I found the LLM to be too pliable as a DM. I kept getting my way, or getting my hand held through scenarios. Maybe this is a feature? 3. My conversation quickly began to approach the context window for the LLM and some RAG engineering is very necessary to keep the LLM informed about the key parts of your history. 4. Most importantly, I found that I most enjoy the human connection that I get through DnD and an LLM with a voice doesn't really satisfy that.

coder543 2y ago
> My conversation quickly began to approach the context window for the LLM and some RAG engineering is very necessary to keep the LLM informed about the key parts of your history Assuming we're talking about GPT-4o, that 128k context window theoretically corresponds to somewhere around 73,000 words. People talk at around 100 words per minute in conversation, so that would be about 730 minutes of context, or about 12 hours. The Gemini models can do up to 2 million tokens of context... which we could extrapolate to 11,400 minutes of context (190 hours), which might be enough? I would say GPT-4o was only good up to about 64k tokens the last time I really tested large context stuff, so let's call that 6 hours of context. In my experience, Gemini's massive context windows are actually able to retain a lot of information... it's not like there's only 64k usable or something. Google has some kind of secret sauce there. One could imagine architecting the app to use Gemini's Context Caching[0] to keep response times low, since it wouldn't need to re-process the entire session for every response. The application would just spin up a new context cache in the background every 10 minutes or so and delete the old one, reducing the amount of recent conversation that would have to be re-processed each time to generate a response. I've just never seen RAG work particularly well... and fitting everything into the context is very nice by comparison. But, one alternative to RAG would be a form of context compression... you could give the LLM several tools/functions for managing the context. The LLM would be instructed to use these tools to record (and update) the names and information of different characters, places, and items that the players encounter, important events that have occurred during the game, as well as information about who the current players are and what items and abilities those players have, and then the LLM would be provided with this "memory" in the context in place of a complete conversational record. The LLM would then just receive (for example) the most recent 15 or 30 minutes of conversation, in addition to that memory. > I found the LLM to be too pliable as a DM. I haven't tried using an LLM as a DM, but in my experience, GPT-4o is happy to hold its ground on things. This isn't like the GPT-3.5 days where it was a total pushover for anything and everything. I believe the big Gemini models are also stronger than the old models used to be in this regard. Maybe you just need a stricter prompt for the LLM that tells it how to behave? I also think the new trend of "reasoning" models could be very interesting for use cases like this. The model could try to (privately) develop a more cohesive picture of the situation before responding to new developments. You could already do this to some extent by making multiple calls to the LLM, one for the LLM to "think", and then another for the LLM to provide a response that would actually go to the players. One could also imagine giving the LLM access to other functions that it could call, such as the ability to play music and sound effects from a pre-defined library of sounds, or to roll the dice using an external random number generator. > 4. Most importantly, I found that I most enjoy the human connection that I get through DnD and an LLM with a voice doesn't really satisfy that. Sure, maybe it's not something people actually want... who knows. But, I think it looks pretty fun.[1] One of the harder things with this would be helping the LLM learn when to speak and when to just let the players talk amongst themselves. A simple solution could just be to have a button that the players can press when they want, which will then trigger the LLM to respond to what's been recently said, but it would be cool to just have a natural flow. [0]: https://ai.google.dev/gemini-api/docs/caching https://ai.google.dev/gemini-api/docs/caching [1]: https://www.youtube.com/watch?v=9oBdLUEayGI https://www.youtube.com/watch?v=9oBdLUEayGI

Deukhoofd 2y ago

Using a fine-tuned GPT-3.5 model is definitely a choice, why not pick one of the newer models? The methodology only states they use "ChatGPT" as LLM, but doesn't clarify the model until way later, nor why they picked that model over other existing models. The methodology also barely explains how they did their baseline experiments with human DMs. Did they do them face-to-face, or via text? Did they have different DMs in the 3 games they used as baseline, or was it the same person? As it stands the research is barely reproducible.

bnetd 2y ago

[dead]

noufalibrahim 2y ago

This is fascinating. I don't have any experience with D&D but wanted to learn it to play it with my kids. I used claude to learn the basic rules and then had it help me create the scenario. Then I played as the dm while it simulated being the players and then vice versa. Really nice experience

wuliwong 2y ago

A while ago I was playing with chatgpt and had it create me a text based game. It took some coaxing but eventually it actually created a pretty compelling beginning to a game at least. This wasn't D&D, I had chatgpt just make the rules but the interesting part was creating a compelling story. Maybe it would have fallen apart after a little bit of game play but it was neat. I could see the fun of having completely unique and instantly created modules to run through.

bsenftner 2y ago

I've made several DMs for a variety of entertainments. It's as much fun to create them as it is to play a scenario with them. I've experimented with a few methods, such as providing them with an ASCII map that looks like a hack/rogue map, giving them a famous book as the scenario, giving them multiple books and or famous fictional characters and then just letting them loose in a DM/AI guided narrative the user is making decisions. Of course, I have a "classic" DM that happens to be visiting here on vacation from the dungeons & dragon's multiverse, where his real job is being a DM for visiting tourists. There's a "Shaggy" and his talking dog one can take LSD with when going to Woodstock in '68. There's a StarTrek strategic tactical simulations instructor. I quite like a Underground railroad "conductor" that guides escaped slaves to freedom before the US Civil War. I think these DM / interactive narrative applications have a huge future in education and training.

SubiculumCode 2y ago

I try to variance of this where I was the DM and the llm played the player characters. It went okay but it was also a bit lame.

drak0n1c 2y ago
An ideal entry role for LLM in tabletop may be NPC companions/followers in the party that players/DM don't care for roleplaying or find tedious. Set it to listen in on conversations or VTT chat, and interject dialog on a rate-limited basis. Could lead to unusual situational ideas or at least humorous moments.

kelseyfrog 2y ago

Last year, during a period of recovery, I hosted a ChatGPT game with my nieces and newphew. I played the role of being a bride between them and the LLM. I read aloud the text, and translated their actions into queries. With slight nudges, GPT performed well. It excelled at NPC dialog, location description, and to some extent improvisation. However, eventually the campaign fell apart on session four or five. There was a disagreement between the players over exactly how many maps we had acquired. As you know, answering this question is a needle in a haystack problem and asking the LLM proved to illicit conflicting responses. It ultimately led to me concluding that pausing the campaign was the right choice and accepting that we all still had an amazing time up to that point. Which leads me back to the questions in the paper: > We believe that with the above evaluation methods we can try to answer the following questions: • How consistent is a LLM Agent in its generative dialogue? • How well can a LLM Agent keep a user engaged in a narrative plot? • How creative is a LLM Agent when generating complex story driven narratives? None of these are exactly relevant to my personal experience running a TTRPG game using ChatGPT and what makes it succeed or fail. What would make such a system an improvement is being able to store and retrieve facts about players (HP, stats, inventory), NPCs(names, location, history), and Places(items, descriptions, inhabitants) in order to solve the needs-haystack problem. Creating a world ontology and being able to CRUD facts against such an ontology would be a game changer(literally). Despite the ultimate demise of the game, the experiment was a success. I'd highly encourage anyone with a laptop, a handful of dice, and a group of interested friends to give a shot. It's fun, memorable, and a unique way to pass the time and bond.

TeMPOraL 2y ago
> What would make such a system an improvement is being able to store and retrieve facts about players (HP, stats, inventory), NPCs(names, location, history), and Places(items, descriptions, inhabitants) in order to solve the needs-haystack problem. Creating a world ontology and being able to CRUD facts against such an ontology would be a game changer(literally). Curiously, those are the exact things LLMs can't do well, and are not really supposed to. This is all stuff you (or an external system) should keep track of and supply to LLM in context "just in time". This is to say, LLMs are an important component, but alone they won't work. They need their tools :). In short, LLMs are pure "system 1", the complete solution still needs "system 2", which fortunately is more amenable to classical computing approaches.

jrpt 2y ago

I made something like this to let people explore the lore of my game. It has an AI act like a dungeon master as you play through a story. You can try it for free at: https://orbsccg.com/adventures https://orbsccg.com/adventures The main game is a strategic collectible card game, but the “adventure mode” is just for fun AI-powered stories.

karaterobot 2y ago

This is a student thesis, which doesn't mean it's bad, but is worth keeping in mind. I see that their control group size was three people (in three different sessions), but I don't see how many people they ran through the AI part of the experiment. I was wondering the other day whether there were good services for running tabletop games over AI. I don't know about good, because I haven't tried them, but there are certainly a few products like this today. Given that ChatGPT plus some fine-tuning was comparable to a human GM in this experiment, I'd be interested in another experiment that looked at whether these products (which charge more money) offer any benefit over just using ChatGPT. For them to be better than ChatGPT, it seems like they'd almost have to be better than a human GM to be worth the money.

devmor 2y ago

Like others, I don't really understand why anyone would desire this. D&D is a social activity, why would I want to remove the most core human component of a social activity. It'd be like someone who enjoys going to a sports bar with friends deciding they'd rather sit next to a vending machine.

bongodongobob 2y ago
Do you take issue with Baldur's Gate 3? Video games in general? Why not?
- devmor 2y ago
  Baldur's Gate 3 (and other games of its nature) are narrative experiences crafted by writers to convey a story. I don't play BG3 when I want to play D&D, and when I do want to play something like BG3 I certainly wouldn't be interested in a machine generated amalgamation of other people's ideas with no thought or emotion behind it - that would be fundamentally pointless and a waste of my time - like reading Lorem Ipsum instead of a novel.

lowsong 2y ago

I am baffled as to why anyone would want this, or anything like it. Sure, it's a cool technical demo. But... the point of D&D is a social game played with others. Even if you play a D&D-like video game such as Baldur's Gate 3 (which I'd argue is a fundamentally different experience to playing a tabletop RPG anyway), you're experiencing a world and a story that someone else has crafted. What's the point of replacing that social interaction, or that connection to another person's creative vision through their art, with an LLM? What value can it ever provide?

mog_dev 2y ago
I imagine for people like me who do not have anyone nearby willing to play D&D
- samwiseg 2y ago
  Right, but why not play BG3? You know that an LLM is not going to provide any meaningful narrative, or craft a thought provoking story.
  - standardly 2y ago
    > You know that an LLM is not going to provide any meaningful narrative, or craft a thought provoking story The title of the paper is "Exploring the POTENTIAL of LLM-based agents..." Right now, yeah, just play BG3. But the context of the discussion is its potential, which is certainly worth discussing imo.
- lowsong 2y ago
  I've been in a weekly D&D game for over a decade. I don't keep playing it because of the game of D&D (certainly not, I could rant for a long time about the game's myriad shortcomings across many editions). I play it because of the friends I play it with, because of the social connection that exists in and around the game. To play a tabletop RPG without that isn't the same thing, and to play with LLM is bankrupt of meaning. You'd be better off picking up one of those old-school choose your own adventure books or a video game like BG3. At least those were written by someone with intent, at least there's meaning in the story and setting. Or, finding an online group.

drak0n1c 2y ago

For those averse to LLM defeating the social aspect of tabletop roleplaying, perhaps a better entry role for an LLM agent is as NPC party companions/followers. Set up the agent with a detailed character sheet and hidden backstory/motivations and set it to listen in on conversations or VTT chat, and interject dialog on a non-spammy basis. Could lead to unusual situational ideas or at least humorous moments.

bongodongobob 2y ago
This is what our DM does. It works really well for NPCs and saves him a shit ton of time and effort.

HanClinto 2y ago

This is extremely interesting to me. I understand why so many people are surprised that people would want this, but for whatever reason, I'm very much in the target audience. I'm a "perpetual DM" in most of my roleplaying groups, having led campaigns spanning years, but very rarely do I get to participate as a player. My kids love me running adventures and one-shots, and while I enjoy it, people don't always recognize how much work it is -- both the prep, as well as the mental effort of being "on" for the entire duration of the adventure. I like afternoon one-shot adventures, but I dislike reading through long sourcebooks. I enjoy crafting puzzles and presenting challenges to the players, but also recognize the mental effort that this takes, and sometimes I just want a copilot to handle the lore and the setting. I already rely on the rules-nerds in my group to act as copilots for the specifics of game mechanics -- I don't have time to read through that many tables and commit them to memory. Re: datasets, the paper authors used a dataset that they created by scraping Critical Role transcripts -- I doubt they had permission for that. I like contributing to open datasets for training open-source LLMs, and I think it would be really sweet for a playgroup to be able to contribute their play transcripts for such a thing. I would love to see a standard open-source dataset be collected for this, and perhaps even a standardized benchmark for quantifying the performance of particular LLMs in RPG contexts -- rules compliance, needle-in-haystack fact finding, consistency of roleplaying, understanding user intent, challenge creation / resolution, etc.

c6o 2y ago

My experience with D&D and LLM is that is a cheat sheet and a great helper, specially for newbies like me, I'm a Dad of 4 that plays DnD to teach some lessons and to get my kids out of the screens

alasdairking 2y ago

A good use for this is people wanting to try an RPG, but not having a bunch of people to do that with. A parent wanting to try out this D&D thing so he or she can run it with the kids, for example.

rexpop 2y ago
Not having a bunch of people to do that with is solved by going out and creating relationships and communities. This is the basis of democracy, of society. It's a tremendous loss for our species if we lose the impetus to connect. Do your kids a REAL favor, and build them a community—a SOCIETY—that they can join.

gunalx 2y ago

I found it to be a pretty OK first draft character creator, scene described and occasional lore/world builder. Not really anything more. Kinda a DM assistant more than a DM.

sentimentscan 2y ago

I've created two AI LLM D&D-style games. The second one will soon be released as a D&D-style game with Visual Novel elements (AI-generated images_. Here's how I can address some common concerns from comments above: Lack of Human and Social Interactions: This is a significant drawback when you look it as the social thing, however, they require scheduling. For me I wanted to venture beyond the typical fantasy/sci-fi settings and good bored of those D&D/warhammer world. LLM Versatility: LLMs can take you on wild (now non-NSFW) adventures. I wanted to play a D&D-style game set in the Venetian Renaissance and GPT delivered with accurate locations, families, and artists. It can do the same with medieval settings from my country, ancient Rome, surrealist landscapes, and whatever your favorite books, movies, or games involve. Customization: Want a game master who replies in poetry, slang, in style of you favourite author? No problem. Context: This is a tough one, but there are two approaches: Retrieval-Augmented Generation (RAG) Summarization, and then summarization of the summary. I've used this approach in my previous startup and in this game. Characters: I utilised the randomness of different character styles. You might meet smart/beautiful/evil/good characters with predefined styles, and the LLM quite effectively supports roleplay. Cons: Lack of Continuity: The biggest downside is the lack of continuity. Even if I generate a world and scenario, after a few rooms, it doesn't come together like a game would, where previous events feed into current ones, even when I provide again LLM with previous details. I kinda lack big picture design of story. I am working on that I'm currently running a private beta and looking for co-founders (game/design/coding) in the EU, preferably Poland. Please email me at contact@sentimentscanner.com for more information for beta membership and so one.

monkeycantype 2y ago

I’ve used an llm to help my niece, who is a performer, and has a flair for story, but no rpg experience dm as a human - llm team up. It was very successful

constantcrying 2y ago

The single greatest appeal of Pen and Paper RPGs is human interaction, where obviously the DM has the most important role. We have a solution for playing RPGs when you have no DM. It is called a "Video game", you can even play it with your friends. What I have used LLMs for is helping me prepare for certain things, giving ideas for organizations, quests, characters, etc. . They are pretty good at that, although I would avoid relying on them during actual gameplay.

standardly 2y ago

Yay, can't wait to read this. I spent a lot of time tuning prompts for gpt4 DM sessions, and I had a lot of fun. It has SO much potential, and I was pleasantly surprised! I am not really considering mainstream commercialization, but I just found it really fun to interact with. We just need a customized model full of fantasy text and a little DM RLHF. It's essentially just a story-telling llm, which obviously will need some work before consistent narratives can be delivered to the user. Not an AI expert or anything, just wanted to share my excitement about this. I urge you to try it out, if you're into this kind of thing. Tell ChaptGPT it's a DM, explain the rules, give it some context on how deliver the narrative and player choices. You can give some story or character exposition to give it a baseline. Its not too bad.

odedbendov 2y ago

[dead]

chaara-dev 2y ago

LLDM (Large Language Dungeon Master)