I've poured more hours than I'll admit into AIDungeon at this point. The Dragon model is ridiculously good. I think everybody should try it, even if they're n

I've poured more hours than I'll admit into AIDungeon at this point. The Dragon model is ridiculously good. I think everybody should try it, even if they're not into text adventures. In fact, it's quite a poor "text adventure". But it's a really fun story telling tool. The hype is worth it in that regard. Having poured that much time into GPT-3 as a storyteller I can relay its weaknesses. The obvious limitation is the limited context (1024 tokens last the dev tweeted). It's really good at playing out isolated "scenes". It constantly surprises me with the creative things it comes up with. But yeah, outside of a scene, it totally loses focus on the bigger picture. We see the same problems with OpenAI's musicbox app, which struggled to be coherent across even a short song. That makes it a poor replacement for a true Dungeon Master, but a really useful tool for writing scenes. There are less publicized limitations. It sometimes gets the subject of sentences confused. When it does, it really wants to stick with that confusion regardless of how many times you have it retry. It often gets confused by who's speaking. Even when I go back and edit she/he said into its responses. AI Dungeon has a mode to turn off quotes presumably for this reason. It seems to have the same failings that image GANs do. Image GANs have "blind spots", types of images that they won't ever generate because they were too hard for it to learn. GPT-3 also has blind spots, certain ideas that it just doesn't understand. It won't generate content based on those ideas, and if it sees those ideas in its context it starts going off the rails. If you're lucky it'll just ignore the idea and generate what I call U-turn responses. That's where it does a 180 degree turn right out of the scene by saying something like "Suddenly there's a knock at the door!" and changing the scene. But half the time it just starts going into a loop and repeating itself. It's not as bad as GPT-2 where the repetition was like "like like like like like like". But it will repeat the same sentences or ideas over and over again. I guess because it doesn't understand the scene anymore it feels like repetition is the safest option. I haven't found any common trope to the ideas/scenes it has trouble with. Maybe it's just stuff it never got exposed to in its training set. I was looking over the GPT-3 paper this week and while the dataset is massive, it's by no means expansive yet. A human story teller is likely to have similar blind spots in terms of ideas we're familiar with, I think we're just far better one-shot learners. All this to say, AI Dungeon (and thus GPT-3) is amazing, in a limited context. If you let it drive the story and give a certain leeway to do crazy stuff, the adventures it will send you on are several orders of magnitude more interesting than I ever thought an AI was capable of in this decade.