6 ms·
An Introduction to AI Story Generation
- deleted 5y ago[deleted]
- haolez 5y agoThere is a relatively new game called Wildermyth that does an amazing job at adaptive storytelling. Would love to see how it works. Maybe they use AI. Who knows!
- bequanna 5y agoThat sounds interesting. Is it deterministic? That is, can you recreate a game story by making the exact same choices?
- haolez 5y agoProbably yes, but there are A LOT of variables that influence the story (e.g. characters personality).
- Qworg 5y agoThis sounded interesting to me as well, so I did some digging - the system is deterministic, as far as I can tell, but with lots of hidden and random information: https://wildermyth.com/wiki/index.php?title=Writer%27s_Guide https://wildermyth.com/wiki/index.php?title=Writer%27s_Guide
- deleted 5y ago[deleted]
- arcastroe 5y agoI recently finished a weekend project for ai generated short stories (GPT-3) https://toldby.ai https://toldby.ai I'm always impressed by what it comes up with, though not all stories are 100% coherent.
- valgaze 5y agoThis is a really compelling experience Curious-- are you "invoking" the model for every user or caching some of the results? I've been obsessed w/ using GPT3 for "personas" in conversations (for the agent to take on a character or follow instructions), but didn't seem feasible to make an API call (esp to Davinci) for every user
- arcastroe 5y agoI agree the API is a bit expensive. I'm fortunate that this particular use-case is fit for caching. It caches every result and only invokes OpenAPI upon clicking the "Generate More" button. The response is then saved for all future users of the site. So far everything on the page has been generated for under $10. But I can see how it would become prohibitively expensive if generating unique text for each user.
- sydthrowaway 5y agoHow did you get access to GPT-3?
- arcastroe 5y agoYou can apply for access to the API at the openai website. They approved pretty quickly
- d13 5y agoI’ve also been doing a lot of experimentation with gpt3 and gptj story generation recently. The problem is that although they always start off with something interesting, they will then begin to ramble, and finally go into an odd repetitive feedback loop. You can break this to some degree by injecting new, human generated content, into the prompt (which is what AI Dungeon does) but two problems always remain: the model has very little memory of context or previous events, and can’t plan into the future. So that while it will produce really interesting events that move the plot forward, it can only do so in response to an event that immediately precedes it. The “story” never finishes and there’s no real plot, it just meanders forward from event to event without development, long term consequences, goals or conclusion. This is because, as mentioned in the article, these models are always backwards looking and they don’t seem to build any kind internal model of the world they’re describing. And, the illusion of creativity is based on carefully constrained randomness, which, with enough brute force and luck, can produce interesting stuff, but can just as easily go off the rails. There may be some way to carefully design prompts so that models learn how to both summarize all past events, which could then used as context for subsequent events. And also, to train them how produce coherent story summaries and then use those as a nucleus to develop properly structured stories - but we’ll need to investigate :)
- sillysaurusx 5y agoOne unexpected epicenter in culture wars was whether it’s ok to generate stories involving children. You can probably guess the stories people had a problem with. Since models are becoming centralized, it’s a bigger deal than it seems. It’ll be interesting to see whether the AI community settles on one specific list of “don’t”’s, or if each company will have their own variant.
- nutanc 5y agoTrying out an experiment, creating a substack with purely AI generated content. GPT3 is pretty good for buzzfeed kind of content. https://aifeed.substack.com/ https://aifeed.substack.com/
- YeGoblynQueenne 5y agoConclusion (from the article): Symbolic story generation systems were capable of generating reasonably long and coherent stories. These systems derived much of their power from well-formed knowledge bases. But these knowledge bases had to be structured by hand, which limited what the systems could generate. When we shifted to neural networks, we gained the power of neural networks to acquire and make use of knowledge from corpora. Suddenly, we could build story generation systems that could generate a larger space of stories about a greater range of topics. But we also set aside a lot of what was known about the psychology of readers and the ability to reason over rich knowledge structures to achieve story coherence. Even increasing the size of neural language models has only delayed the inevitability of coherence collapse in stories generated by neural networks. A primer such as this one makes it easier to remember the paths that were trodden previously in case we find opportunities to avoid throwing the baby out with the bath water. This is not to say that machine learning or neural network based approaches should not be pursued. If there was a step backward it was because doing so gave us a powerful new tool with the potential to take us further ahead. The exciting thing about working on automated story generation is that we genuinely don’t know the best path forward. There is a lot of room for new ideas. My personal view is more critical than the author's. There was an approach that worked reasonably well but needed more work to improve it. It was abandoned for a flashy new approach that doesn't work and doesn't seem like it could be made to work ("Even increasing the size of neural language models has only delayed the inevitability of coherence collapse in stories generated by neural networks"). Why was the working approach abandoned? The answer seems to do with a certain tendency in academia to blindly follow new trends with the only criterion being how many published papers can come out of it. That is no way to make progress. That is the way to waste time. Besides, this is not true: >> There is a lot of room for new ideas. Once your field has been taken over by neural networks, there is no room for new ideas. You are destined to tweak hyperparameters for the rest of your academic career.
- andreyk 5y agoI think this is a bit too cynical/negative. It's not true that neural net approaches don't work, they just have different strengths and weaknesses (strength being generalization/flexibility/ease of use, weakness being long term coherence, interpretability, reliability). There are plenty of examples of traditional methods being combined with neural net methods (eg https://ai.stanford.edu/blog/qagnn/ https://ai.stanford.edu/blog/qagnn/ , https://arxiv.org/abs/1907.04799 https://arxiv.org/abs/1907.04799 , NeRF), and I think that is likely to happen in this context as well to some extent.