5 ms·
Conclusion (from the article): Symbolic story generation systems were capable of generating reasonably long and coherent stories. These systems derived much of
by YeGoblynQueenne 5y ago
Conclusion (from the article):
Symbolic story generation systems were capable of generating reasonably long and coherent stories. These systems derived much of their power from well-formed knowledge bases. But these knowledge bases had to be structured by hand, which limited what the systems could generate. When we shifted to neural networks, we gained the power of neural networks to acquire and make use of knowledge from corpora. Suddenly, we could build story generation systems that could generate a larger space of stories about a greater range of topics. But we also set aside a lot of what was known about the psychology of readers and the ability to reason over rich knowledge structures to achieve story coherence. Even increasing the size of neural language models has only delayed the inevitability of coherence collapse in stories generated by neural networks.
A primer such as this one makes it easier to remember the paths that were trodden previously in case we find opportunities to avoid throwing the baby out with the bath water. This is not to say that machine learning or neural network based approaches should not be pursued. If there was a step backward it was because doing so gave us a powerful new tool with the potential to take us further ahead. The exciting thing about working on automated story generation is that we genuinely don’t know the best path forward. There is a lot of room for new ideas.
My personal view is more critical than the author's. There was an approach that worked reasonably well but needed more work to improve it. It was abandoned for a flashy new approach that doesn't work and doesn't seem like it could be made to work ("Even increasing the size of neural language models has only delayed the inevitability of coherence collapse in stories generated by neural networks"). Why was the working approach abandoned? The answer seems to do with a certain tendency in academia to blindly follow new trends with the only criterion being how many published papers can come out of it. That is no way to make progress. That is the way to waste time.
Besides, this is not true:
>> There is a lot of room for new ideas.
Once your field has been taken over by neural networks, there is no room for new ideas. You are destined to tweak hyperparameters for the rest of your academic career.
- andreyk 5y agoI think this is a bit too cynical/negative. It's not true that neural net approaches don't work, they just have different strengths and weaknesses (strength being generalization/flexibility/ease of use, weakness being long term coherence, interpretability, reliability). There are plenty of examples of traditional methods being combined with neural net methods (eg https://ai.stanford.edu/blog/qagnn/ https://ai.stanford.edu/blog/qagnn/ , https://arxiv.org/abs/1907.04799 https://arxiv.org/abs/1907.04799 , NeRF), and I think that is likely to happen in this context as well to some extent.
- YeGoblynQueenne 5y agoI really don't see generalisation, flexibility and ease of use (!) as strengths of neural networks and I'm shocked you think any of those are actually characteristics of neural networks. To clarify I didn't mean that neural network approaches don't work. I meant that they don't work _for story generation_ certainly not if the goal is to generate coherent stories. In the article, the best examples of stories generated from neural network approaches (basically, nn-trained language models + x, where x is various clever tricks that try to patch up the gaping holes in neural nets' inability to generate a coherent story) look decidedly worse than the stories generated by symbolic approaches listed in earlier sections. In my mind, the author of the article is being polite and trying to present the latest work in his field in a positive manner, possibly because of the natural tendency of researchers to not take a huge dump on each other's work. But some of the expressions he uses, like "in case we find opportunities to avoid throwing the baby out with the bath water" make it clear that he doesn't see the takeover of his field by neural networks as a step forward. Btw, "cynical"? Should I take that as a personal comment? It sounds a little insulting.
- pnut 5y agoAfter reading so many samples of machine generated text throughout the article, the conclusion paragraph felt entirely artificial and machine generated to me.