This has been tried. The code looks kind of plausible, but doesn't quite compile. Some of the other examples (Shakespeare, Latex, wikipedia source code) are pre

This has been tried. The code looks kind of plausible, but doesn't quite compile. Some of the other examples (Shakespeare, Latex, wikipedia source code) are pretty good at a structural level. http://karpathy.github.io/2015/05/21/rnn-effectiveness/ http://karpathy.github.io/2015/05/21/rnn-effectiveness/ This had about 10 million parameters though, compared to OpenAi's 1.5B. I don't think their text is nonsense though. There's some very interesting examples. They have since released publicly a 345M parameter version, there are some nice examples from that model pre-trained with modern poetry on this twitter account: https://twitter.com/rossgoodwin https://twitter.com/rossgoodwin