Karpathy states in the blog post that word-level models currently tend to beat character models, across the broad field of NLP related RNNs. But he argues that

Karpathy states in the blog post that word-level models currently tend to beat character models, across the broad field of NLP related RNNs. But he argues that character models will eventually overtake (much in the same way that ConvNets have "replaced" manual feature extraction). That said, I think the RNNs here are limited by the corpus. They need to be exposed to more writing. Even if all you want is a Shakespeare generator, you still need to expose it to other literature. That will give it greater context, and more freedom of expression and, dare I say, creativity. I mean, imagine if all you were exposed to your whole life was Shakespeare. Nothing else (no other senses). Even with your superior mind, I doubt you'd generate anything better than what this RNN spits out. So yeah, it needs a large corpus to build a broader model. Then we need a way to instruct the broadly trained RNN to generate only Shakespeare-like text. Perhaps by adding an "author" or "style" input.