An RNN conditions the generation of each character on the entire previous history of characters generated. A Markov chain only conditions on a fixed window. Per

An RNN conditions the generation of each character on the entire previous history of characters generated. A Markov chain only conditions on a fixed window. Perhaps a particular RNN will learn to truncate its conditioning context and behave as a Markov chain, or perhaps not; but RNNs in general certainly can generate formal languages that Markov chains cannot. For example, consider that the RNN in Karpathy's post (the one linked to in this article) was capable of generating well-formed XML, generating matching opening and closing tags with an apparently unbounded amount of material between them. No Markov chain can do this.