You should use more context than one previous word. You should use N previous words and you'll get more realistic results. I wrote a word game using markov cha

You should use more context than one previous word. You should use N previous words and you'll get more realistic results. I wrote a word game using markov chains to generate fake (but real looking) words. http://www.michaelfogleman.com/wug/ http://www.michaelfogleman.com/wug/ Read about the algorithm here: http://www.michaelfogleman.com/wug/algorithm http://www.michaelfogleman.com/wug/algorithm Also, you can leave punctuation in - you'd just need a larger training set, probably.