This should not be on HN for two reasons: 1) Only people how already know and used RNN could be interested 2) It is not a good paper Let me explain. MNIST is

This should not be on HN for two reasons: 1) Only people how already know and used RNN could be interested 2) It is not a good paper Let me explain. MNIST is not a good dataset for testing LSTMs. LSTMs were designed to handle sequential data. Of course you can use them with MNIST, but simple feed-forward network would be much easier and faster to train. RNNs give no advantage here. IMDb dataset is a decent choice for tests, but there is no convincing argument (or result) that makes LSTM1/LSTM2/LSTM3 a good choice. "Best accuracies of different LSTMs" table tells us absolutely nothing! You can just cherry-pick best results ignoring others. Other thing is authors didn't test GRU. GRUs are based on LSTM design, but simplified a bit. GRUs are known to perform better in many cases and are faster to train. The paper recommended by user Smerity is the best paper comparing RNN cell architectures and their performance I know and I recommend it too ("An Empirical Exploration of Recurrent Network Architectures")