"We know that Recurrent Neural Networks (LSTM/GRU) based models are good at capturing long term dependencies" This seems like a bold statement. We know they ar

"We know that Recurrent Neural Networks (LSTM/GRU) based models are good at capturing long term dependencies" This seems like a bold statement. We know they are good at capturing some structure from sequential data at least, but they seem lacking in some regards [0]. However it seems to yield some interesting results. The architecture is like an autoencoder, but with RNNs connected at each layer? Have you tried using NTMs? [0]: https://rylanschaeffer.github.io/content/research/neural_turing_machine/main.html https://rylanschaeffer.github.io/content/research/neural_tur... (section on LSTM Copy Performance)