3 ms·
Naive RNN have vanishing gradient, but LSTMs and GRUs are much better in this respect.
by solomatov 4y ago
Naive RNN have vanishing gradient, but LSTMs and GRUs are much better in this respect.
- jszymborski 4y agoWhile this is true, and was a major advantage of LSTMs/GRUs, they still suffer from vanishing gradients. w.r.t proteins, our sequences often surpass 1500 amino acids and that is really tough for an LSTM to stably train on.