7 ms·
Understanding LSTM and its diagrams
- PaulHoule 9y agoPro Tip: The GRU is a much simpler unit (easier to explain) and usually works better than the LSTM.
- nonbel 9y agoWhat do you mean by "works better"? Do you mean something beyond having fewer parameters to store/calculate?
- PaulHoule 9y agoUsually it requires fewer epochs to train the GRU than the LSTM. The learning capacity of the LSTM and GRU is proportional to the number of weights, so you need more GRUs to do the job of a given number of LSTMs. It drives me nuts to see so many tutorials where people do the LSTM because they saw other people do the LSTM when doing the same with the GRU is simpler and better!
- reachtarunhere 9y agoInteresting paper that compares performance of GRU vs LSTM. http://proceedings.mlr.press/v37/jozefowicz15.pdf http://proceedings.mlr.press/v37/jozefowicz15.pdf TL;DR - For most tasks other than Language modeling GRU is better.
- cdancette 9y agowhy is LSTM better for language modeling?
- reachtarunhere 9y agoThe paper doesn't speak much. Empirical results.
- nafizh 9y ago[dupe] https://news.ycombinator.com/item?id=13612938 https://news.ycombinator.com/item?id=13612938 https://news.ycombinator.com/item?id=12960989 https://news.ycombinator.com/item?id=12960989
- sctb 9y ago> Are reposts ok? > If a story has had significant attention in the last year or so, we kill reposts as duplicates. If not, a small number of reposts is ok. https://news.ycombinator.com/newsfaq.html https://news.ycombinator.com/newsfaq.html