3 ms·
None of the students who have taken the classes I TA pass w/I learning about RNNs.
by jszymborski 2y ago
None of the students who have taken the classes I TA pass w/I learning about RNNs.
- dekhn 2y agoIs that true also of LSTMs?
- jszymborski 2y agoYes. We cover Jordan and Elman RNN, LSTMs, and GRUs. Assignments only really test for LSTM knowledge, though.
- dekhn 2y agoThanks. The reason I asked the question is that I've struggled to understand RNNs and other networks (compared to MLPs, CNNs, and transformers) due to the subtlety of their design and my hope was that I could simply forget about them. I'm surprised about only testing for LSTMs- of all the sequence/memory models, they seem like the most arbitrary and hacky, but I've never been able to determine if that's simply because I don't understand those types of models (my training is in HMMs- do you teach/test those?)
- jszymborski 2y agoNo, we don't teach HMMs (although that would be super cool). It's strictly a neural networks class. A lot of my research has focused on LSTMs, and so I am partial to them. I think they are super useful and have a lot of properties, but frankly speaking if you had to choose one architectures of the ones you mentioned, LSTMs/RNNs are probably the most OK to skip. That said, if you just look at a simple RNN like the Jordan RNNs and focus on understanding that, then LSTMs just become fancy RNNs with some forgetting and remembering logic.