3 ms·
how is this any different than this work(see figure 1)? https://aclweb.org/anthology/D16-1053 https://aclweb.org/anthology/D16-1053
by volkancirik 10y ago
how is this any different than this work(see figure 1)?
https://aclweb.org/anthology/D16-1053 https://aclweb.org/anthology/D16-1053
- jostmey 10y agoI've only skimmed it, so what I am about to say could be incorrect. The big difference between the models is how the attention mechanism is computed. The attention mechanism is just a weighted average. The weighted average can be computed as a running (or moving) average by saving the numerator and denominator at each step. This is what the RWA model is doing. I think the LSTM-N model, referenced in the paper you give, tries to brute force the calculation of the attention mechanism. Again, I could be wrong. I've only skimmed the paper you speak of