4 ms·
> new approach discovery a good example is discovery of attention mechanisms/transformers replacing more cumbersome and computationally expensive RNNs and LSTM
by sayonaraman 5y ago
> new approach discovery
a good example is discovery of attention mechanisms/transformers replacing more cumbersome and computationally expensive RNNs and LSTMs in NLP and more recently outperforming more expensive models in computer vision.
- visarga 5y agoTransformers are pretty huge and expensive to run, LSTMs are lighthweight by comparison.