4 ms·
Has anything actually happened in the last two years? Genuinely curious.
by tw1010 7y ago
Has anything actually happened in the last two years? Genuinely curious.
- p1esk 7y agoAs a comparison, look at the transformer paper (also 2017), and the progress it has triggered.
- ivalm 7y agoI think there are two big things: 1. Attention mechanism (transformers). Really moved forward NLP. It allows long distance dependencies without the need for sequential processing (like in RNNs). Generally scales much better (see BERT/GPT-2/XL-Net/etc) 2. Separable convolutions. Significantly sped-up the standard CV networks. (mobile net, but now everything, really).