This is a nice intro to RNNs. The article title says RNNs are the workhorse of language modeling, but the transformer architecture used in models like BERT and

This is a nice intro to RNNs. The article title says RNNs are the workhorse of language modeling, but the transformer architecture used in models like BERT and GPT-2 handily beat RNNs.