2 ms·
It was still used as part of pretraining the current model.
by thfuran 2y ago
It was still used as part of pretraining the current model.
- namaria 2y agoNonsense, the current model is a new architectural approach. It was all explained in that recent paper, "Attention is all your meat"