2 ms·
This is really a strange take. Encoder-only transformers like BERT and RoBERTa have been wildly popular in NLP for years now, replacing pretty much every model
by microtonal 3y ago
This is really a strange take. Encoder-only transformers like BERT and RoBERTa have been wildly popular in NLP for years now, replacing pretty much every model that came prior to it and beating pretty much all traditional NLP benchmarks (tagging, parsing, etc.).
- simonster 3y agoAlso BERT was integrated into Google search in 2019 (see https://blog.google/products/search/search-language-understanding-bert/amp/ https://blog.google/products/search/search-language-understa...)