9 ms·
As a sibling said, I think this is a focus of the article. At the very least Google has release multiple variations of one SOTA model this year: * original BE
by obmelvin 7y ago
As a sibling said, I think this is a focus of the article.
At the very least Google has release multiple variations of one SOTA model this year:
* original BERT paper:https://arxiv.org/abs/1810.04805 https://arxiv.org/abs/1810.04805
* ALBERT (smaller model): https://arxiv.org/abs/1909.11942 https://arxiv.org/abs/1909.11942
- gradys 7y agoT5 is another architecture from Google that set the SOTA on GLUE and SuperGLUE this year: https://arxiv.org/abs/1910.10683 https://arxiv.org/abs/1910.10683 It uses a Transformer and a BERT-style masked language model loss, but is fairly different in other regards. For instance, it formulates every problem as text-to-text, allowing it to work for just about any NLP problem.
- obmelvin 7y agoVery interesting, thank you! This may actually be useful for a project I had in mind given that it can be used on more problems :)