5 ms·
After skimming through this, it seems fair to say a lot of the research to come out of this group has been relatively "soft", i.e. ethics and interpretability r
by mpoteat 7y ago
After skimming through this, it seems fair to say a lot of the research to come out of this group has been relatively "soft", i.e. ethics and interpretability rather than e.g. the newest most powerful models or innovations like gauge CNNs.
Not bad, just surprising.
- clSTophEjUdRanu 7y agoWhy is it surprising?
- paxys 7y agoIt's more like that is what they are choosing to focus on in their annual report. If you scroll through the content in the same blog over the past year you'll see that the posts are very technical in nature.
- goatinaboat 7y agoethics Something that Google definitely needs to do more research on.
- sgt101 7y agoNot surprising, important.
- hooande 7y agoethics and interpretability were the major trends of 2019, across the field
- obmelvin 7y agoAs a sibling said, I think this is a focus of the article. At the very least Google has release multiple variations of one SOTA model this year: * original BERT paper:https://arxiv.org/abs/1810.04805 https://arxiv.org/abs/1810.04805 * ALBERT (smaller model): https://arxiv.org/abs/1909.11942 https://arxiv.org/abs/1909.11942
- gradys 7y agoT5 is another architecture from Google that set the SOTA on GLUE and SuperGLUE this year: https://arxiv.org/abs/1910.10683 https://arxiv.org/abs/1910.10683 It uses a Transformer and a BERT-style masked language model loss, but is fairly different in other regards. For instance, it formulates every problem as text-to-text, allowing it to work for just about any NLP problem.
- obmelvin 7y agoVery interesting, thank you! This may actually be useful for a project I had in mind given that it can be used on more problems :)