47 ms·
I think the "... is all you need" title here is particularly misleading as the paper does in fact use a BERT model for generating the vectors. So if the implic
by xg15 3y ago
I think the "... is all you need" title here is particularly misleading as the paper does in fact use a BERT model for generating the vectors.
So if the implication was that no language model was needed at all and you can just do nearest neighbour on string similarity and patch results together, that implication was clearly wrong.
I think what the paper does show though is that there are methods that can make language models topic-specific without fine-tuning and that yield competitive results even with older models.
- moffkalast 3y agoNext thing you'll say the Beatles are being misleading with 'All you need is love' because people also need food and shelter.
- Zacharias030 3y agommd! <3
- xg15 3y agoEh, the "attention is all you need" paper was kinda arguing that. And this paper doesn't.
- usgroup 3y agoYeah I thought the same -- it struck me at first blush as if it was some kind of super simple architecture that didn't use transformers, and then in the diagram i saw they used BERT to produce the embeddings!