3 ms·
I don't see anything impressive in this paper. They basically say what they want to do, no results. They are not even original with the idea for this type of p
by Isinlor 5y ago
I don't see anything impressive in this paper. They basically say what they want to do, no results.
They are not even original with the idea for this type of paper.
In September last year Aran Komatsuzaki posted: "Current Limitations of Language Models: What You Need is Retrieval" https://arxiv.org/abs/2009.06857 https://arxiv.org/abs/2009.06857
I have also a prediction market from August last year with this question:
Will a language model with at least 100B parameters trained to do external information retrieval exist before 2023?
Seems like people gave it 61% in March this year. I was at 75%.
https://www.metaculus.com/questions/4876/will-a-language-model-with-at-least-100b-parameters-trained-to-do-external-information-retrieval-exist-before-2023/ https://www.metaculus.com/questions/4876/will-a-language-mod...
- ColinHayhurst 5y agoGood points. Thanks for the links.
- YeGoblynQueenne 5y ago>> I don't see anything impressive in this paper. They basically say what they want to do, no results. These are often called "position papers". There's no reason why they shouldn't be written, and published.
- Isinlor 5y agoYes, indeed. My point is that there is nothing impressive about it. It is just a good idea.
- YeGoblynQueenne 5y agoThe way I see it is that it doesn't have to be "impressive", as long as it's interesting and fosters debate. Science is a debate- not a circus with acrobats and trained horses. We can discuss problems and how to solve them without having to overhype everything all the time. In fact, for me that's the best way to do it. Even more so in machine learning where hype is over-abundant.