3 ms·
1) It is extremely rare for a field to ever 'be solved'. There is still active research into how to multiply 2 numbers together. NLP is not anywhere close to so
by roenxi 3y ago
1) It is extremely rare for a field to ever 'be solved'. There is still active research into how to multiply 2 numbers together. NLP is not anywhere close to solved.
2) LLMs have different trade-offs to fundamental techniques. Linear regression still gets lots of use despite there usually being a theoretically better method for any specific application. There will be parallels to that in NLP.
3) Isn't the article talking about training things like LLM? It is right there - "Chapter 4: Training a neural network model".
- visarga 3y agoFunny thing, you can get LLM+Code Interpreter to do your linear regression with sklearn. So LLM+toys "can do" linear regression, or any other library algorithm. And I think I read in some paper that pure LLMs can do regression tasks in few shot mode. Maybe not as good as sklearn, but a decent result.
- roenxi 3y agoI'm unimpressed. I'd guess that every modelling technique can do linear regression. It is literally drawing a line through data. If you are lucky enough, even a one parameter model can get pretty close to what a linear regression does. But doing linear regression with LLMs is wasteful. The computational power alone makes it a party trick and not much more.
- axpy906 3y agoLarge Language Model in my opinion means that it has billions of parameters and possibly cannot fit on a single machine. Not sure what Spacy is doing under the hood these days but always thought of “neural net” as word2vec type model which won’t hit the above definition.