4 ms·
Interesting I think soon we are going to realize that we don’t really need training the models We just need good indexing and sampling Essentially at some le
by nico 2y ago
Interesting
I think soon we are going to realize that we don’t really need training the models
We just need good indexing and sampling
Essentially at some level any LLM is equivalent to a DB of the dataset, with a great NLP interface on top
Both are just different methods of navigating stored data
- nsagent 2y agoYou might like, the Infinigram paper then. It was discussed recently: https://news.ycombinator.com/item?id=40266791 https://news.ycombinator.com/item?id=40266791
- sdrg822 2y agoBut indexing *is* training. It's just not using end-to-end gradient descent.
- tempusalaria 2y agoLLMs can easily produce data not in training dataset. LLMs do not navigate stored data. An LLM is not a DB of the training data.
- carlthome 2y agoI've had the same thought as above but unfounded (just a feeling, pretty much) so I'm curious to learn more. Do you have any references I can check out that supports these claims?
- JoannaWongs 2y ago[flagged]
- PeterisP 2y agoThe models are multiple orders of magnitude smaller than the compressed versions of their training data, they can not be the equivalent of a DB of it.
- lainga 2y agoThe training data is ideo-semantically compressed? News to me... is it perhaps stored in kanji?