4 ms·
People are just automatically assuming that because we had this big leap in LLMs for chat responses, we would have an equivalent jump in LLMs for embedding base
by lysecret 3y ago
People are just automatically assuming that because we had this big leap in LLMs for chat responses, we would have an equivalent jump in LLMs for embedding based retrieval. And to my knowledge there is no evidence for that.
Quite to the contrary the recent gzip paper (even if it was badly done) still shows that retrieval is a very different problem and LLMs are much less extraodinary than expected.
In my mind the whole embedding / vector DB craze will come crushing down.
- mrfox321 3y agoAre you aware that the gzip paper fudged their accuracy numbers by assuming an oracle could correctly pick from the nearest 2 neighbors with 100% accuracy? In other words, they published top-1 accuracy from top-2 accuracy calculations. I would not over-index on that paper. However, I would err in favor of simpler methods.
- stormfather 3y agoI think a lot of the recent interest in embedding comes from the fact that it's so much more useful now. Now there is a way to usefully process natural language queries, and embedding is the way to retrieve related information in the course of processing the query.