3 ms·
I think it's still overkill though for semantic embedding, SBERT is on order of ~250mm parameters while smallest llama at 7b parameters.
by jerrygenser 3y ago
I think it's still overkill though for semantic embedding, SBERT is on order of ~250mm parameters while smallest llama at 7b parameters.
- eigenvalue 3y agoIf all you want to do is make some basic semantic search, that’s probably true. But I strongly suspect we are only just now starting to scratch the surface of what’s possible with embeddings that come from much more powerful LLMs like Llama2 that can clearly manifest much greater demonstrated “understanding” of sentences they are shown (whatever that means, but intuitively, it seems obvious to me). That’s partly why I made this tool—- to aid in my investigations of LLM embeddings in a convenient and performant way.
- gsuuon 3y agoI'm really curious to see where that investigation leads - have you done any comparisons between Llama 2 and the embedding focused models? I wonder if it'll be better able to provide more 'intuitively correct' similarities?