4 ms·
How do you handle different LLMs having different vector spaces?
by joshspankit 4y ago
How do you handle different LLMs having different vector spaces?
- zh217 4y agoThey need to be put into distinct indices and unfortunately you cannot “jump” between them in this case (if someone knows a way to achieve this, I would love to hear!)
- joshspankit 4y agoI think I’m understanding that the item’s vector in one LLM can be stored as one index and the vector in another LLM can be stored as a second index without them colliding or one having to overwrite the other. Is that right?
- zh217 4y agoYes, actually I already do that. Sbert is better than openai ada embeddings for many use cases.
- joshspankit 4y agoAmazing. Then as far as I’m concerned that functionally solves the problem until the industry figures out cross-embedding jumps