3 ms·
Curious about the choice of FAISS. It's a bit older now, and there are many more options for creating and selecting embeddings. Does FAISS still offer some adva
by turnsout 3y ago
Curious about the choice of FAISS. It's a bit older now, and there are many more options for creating and selecting embeddings. Does FAISS still offer some advantages?
- simonw 3y agoWhat options do you think work better?
- J_Shelby_J 3y agoI’m trying build an exhaustive list of realistic options. See the spreadsheet here https://shelbyjenkins.github.io/blog/retrieval-is-all-you-need-2/ https://shelbyjenkins.github.io/blog/retrieval-is-all-you-ne...
- turnsout 3y agoI don’t have an opinion, just wondering why they didn’t choose other another option such as Sentence Embeddings, OpenAI embeddings, etc.
- simonw 3y agoFAISS can be used with OpenAI embeddings (and any other embedding model). FAISS is technology for fast indexed similarity vector search - using it is an independent decision from which model you use to create those vectors.
- alexpinel 3y agoHi! I'm the guy who made Dot. I remember experimenting with a few different vector stores in the early stages of the project but decided to settle with FAISS. I mainly chose it because it made it easy to perform the whole embedding process locally and also because it allows to merge vector stores which is what I use to load multiple types of documents at once. But I am definately not an expert on the topic and would really appreciate suggestions on other alternatives that might work better! :)