5 ms·
Why does every one of these example use pinecone when that costs money and Faiss[0] is free? If you’re going to let something run a bunch of fee-per-use API cal
by bugglebeetle 4y ago
Why does every one of these example use pinecone when that costs money and Faiss[0] is free? If you’re going to let something run a bunch of fee-per-use API calls on its own, why double up on getting charged?
[0] https://github.com/facebookresearch/faiss https://github.com/facebookresearch/faiss
- amrrs 4y agoThe code in this tutorial uses Faiss. Also you can experiment swapping the LLMs thanks to LangChain https://youtu.be/WosPGHPObx8 https://youtu.be/WosPGHPObx8
- digdugdirk 4y agoThis is run and accessed local-only, correct? I've been looking for a local-only vector database for LLM agent work like these.
- bugglebeetle 4y agoYup, it’s local and free! Best of both worlds!
- digdugdirk 4y agoI'll check it out! Any notable issues/differences between this and pinecone that I should keep an eye out for when getting started?
- bugglebeetle 4y agoI almost exclusively use Faiss, so I can’t really say. Apologies. Maybe someone else can weigh in.
- yinser 4y agoUsing langchain I’ve been introduced to chroma which uses a local store and it’s perfect. Check out their question and answer demo, the VectorstoreIndexCreator utilizes chroma and duckdb it seems. https://python.langchain.com/en/latest/use_cases/question_answering.html https://python.langchain.com/en/latest/use_cases/question_an...
- leobg 4y agoBecause FAISS and hnswlib don’t have “pine cronies” as advocates on HN, I guess.
- SomaticPirate 4y agoAlso pg_vector seems to be gaining traction
- spacetime_cmplx 4y agoUnless you have several hundred million documents, just write a simple encoder that serializes the embedding vectors to a flat binary file. Writing code from scratch to process and search 200k unstructured documents -- parsing, cleaning, chunking, OpenAI embedding API, serialization code, linear search with cosine similarity, and the actual time to debug, test and run all this -- took me less than 3 hours in Go. The flat binary representation of all vectors is under 500 MB. I even went ahead and made it mmap-friendly for the fun of it even though I could read it into all into memory. Even the dumb linear search I wrote takes just 20-30ms per query on my Macbook for the 200k documents. The search results are fantastic.
- dnadler 4y agoLess than 3 hours is impressive, but it took me less than 10 minutes to do the same with Chroma.
- qualudeheart 4y agoCould you share the code with us?
- spacetime_cmplx 4y agoSure! https://pastebin.com/xm7D1c30 https://pastebin.com/xm7D1c30 I didn't bother cleaning it so it's just a code dump, but it's fairly straightforward. Not included are a Python script to parse and clean the raw documents into JSON files (used in `summarize` to output results), code to read these files and get the embeddings from OpenAI for use in `newEmbeddingJSON `, and a bunch of random parallelization shell scripts that I didn't save. To use it, I call newDBFromJSON from a directory of JSON embedding vectors and serialize the binary representation. This takes a few minutes mostly because parsing JSON is slow, but you I only needed to do this once. When I need to search for the top 10 documents most similar to document X, I call `search` with the embedding vector for that doc. Alternatively if I need to do semantic search with natural language, I'll call the OpenAI API to get the embedding vector for the query and call `search` with that vector. It's pretty fast thanks to Go concurrency maxing out my CPU. It's super accurate with the search results thanks to OpenAI's embeddings. It's nowhere close to production-ready (it's littered with panics), but it was good enough for me. Hope this helps! Edit: oh and don't use float64 (OpenAI's vectors are float16)