7 ms·
Others have chimed in as well, but I'll mention that we've been live with our product, for all users, for several months now doing RAG with OpenAI vector embedd
by phillipcarter 3y ago
Others have chimed in as well, but I'll mention that we've been live with our product, for all users, for several months now doing RAG with OpenAI vector embeddings stored in Redis.
We then just fetch up to the vectors related to a customer's schema in memory (largest is ~200MB) and run cosine similarity in a few ms in Go (handwritten, ~25 lines of code), and then we've got out top N things to place in our prompt.
Primitive? You betcha. Works extremely well for our entire customer base? Yup. You definitely don't need a Vector DB unless you have an enormous amount of vectors. For us it means having to run our own Redis clusters, but we know how to do that, and so we don't need to involve another vendor.
- gsuuon 3y agoFor local stuff with a handful of documents, you can even just throw it into a json and call it a day. The similarity search is as simple as an np.dot: https://github.com/gsuuon/llm.nvim/blob/main/python3/store.py#L254 https://github.com/gsuuon/llm.nvim/blob/main/python3/store.p...