3 ms·
Yeah what you mentioned might be true. Currently our understanding on how LLMs really work behind the screens is limited. For example, there was a recent resear
by chandureddyvari 3y ago
Yeah what you mentioned might be true. Currently our understanding on how LLMs really work behind the screens is limited. For example, there was a recent research[1] where LLM's accuracy is better if the context is added at the beginning when compared to the end of the prompt. So it's mostly by trial & error to figure out what works out best for you. You can use FAISS or similar to have the embeddings in-memory instead of a full fledged vector DB. But pg vector is convenient plugin if you already have postgres instance running
[1]- https://towardsdatascience.com/in-context-learning-approaches-in-large-language-models-9c0c53b116a1 https://towardsdatascience.com/in-context-learning-approache...
- halflings 3y agoWhat I mentioned doesn't depend on how LLMs work, the end result is the same (retrieving useful inputs to pass to your LLM). Just meant that a lot of people can just do this in-memory or in ad-hoc ways if they're not too latency constrained.
- bayesian_limit 3y agoZilliz just published an article comparing QPS (queries per second) with pg vector vs. Milvus. The results are clear - Milvus, a database designed ground-up for handling vector indexes, outperformed in terms of speed and latency. Dive into the details here. https://zilliz.com/blog/getting-started-pgvector-guide-developers-exploring-vector-databases https://zilliz.com/blog/getting-started-pgvector-guide-devel... Full disclosure, I just joined Zilliz this week as a Dev Advocate.