3 ms·
It stores everything on cheap storage, with no compute associated (i.e. S3) and uses the client to compute the query embeddings and to retrieve the embedding in
by avereveard 3y ago
It stores everything on cheap storage, with no compute associated (i.e. S3) and uses the client to compute the query embeddings and to retrieve the embedding index and to run an indexed search to identify the data to be retrieved, and likewise the client does the work of updating the index structure on writing.
The benefit is that you don't have to pay for the compute part of a database, and the storage layer is as cheap as it could be on the cloud.
- deepGem 3y agoAt the expense of latency ? when in fact latency is the most important aspect for any search. Any idea on how fast the searches from the client are ? retrieve the embedding index and to run an indexed search to identify the data to be retrieved. Please bear with the layman like questioning - So if the data is {obj: "obj1, "data": {"name": "atlas", "embedding": "1123124234" } What is an embedding index ? Is it something like {"1123124234": "obj1"} ? From what I understand the query will be "geography" whose embedding will be "12311111" and now you have to run a KNN for a match which will return {"name": "atlas", "embedding": "1123124234"} Not sure where the embedding index comes into play here.
- avereveard 3y agoEh, sure latency is suboptimal. But if you have a LLM in the mix, that latency will dominate the overall response time. At that point you might not care about how performant your index is, and since performance/cost is non linear, it can translate to very significant savings