3 ms·
At the expense of latency ? when in fact latency is the most important aspect for any search. Any idea on how fast the searches from the client are ? retrieve
by deepGem 3y ago
At the expense of latency ? when in fact latency is the most important aspect for any search. Any idea on how fast the searches from the client are ?
retrieve the embedding index and to run an indexed search to identify the data to be retrieved. Please bear with the layman like questioning -
So if the data is {obj: "obj1, "data": {"name": "atlas", "embedding": "1123124234" } What is an embedding index ? Is it something like {"1123124234": "obj1"} ?
From what I understand the query will be "geography" whose embedding will be "12311111" and now you have to run a KNN for a match which will return {"name": "atlas", "embedding": "1123124234"}
Not sure where the embedding index comes into play here.
- avereveard 3y agoEh, sure latency is suboptimal. But if you have a LLM in the mix, that latency will dominate the overall response time. At that point you might not care about how performant your index is, and since performance/cost is non linear, it can translate to very significant savings