4 ms·
"Most (if not all) embedding-based retrieval use approximate nearest neighbours (ANN). If we use exact nearest neighbours, we would get perfect recall of 1.0 bu
by pmoriarty 4y ago
"Most (if not all) embedding-based retrieval use approximate nearest neighbours (ANN). If we use exact nearest neighbours, we would get perfect recall of 1.0 but with higher latency (think seconds). In contrast, ANN offers good-enough recall (~0.95) with millisecond latency. I’ve previously compared several open-source ANNs and most achieved ~0.95 recall at throughput of hundreds to thousands of queries per second."
Can the results of multiple very fast, approximate queries somehow be used to get the equivalent of one very slow, reliable query?
- reissbaker 4y agoMight be useful to have an LLM generate a few versions of the query in order to account for imperfect recall — it seems like something gpt-3.5-turbo and friends would be pretty good at doing.