4 ms·

The other hard retrieval problems

26 points by softwaredoug 3y ago

esafak 3y ago
What I got out of this is that you need hybrid search, and that involves reranking. I agree, and do not find the point controversial. Is there some thesis beyond this? I think the author needs to elaborate. Since we're here, what libraries are y'all using for reranking in hybrid search these days? Anybody doing personalization or contextualization?
pmc00 3y ago
Here's a bit of a quantification of the point Doug makes. Indeed, for a number of scenarios you get better results if you combine vector search and keyword search into a hybrid retrieval step, and do reranking on top of that. https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/azure-ai-search-outperforming-vector-search-with-hybrid/ba-p/3929167 https://techcommunity.microsoft.com/t5/ai-azure-ai-services-... (disclaimer: I work in that team)
- anon373839 3y ago
  This is interesting. I recently built a search tool that needed to locate documents by keyword or by semantics, so I implemented a hybrid search straight away: BM25 + embeddings (from `gte-base`), with a cross-encoder for reranking. I found that the lexical search was adding nothing; the embeddings alone produced almost identical results for keyword queries. (The re-ranker, however, made a big difference.) Is this unusual?
  - pmc00 3y ago
    It depends on the scenario. For example, for concept-seeking queries, vectors tend to do better (less likely to be an overlap in words between query and content), whereas for keyword searches (a product name, a serial number, project codenames, etc.) BM25 + keywords does much better. If your workload is all concept-seeking queries, it's reasonable that keywords don't add much. If you look at the table in the section "3. Hybrid Retrieval brings out the best of Keyword and Vector Search" of that article, we shared there the significant variability of metrics as a function of query types.
peter_l_downs 3y ago
Quotidian viewpoint that everyone working in search already agrees with, for years, but nice to see it written down. Embedding-only search has never survived its first contact with an actual user.
ganzuul 3y ago
If you ensemble a bunch of optimizations you get something that behaves like a neuron. Neurons approximate the generator of the layers of 'blurs'. - The distribution. Such generators have to have separate origin to be truly orthogonal. E.g. evolutionary algorithms for rodents, particle swarm for tool use. If you have orthogonality in your evolution generator, you are not modeling evolution. To my intuition, when you are making orthogonal insertions into data with these constraints you are operating in a regime where computational complexity has exploded several times over what we are used to. Non-spoofed squirrels with screwdrivers would be extremely rare data so having an encoding scheme which allows for that would be a weird flex. Thoughts still baking...
brazzy 3y ago
Yeah, you're not the first person noticing the Leopard Print Sofa Problem: https://redsails.org/suddenly-a-leopard-print-sofa-appears/ https://redsails.org/suddenly-a-leopard-print-sofa-appears/

26 points by softwaredoug 3y ago

What I got out of this is that you need hybrid search, and that involves reranking. I agree, and do not find the point controversial. Is there some thesis beyond this? I think the author needs to elaborate. Since we're here, what libraries are y'all using for reranking in hybrid search these days? Anybody doing personalization or contextualization?

pmc00 3y ago

Here's a bit of a quantification of the point Doug makes. Indeed, for a number of scenarios you get better results if you combine vector search and keyword search into a hybrid retrieval step, and do reranking on top of that. https://techcommunity.microsoft.com/t5/ai-azure-ai-services-blog/azure-ai-search-outperforming-vector-search-with-hybrid/ba-p/3929167 https://techcommunity.microsoft.com/t5/ai-azure-ai-services-... (disclaimer: I work in that team)

anon373839 3y ago
This is interesting. I recently built a search tool that needed to locate documents by keyword or by semantics, so I implemented a hybrid search straight away: BM25 + embeddings (from `gte-base`), with a cross-encoder for reranking. I found that the lexical search was adding nothing; the embeddings alone produced almost identical results for keyword queries. (The re-ranker, however, made a big difference.) Is this unusual?
- pmc00 3y ago
  It depends on the scenario. For example, for concept-seeking queries, vectors tend to do better (less likely to be an overlap in words between query and content), whereas for keyword searches (a product name, a serial number, project codenames, etc.) BM25 + keywords does much better. If your workload is all concept-seeking queries, it's reasonable that keywords don't add much. If you look at the table in the section "3. Hybrid Retrieval brings out the best of Keyword and Vector Search" of that article, we shared there the significant variability of metrics as a function of query types.

peter_l_downs 3y ago

Quotidian viewpoint that everyone working in search already agrees with, for years, but nice to see it written down. Embedding-only search has never survived its first contact with an actual user.

ganzuul 3y ago

If you ensemble a bunch of optimizations you get something that behaves like a neuron. Neurons approximate the generator of the layers of 'blurs'. - The distribution. Such generators have to have separate origin to be truly orthogonal. E.g. evolutionary algorithms for rodents, particle swarm for tool use. If you have orthogonality in your evolution generator, you are not modeling evolution. To my intuition, when you are making orthogonal insertions into data with these constraints you are operating in a regime where computational complexity has exploded several times over what we are used to. Non-spoofed squirrels with screwdrivers would be extremely rare data so having an encoding scheme which allows for that would be a weird flex. Thoughts still baking...

brazzy 3y ago

Yeah, you're not the first person noticing the Leopard Print Sofa Problem: https://redsails.org/suddenly-a-leopard-print-sofa-appears/ https://redsails.org/suddenly-a-leopard-print-sofa-appears/