3 ms·
Has anyone systematically compared embedding based retrieval against traditional full text search? With all of the focus on vector stores lately, it feels like
by anon373839 3y ago
Has anyone systematically compared embedding based retrieval against traditional full text search? With all of the focus on vector stores lately, it feels like the whole field of information retrieval has been nearly set aside.
- svcrunch 3y agoTake a look at the BEIR benchmark, which has served as one of the main drivers for development of neural IR systems since its introduction in 2020. BM25 presents a challenging cross-domain benchmark, and it wasn't till ~2022 that neural methods overtook it. If memory serves, it was the sparse neural methods like Splade, although recent dense models can also beat it. The caveat is that BEIR is suffering from overfitting at this point.