3 ms·
I agree as well. Especially in the context of recommendation systems, the decoupling of retrieval from a heavy ranker has a lot of benefits. It allows for 1) fa
by laszlo_cravens 1y ago
I agree as well. Especially in the context of recommendation systems, the decoupling of retrieval from a heavy ranker has a lot of benefits. It allows for 1) faster experimentation, and 2) the use of different retrieval sources. In reality, the retrieval might consist of a healthy mix of different algorithms (collaborative filtering, personalized page rank, word2vec/2tower embeddings, popular items near the user, etc.) and fallback heuristics
- supo 1y agoIt allows faster experimentation because you can't do things like partial embedding updates and reasonable schema migrations on your vector search index - if you could, you'd experiment in retrieval... and with better retrieval you don't have to move 100s or 1000s of candidates from a database and pay a ton for a ranker inference on every query (not even mentioning the latency impact of that).