2 ms·
We're going to use BM25. Currently it is just dense search. Coming very soon
by GeorgeCurtis 1y ago
We're going to use BM25. Currently it is just dense search. Coming very soon
- elpalek 1y agohave you thought about SPALDE models? ex: https://arxiv.org/abs/2109.10086 https://arxiv.org/abs/2109.10086
- GeorgeCurtis 1y agoLooks really interesting, I'll have a proper read. What would be your reasoning to incorporate this if we already have vector functionality and semantic search?
- elpalek 1y agomy project deals w/ non-english text, bm25 performance is middeling. Language specific sparse model helps.
- xavcochran 1y agoWe will definitely look into it. The SPLADE models look promising!
- xavcochran 1y agoSPALDE*