3 ms·
I have a RAG setup that doesn't work on documents but other data points that we use for generation (the original data is call recordings but it is heavily proce
by jascha_eng 1y ago
I have a RAG setup that doesn't work on documents but other data points that we use for generation (the original data is call recordings but it is heavily processed to just a few text chunks).
Instead of a reranker model we do vector search and then simply ask GPT-5 in an extra call which of the results is the most relevant to the input question. Is there an advantage to actual reranker models rather than using a generic LLM?
- tifa2up 1y agoOP here. rerankers are finetuned small models, they're cheap and very fast compared to an additional GPT-5 call.
- jascha_eng 1y agoIt's an async process in my case (custom deep research like) so speed is not that critical
- alansaber 1y agoI think you should do both in parallel, rather than sequentially. Main reason is vector scoring could cut off something that an LLM will score as relevant