3 ms·
Yeah good to mention these. I think they may be slightly too slow running for now, as BERT was for Google search when first released, but one day these will hel
by james-revisoai 4y ago
Yeah good to mention these. I think they may be slightly too slow running for now, as BERT was for Google search when first released, but one day these will help greatly.
- PaulHoule 4y agoCross-encoders are fine for reranking. You're certainly not going to run them for every document in the collection but can you run them for the top N? I think the answer to that is yes.
- james-revisoai 4y agoYes that's true, but performance vs run time isn't an easy win as compared to say using bicoders with pre-embedded retrieval embeddings and just embedding the query (which is almost always a < 0.1s operation even on a low-RAM EC2 instnace). It takes > 1 second(perhaps 1.5-3s if you embed paragraphs not sentences) for re-ranking k >= 8 on a decent say 32GB CPU instance..(requires encoding each document passage with the search query top k times)... if you have GPUs available for your search inference sure it's just a fraction of a second, but right now I would say it's significantly more onerous than bicoders that get you the first jump in semantic search performance (10-40x slower in practise on CPUs) so the use case differs.