2 ms·
What’s the expected additional latency due to running this re-ranker?
by yalok 1y ago
What’s the expected additional latency due to running this re-ranker?
- ghita_ 1y agoIt actually runs pretty fast, our benchmarks show ~149ms for 12665 bytes. It's faster than many other models
- esafak 1y agoI would prominently display your benchmarks (against your competitors, of course). That's your selling point, right?
- ghita_ 1y agoYes! We did this here: https://www.zeroentropy.dev/blog/announcing-zeroentropys-first-reranker https://www.zeroentropy.dev/blog/announcing-zeroentropys-fir... We wanted to share the approach with the community in this post. It does do better than competitors though!