4 ms·
Hi, there is a single loss right now, but I plan to add some Sentence Transformers losses. ColBERT is slow as a retriever, but is quite efficient as a Ranker on
by raphaelty 3y ago
Hi, there is a single loss right now, but I plan to add some Sentence Transformers losses. ColBERT is slow as a retriever, but is quite efficient as a Ranker on GPU (way faster than cross-encoder). I plan to release pre-trained checkpoints on HuggingFace with benchmarks using BEIRand inference speed info.
- aashu_dwivedi 3y agoDo you mean it's faster when the embeddings are pre-computed or is it faster when the embeddings are computed on the fly as well. Also, what's the recommended way to store the colbert embeddings as, because of the 2d nature of the embeddings it's not practical to store in a vector database.
- raphaelty 3y agoYes, ColBERT is fast because you can pre-compute most embeddings. It's important to compute documents embeddings only once. neural-cherche do not compute embeddings on the fly and the retrieve method ask for queries and documents embeddings rather than queries and documents texts. Documents and queries embeddings can be obtained using .encode_documents and .encode_queries methods I save most of my embeddings (python dictionnary with documents id as key and embeddings as values) using joblib in a Bucket in the cloud. I don't really know if it's a good pratice but it does scale fine to few millions documents for offline (no real-time) applications.