3 ms·
Show HN: Zero downtime embedding model upgrades
People use embedding models all the time for rag/semantic retrieval. However, when a newer, more desireable model comes out, there is an expensive (both in time and computational) cost of re-embedding every document in the database.
However, I figured out an interesting way to forgo that upfront embedding cost.
algo:
old model/index -> retrieve top-K docs -> score those docs with the new model -> cache/materialize the new embeddings
so instead of rebuilding the entire vector store upfront, the old index keeps getting retrieved from, while the new model reranks those candidates.
This works surprisingly well for some model pairs, (i tested 63 source-> target migrations on h100s, on upto 1M documents).
For example, on a 1M document Natural Questions dataset,
native Qwen3-Embedding-8B: 0.6812 nDCG@10
Qwen3-4B -> Qwen3-8B, K=50: 0.6816
Qwen3-0.6B -> Qwen3-8B, K=50: 0.6638
MiniLM -> Qwen3-8B, K=50: 0.6486
(the hard part is determining k, I held the k constant above to give some sense of migratability).
You can install it with pip
pip install embedflow
and the code is on github
https://github.com/arnsri33/embedflow https://github.com/arnsri33/embedflow
- dancemonster35 2mo agoThis is not bad, has this been validated at a billion to a trillion documents?
- coolArnav 2mo agoNo, I haven't, but it has been validated at 1 million artifacts. Doing a billion to a trillion would have significant gpu cost.
- dancemonster35 2mo agookay, thats alright, this is good. have you looked at other vector database or are you still working on those now
- odesha 2mo ago[flagged]
- alexpran 2mo ago[flagged]