3 ms·
Their 1024 dimension outperforms their 2048 dimension? What am I missing here.
by qeternity 2y ago
Their 1024 dimension outperforms their 2048 dimension? What am I missing here.
- fzliu 2y agoFor float and int8, 1024 does indeed outperform 2048. However, for binary, 2048 outperforms 1024. For our general-purpose embedding model (voyage-3-large), embedding vectors with 2048 dimensions outperform 1024 across the board: https://blog.voyageai.com/2025/01/07/voyage-3-large/ https://blog.voyageai.com/2025/01/07/voyage-3-large/
- qeternity 2y agoDo you have any insights into this? Perhaps scaling degradation with Matryoshka learning?