I think there is a deeper technical truth to this that hints at how much space there is to be gained in optimization. 1) that matryoshka representations work s

I think there is a deeper technical truth to this that hints at how much space there is to be gained in optimization. 1) that matryoshka representations work so well, and as few as 64 dimensions account for a large majority of the performance 2) that dimensional collapse is observed. Look at your cosine similarity scores and be amazed that everything is pretty similar and despite being a -1 to 1 scale, almost nothing is ever less than 0.8 for most models I think we’re at the infancy in this technology, even with all of the advances in recent years.