For those unaware, OpenAI recently announced [0] an API change where they said their newer models are using Matryoshka representation learning for shortening em

For those unaware, OpenAI recently announced [0] an API change where they said their newer models are using Matryoshka representation learning for shortening embeddings. Basically you can use a shorter prefix of the full representation to do query/lookup for cheaper without losing much quality. Quote: “Native support for shortening embeddings: Using larger embeddings, for example storing them in a vector store for retrieval, generally costs more and consumes more compute, memory and storage than using smaller embeddings. Both of our new embedding models were trained with a technique [Matryoshka Representation Learning] that allows developers to trade-off performance and cost of using embeddings. Specifically, developers can shorten embeddings (i.e. remove some numbers from the end of the sequence) without the embedding losing its concept-representing properties by passing in the dimensions API parameter. For example, on the MTEB benchmark, a text-embedding-3-large embedding can be shortened to a size of 256 while still outperforming an unshortened text-embedding-ada-002 embedding with a size of 1536.” [0] https://openai.com/blog/new-embedding-models-and-api-updates https://openai.com/blog/new-embedding-models-and-api-updates