LLM embedding contain super positions of many concepts so while they might predict the next token they don’t actually out perform contrastively pretrained embed

LLM embedding contain super positions of many concepts so while they might predict the next token they don’t actually out perform contrastively pretrained embedding models.