3 ms·
Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective
- Klathmon 1y ago> We propose adopting diffusion language models for text embeddings, motivated by their inherent bidirectional architecture and recent success in matching or surpassing LLMs especially on reasoning task. I didn't realize diffusion language models were at this point yet. But what's the catch? why aren't diffusion models (or some kind of hybrid) taking over?