140 ms·
Not everything has to be directly informative or solve a problem. Sometimes data visualization can look pretty for pretty's sake. Dimensionality reduction/clus
by minimaxir 1y ago
Not everything has to be directly informative or solve a problem. Sometimes data visualization can look pretty for pretty's sake.
Dimensionality reduction/clustering like this may be less useful for identifying trends in token embeddings, but for other types of embeddings it's extremely useful.
- diwank 1y agoAgreed. The fact that it has any structure at all is fascinating (and super pretty). Could signal at interesting internal structures. I would love to see a version for Qwen-3 and Mistral too! I wonder if being trained on significant amounts of synthetic data gave it any unique characteristics.