3 ms·
Token Embeddings Violate the Manifold Hypothesis
- verdverm 11mo agoIf this interests you, https://bsky.app/profile/jefferyharrell.bsky.social https://bsky.app/profile/jefferyharrell.bsky.social has been making some super cool visualizations of Qwen3's internal representations One thing discovered was that 800+ tokens converged to a single point in the 2500+ dimensional unembedding space Some examples: https://bsky.app/profile/jefferyharrell.bsky.social/post/3m4w4bmwshc2c https://bsky.app/profile/jefferyharrell.bsky.social/post/3m4... https://bsky.app/profile/jefferyharrell.bsky.social/post/3m52edib3j32o https://bsky.app/profile/jefferyharrell.bsky.social/post/3m5...