3 ms·
The relation among the internal model representations inside its latent space and the embedding of the CoT compressed with a text embedding model is, more or le
by antirez 2y ago
The relation among the internal model representations inside its latent space and the embedding of the CoT compressed with a text embedding model is, more or less, minimal. Then we take this and map it to a 2D space, which captures more or less nothing of the original dimentionality and meaning. That's basically plotting random points.
- higuidebot 2y agoPotentially it's useful to understand a model "on its own terms" via its observable outputs. >The relation among the internal model representations inside its latent space and the embedding of the CoT compressed with a text embedding model is, more or less, minimal. This may or may not be correct but one way to find out is by taking a look!