3 ms·
I think you are confusing tokens with vectors/embedding/parameters. king and rex (king in latin) map to different tokens but will map to very similar vectors.
by TZubiri 1y ago
I think you are confusing tokens with vectors/embedding/parameters.
king and rex (king in latin) map to different tokens but will map to very similar vectors.