>Text tokens are high-dimensional vectors, You are conflating tokens with embeddings. Tokens fit in a single word, modern gpt uses a vocabulary with 200k poss

>Text tokens are high-dimensional vectors, You are conflating tokens with embeddings. Tokens fit in a single word, modern gpt uses a vocabulary with 200k possible values, which would fit into 18 bits. Have a good one