3 ms·
Very little most likely. The first step of GPT retrieves for each token a corresponding embedding vector, which is then what's used in the rest of the model. I'
by n2d4 3y ago
Very little most likely. The first step of GPT retrieves for each token a corresponding embedding vector, which is then what's used in the rest of the model. I'd assume those vectors are nearly the same for "color" and "colour".