3 ms·
These tokens in theory should affect anything leveraging the original GPT-2 tokenizer. (OpenAI has switched to a different tokenizer for their new embeddings of
by minimaxir 4y ago
These tokens in theory should affect anything leveraging the original GPT-2 tokenizer. (OpenAI has switched to a different tokenizer for their new embeddings offering and I suspect GPT-4)
I can confirm that GPT-3 does similar weird things, but only for smaller prompts.