3 ms·

Additionally, the tokenizer vocabulary is unchanged from GPT-2. You can use HuggingFace's GPT-2 tokenizer as well. (some of OpenAI's GPT-3 notebooks do just th

Additionally, the tokenizer vocabulary is unchanged from GPT-2. You can use HuggingFace's GPT-2 tokenizer as well. (some of OpenAI's GPT-3 notebooks do just that).