ChatGPT presumably adds them as special tokens to the cl100k_base tokenizer, as they demo in the tiktoken documentation: https://github.com/openai/tiktoken#exte

ChatGPT presumably adds them as special tokens to the cl100k_base tokenizer, as they demo in the tiktoken documentation: https://github.com/openai/tiktoken#extending-tiktoken https://github.com/openai/tiktoken#extending-tiktoken In theory they could be added in normal input but it's possible OpenAI has safeguards against it.