3 ms·
It would make sense to repeat the experiment with the tokenizer of an LLM.
by samus 10mo ago
It would make sense to repeat the experiment with the tokenizer of an LLM.
- sebastianmestre 10mo agoPretty sure that OpenAI uses BPE in their GPT models