3 ms·
GPT-4 has a really good tokenizer that is able to retain and use more information about input tokens than one might naively think.
by ComputerGuru 3y ago
GPT-4 has a really good tokenizer that is able to retain and use more information about input tokens than one might naively think.
- planb 3y agoInteresting. Given the example with made up words someone posted here it really looks like I was wrong.