2 ms·
Does OpenAI’s tokenizer issues cash out into having worse results for Greek rather than just being more expensive for gpt-4? (gpt-3.5-turbo already costs peanut
by hombre_fatal 4y ago
Does OpenAI’s tokenizer issues cash out into having worse results for Greek rather than just being more expensive for gpt-4? (gpt-3.5-turbo already costs peanuts)
If not, then this response seems overblown. The competitive advantage in LLM at this point probably is not tokenizer optimizations and more about having results worth a damn.
- v4dok 4y agoThe usability is worse. Token limits are so much easier to reach. It's like using a model with dementia.
- hombre_fatal 4y agoI didn't think about that but now it's obvious. Good point.