Given the performance of tokenizers by language the answer must be “worse,” there are a variety of sources for this .. they typically focus on cost, but it’s al

Given the performance of tokenizers by language the answer must be “worse,” there are a variety of sources for this .. they typically focus on cost, but it’s also borne out in quality. https://denyslinkov.medium.com/why-is-gpt-3-15-77x-more-expensive-for-certain-languages-2b19a4adc4bc https://denyslinkov.medium.com/why-is-gpt-3-15-77x-more-expe... https://arxiv.org/pdf/2305.13707 https://arxiv.org/pdf/2305.13707 https://arxiv.org/pdf/2305.15425 https://arxiv.org/pdf/2305.15425 A quick Kagi can find a lot more.