2 ms·
Thank you for this comment, you've done a fair better job criticising cultural bias in LLMs than most. > But how do you create the terabytes of Hindi language
by versteegen 3y ago
Thank you for this comment, you've done a fair better job criticising cultural bias in LLMs than most.
> But how do you create the terabytes of Hindi language content required to make ChatGPT understand Hindi as well as it understands English?
It's wrong to assume that every language needs as much training data as English to match the English performance, since knowledge that isn't language- or culture-specific can transfer, in theory and (what used to be considered amazing) to a large extent in practice, once it understands the language very well. Of course a very large proportion, maybe the majority, of what the largest LLMs learn is cultural, but the actual language learning is only a small fraction: A transformer with only tens of millions of parameters can translate between two languages.
Are you saying ChatGPT (is that GPT 3.5?) is much worse in Hindi when given the same prompt in English and Hindi? I'm disappointed to hear that, I had thought OpenAI had done better at multi-lingual performance than many others.