7 ms·

Curious to know if it’s more than just a representational problem. Are some languages harder in some deep way than others? And with what human consequences, let

Curious to know if it’s more than just a representational problem. Are some languages harder in some deep way than others? And with what human consequences, let alone the LLM costs?

qeternity 3y ago
It’s not. That plays a role, but language efficiency is a real thing.
deleted 3y ago
[deleted]
ajuc 3y ago
I don't think languages are significantly easier or harder, there's a limit on how hard it can get for average person to be able to speak it comfortably. I think there are different tradeoffs taken and if you come from a language with similar tradeoffs it's easier to learn. My native language is Polish, I know English and a little German and Spanish. Slavic languages often have the reputation of being difficult, but IMHO that's because they are easy in places English-speakers expect to be difficult, and difficult in places English-speakers expect to be easy. There's 15 tenses in English and 3 in Polish. There's no articles in Polish, and the pronunciation is almost perfectly regular. And there's probably 20 times fewer word roots, because of the pre/post-fix system. What in English is 20 unrelated words in Polish is one word root + 20 different combinations of pre/post fixes :) But to take advantage of this when you're learning you have to think in the language you are learning - to realize these words are related and how the postfixes modify the meaning. Otherwise you'll still need to memorize 20 separate words - and on top of that all that crap that is harder in Polish, like cases. I wonder if this influences LLMs (for example if they "think in Polish" when producing Polish text, or "think in English" and translate on the fly). I noticed GPT-3 was much better at rhyming in English than in Polish, despite the fact that rhyming in Polish is very easy (if the final letters match - it rhymes). When I explained this rule to it - it started rhyming better :)
- tempotemporary 3y ago
  I'd argue it's a bit easier in English if it comes to learning a grammatically correct expression of a thought. Polish or other slavic languages have quite a complexity hidden in "odmiany", "przypadki", and "koniugacje", etc.
- TRiG_Ireland 3y ago
  One could argue that English has only two true tenses: past and non-past. Everything else is a mood or aspectual distinction.
ajsnigrutin 3y ago
Some languages have some specifics that are not common in other languages, and if the language is small, it's hard to account for that too. In slovene, you have singular, plural but also dual forms, so even the basic "strings.xml" types of localizations don't work: eg: "I eat" would be "Jaz jem", if it was two of us the "We eat" would be "Midva jeva", and if 3+ of us would be eating, it would be "Mi jemo". Also when counting, we have a different form for one thing, two things, three-or-four things and five+ things, So, "(1-5) beer/s" would be "1 pivo, 2 pivi, 3 piva, 4 piva, 5 piv" So yeah, good luck :)
- orthoxerox 3y ago
  Is it "piv" all the way to 20 and then it's "21 pivo" and the pattern repeats every 10 items?
  - ajsnigrutin 3y ago
    All the way luckily, "19 piv, 20 piv, 21 piv,...", so at least this is consistent :)
- HexDecOctBin 3y ago
  > In slovene, you have singular, plural but also dual forms This is not unusual amongst Indo-European Languages, Sanskrit is similar. In fact, Baltic languages like Lithuanian seem to be a close sister to Indic languages like Sanskrit. [1] [1]: https://www.news9live.com/art-culture/why-lithuanian-sanskrit-similarities-continue-to-intrigue-linguists-two-centuries-on-158492 https://www.news9live.com/art-culture/why-lithuanian-sanskri...
- numpad0 3y ago
  Do "strings.xml" types of localization work for anything anyway? To me it just seems to let garbages match originals easily and giving false sense of accuracy to English speakers.
v4dok 3y ago
No its not. Greek are tokenized on a per character level, while the language structure while not as simple as english or latin languages in General, is not that different.
rsiqueira 3y ago
No. It's because OpenAI's token model is highly biased towards English words. In this way, other languages are greatly disadvantaged, reducing competitiveness in non-English-speaking countries. In my language (Portuguese and Spanish), for example, my cost increases by 60% to 75% due to the content I need to process not being in English.
numpad0 3y ago
I have a suspicion that LLMs has primary language that it must be always consistent with. If so it makes sense that it performs worse in non-native languages.

Curious to know if it’s more than just a representational problem. Are some languages harder in some deep way than others? And with what human consequences, let alone the LLM costs?

qeternity 3y ago

It’s not. That plays a role, but language efficiency is a real thing.

deleted 3y ago

[deleted]

ajuc 3y ago

I don't think languages are significantly easier or harder, there's a limit on how hard it can get for average person to be able to speak it comfortably. I think there are different tradeoffs taken and if you come from a language with similar tradeoffs it's easier to learn. My native language is Polish, I know English and a little German and Spanish. Slavic languages often have the reputation of being difficult, but IMHO that's because they are easy in places English-speakers expect to be difficult, and difficult in places English-speakers expect to be easy. There's 15 tenses in English and 3 in Polish. There's no articles in Polish, and the pronunciation is almost perfectly regular. And there's probably 20 times fewer word roots, because of the pre/post-fix system. What in English is 20 unrelated words in Polish is one word root + 20 different combinations of pre/post fixes :) But to take advantage of this when you're learning you have to think in the language you are learning - to realize these words are related and how the postfixes modify the meaning. Otherwise you'll still need to memorize 20 separate words - and on top of that all that crap that is harder in Polish, like cases. I wonder if this influences LLMs (for example if they "think in Polish" when producing Polish text, or "think in English" and translate on the fly). I noticed GPT-3 was much better at rhyming in English than in Polish, despite the fact that rhyming in Polish is very easy (if the final letters match - it rhymes). When I explained this rule to it - it started rhyming better :)

tempotemporary 3y ago
I'd argue it's a bit easier in English if it comes to learning a grammatically correct expression of a thought. Polish or other slavic languages have quite a complexity hidden in "odmiany", "przypadki", and "koniugacje", etc.
TRiG_Ireland 3y ago
One could argue that English has only two true tenses: past and non-past. Everything else is a mood or aspectual distinction.

ajsnigrutin 3y ago

Some languages have some specifics that are not common in other languages, and if the language is small, it's hard to account for that too. In slovene, you have singular, plural but also dual forms, so even the basic "strings.xml" types of localizations don't work: eg: "I eat" would be "Jaz jem", if it was two of us the "We eat" would be "Midva jeva", and if 3+ of us would be eating, it would be "Mi jemo". Also when counting, we have a different form for one thing, two things, three-or-four things and five+ things, So, "(1-5) beer/s" would be "1 pivo, 2 pivi, 3 piva, 4 piva, 5 piv" So yeah, good luck :)

orthoxerox 3y ago
Is it "piv" all the way to 20 and then it's "21 pivo" and the pattern repeats every 10 items?
- ajsnigrutin 3y ago
  All the way luckily, "19 piv, 20 piv, 21 piv,...", so at least this is consistent :)
HexDecOctBin 3y ago
> In slovene, you have singular, plural but also dual forms This is not unusual amongst Indo-European Languages, Sanskrit is similar. In fact, Baltic languages like Lithuanian seem to be a close sister to Indic languages like Sanskrit. [1] [1]: https://www.news9live.com/art-culture/why-lithuanian-sanskrit-similarities-continue-to-intrigue-linguists-two-centuries-on-158492 https://www.news9live.com/art-culture/why-lithuanian-sanskri...
numpad0 3y ago
Do "strings.xml" types of localization work for anything anyway? To me it just seems to let garbages match originals easily and giving false sense of accuracy to English speakers.

v4dok 3y ago

No its not. Greek are tokenized on a per character level, while the language structure while not as simple as english or latin languages in General, is not that different.

rsiqueira 3y ago

No. It's because OpenAI's token model is highly biased towards English words. In this way, other languages are greatly disadvantaged, reducing competitiveness in non-English-speaking countries. In my language (Portuguese and Spanish), for example, my cost increases by 60% to 75% due to the content I need to process not being in English.

numpad0 3y ago

I have a suspicion that LLMs has primary language that it must be always consistent with. If so it makes sense that it performs worse in non-native languages.