3 ms·

Has anybody else noticed the map indicating that there are barely any fluent English speakers outside of the "English-speaking" countries? Or is the threshold f

by samus 3y ago

Has anybody else noticed the map indicating that there are barely any fluent English speakers outside of the "English-speaking" countries? Or is the threshold for fluency so high that no place even in Europe except the Ireland and the UK qualify at all?

airspresso 3y ago
This caught my eye as well. I interpreted it as the map only shows countries that have English as primary language. But the article was not precise enough about this. I applaud them for focusing on multilingual performance though, as that is an important area of NLP which still has lots of room for improvement.
- samus 3y ago
  I highly welcome the effort as well*, but I don't see why they would have to mistake first-language ability for fluency to argue for that. The difference is vast and relevant: anyone with good English reading and writing skills can take advantage of a model and might prefer it over a worse model in their native language. *: Just sceptical whether there's enough content out there which isn't just (often badly or too straightforwardly) translated from English. Not an issue for the languages with let's say >10 Mio. speakers, but for everything smaller.
  - pico_creator 3y ago
    Yea, thats why I focused only on the top 25 languages, despite the model being trained for 100+ languages. Was not confident, on the languages beyond the 25th cut-off, until we build better datasets (which we are in works on with various regional groups!)
  - vidarh 3y ago
    Norwegian has ca. 5m speakers, and ChatGPT does not just do fine with both the (mutually intelligible) Norwegian written languages, but also has no problem "translating" to/from several regional dialects when I've experimented. And that is, I presume - I could be wrong -, before anyone has tried to really mine the Norwegian national library, as even much of what is online is not easily accessible for crawling. I think there'll be plenty of content for even much smaller languages - especially anywhere with depositary laws -, but it's often going to require cumbersome collection efforts and negotiating access.
    - samus 3y ago
      It's an arbitrary threshold for the sake of discussion. Of course Norway, one of the most developed countries and richest countries per capita, manages to have good content in its written languages on the internet. Model training also benefits from its linguistic and cultural similarities to Swedish and Danish, and English, Dutch, and German to a lesser degree. I suspect many people from Eastern Europe find Russian models useful for the same reason. I agree that the situation is not hopeless for languages with a thriving written culture. But for many minority languages there might only be chat messages, some literary works, and vast amounts of machine-translated websites accessible for crawlers. I hope that in the future improved model architectures and training strategies can push the required amount of raw content way down.
- pico_creator 3y ago
  Yea, the map is poorly generated IMO as well - sadly there is only a few online tools online i could fine that "help me highlight, places where this list of languages is supported" So if we want a better map, we might need to redo from scratch That being said, depending on your sources, you can find multiple citations for 15-18% of the world population support english With approximately 25% being native speaker (english as first language) and 75% as non-native [PS: i realise later we were talking about different maps, the comment is for the 2nd map, for the languages we support]
Y_Y 3y ago
English is an official language in Malta and about 70% of the population is classed as "advanced" in English[0]. That only adds up to 280k people though, you'd probably find as many native-level speakers in any big European country, though not at the same density. [0] https://nso.gov.mt/wp-content/uploads/Skills-Preliminary.pdf https://nso.gov.mt/wp-content/uploads/Skills-Preliminary.pdf
pico_creator 3y ago
Thanks for flagging that out, swapped it out with another map I found, that shows different colors for different thresholds So its not "as strict"
- viraptor 3y ago
  That looks better. I think the original really was for native speakers only. Iceland is a nice quick test for that, since it should be >70% fluent in English, even with next to no native speakers. On the previous map it was almost completely white.

by samus 3y ago

airspresso 3y ago

This caught my eye as well. I interpreted it as the map only shows countries that have English as primary language. But the article was not precise enough about this. I applaud them for focusing on multilingual performance though, as that is an important area of NLP which still has lots of room for improvement.

samus 3y ago
I highly welcome the effort as well*, but I don't see why they would have to mistake first-language ability for fluency to argue for that. The difference is vast and relevant: anyone with good English reading and writing skills can take advantage of a model and might prefer it over a worse model in their native language. *: Just sceptical whether there's enough content out there which isn't just (often badly or too straightforwardly) translated from English. Not an issue for the languages with let's say >10 Mio. speakers, but for everything smaller.
- pico_creator 3y ago
  Yea, thats why I focused only on the top 25 languages, despite the model being trained for 100+ languages. Was not confident, on the languages beyond the 25th cut-off, until we build better datasets (which we are in works on with various regional groups!)
- vidarh 3y ago
  Norwegian has ca. 5m speakers, and ChatGPT does not just do fine with both the (mutually intelligible) Norwegian written languages, but also has no problem "translating" to/from several regional dialects when I've experimented. And that is, I presume - I could be wrong -, before anyone has tried to really mine the Norwegian national library, as even much of what is online is not easily accessible for crawling. I think there'll be plenty of content for even much smaller languages - especially anywhere with depositary laws -, but it's often going to require cumbersome collection efforts and negotiating access.
  - samus 3y ago
    It's an arbitrary threshold for the sake of discussion. Of course Norway, one of the most developed countries and richest countries per capita, manages to have good content in its written languages on the internet. Model training also benefits from its linguistic and cultural similarities to Swedish and Danish, and English, Dutch, and German to a lesser degree. I suspect many people from Eastern Europe find Russian models useful for the same reason. I agree that the situation is not hopeless for languages with a thriving written culture. But for many minority languages there might only be chat messages, some literary works, and vast amounts of machine-translated websites accessible for crawlers. I hope that in the future improved model architectures and training strategies can push the required amount of raw content way down.
pico_creator 3y ago
Yea, the map is poorly generated IMO as well - sadly there is only a few online tools online i could fine that "help me highlight, places where this list of languages is supported" So if we want a better map, we might need to redo from scratch That being said, depending on your sources, you can find multiple citations for 15-18% of the world population support english With approximately 25% being native speaker (english as first language) and 75% as non-native [PS: i realise later we were talking about different maps, the comment is for the 2nd map, for the languages we support]

Y_Y 3y ago

English is an official language in Malta and about 70% of the population is classed as "advanced" in English[0]. That only adds up to 280k people though, you'd probably find as many native-level speakers in any big European country, though not at the same density. [0] https://nso.gov.mt/wp-content/uploads/Skills-Preliminary.pdf https://nso.gov.mt/wp-content/uploads/Skills-Preliminary.pdf

pico_creator 3y ago

Thanks for flagging that out, swapped it out with another map I found, that shows different colors for different thresholds So its not "as strict"

viraptor 3y ago
That looks better. I think the original really was for native speakers only. Iceland is a nice quick test for that, since it should be >70% fluent in English, even with next to no native speakers. On the previous map it was almost completely white.