4 ms·

Slight side tangent but reading this article it hit me how much this generation of work may be reinforcing English as the global language for generations to com

Slight side tangent but reading this article it hit me how much this generation of work may be reinforcing English as the global language for generations to come. It seems like we are headed towards a phase of technology where learning how to feed well-crafted prompts into the AI system will be a highly valuable productivity skill. And since the major AI systems seem to be built around English, that would make English fluency even more valuable than it already is. I’m sure that’s obvious to non-native speakers who have worked hard to master English, I just hadn’t thought of it before. Less likely but still interesting, I wonder if the way we’re building these models will at some point begin to layer on top of each other such that English as it is used now becomes something deeply embedded in AI, and whether that will evolve with the spoken language or not. It’s funny to imagine a future where people would need to master an archaic flavor of English to get the best results working with their AI helpers.

affgrff2 4y ago
But the same work is also removing the language barriers at the same time with really good translation tools. I rather guess being fluent in English will be not as important as it is now. Edit: the same work = transformer based language models
mtlmtlmtlmtl 4y ago
Also worth noting that the internet has massively accelerated the importance of English already. As an ESL speaker who grew up on the internet, Norwegian was more or less useless to me outside school and family. Most of my time was spent on the internet, reading and writing lots of English. Norwegian wikipedia is pretty much useless unless you don't know English. That's still true today for the vast majority of articles, but back then was universally the case. There were Norwegian forums, but with a population of just 4 million and change at the time, they were never as interesting or active as international/American forums and IRC channels. In fact I'd say Norwegian is only my native language in spoken form, whereas English feels more natural to me to write and read. Doesn't help that Norwegian has two very divergent written forms, either. I even write my private notes in English, even though I will be the only one reading them.
Ajedi32 4y ago
Perhaps, but another possibility is that the more advanced models all end up being polyglots. The state of the art in machine translation already uses a single model trained on multiple languages[1], which results in better translations between languages it doesn't have a lot of examples for. If the same principle applies to other types of models, then training them on every possible dataset available regardless of language might yield better results. That could result in models that are fluent in hundreds of languages. (I'd be curious as to whether DALL-E understands prompts in languages other than English, has anyone tried?) [1]: https://ai.googleblog.com/2019/10/exploring-massively-multilingual.html https://ai.googleblog.com/2019/10/exploring-massively-multil...
- visarga 4y ago
  BLOOM is the most recent model trained on multi-lingual data - 46 natural languages and 13 programming languages.
- Ajedi32 4y ago
  Found this prompt for DALL-E: https://labs.openai.com/s/WzRG0r1vMVvqi0MYBY489OZi https://labs.openai.com/s/WzRG0r1vMVvqi0MYBY489OZi It clearly understands at least basic Spanish. Not sure how it'd handle more complex phrases or other, more obscure languages.
planede 4y ago
I wonder if these models performed better if they were trained on an other language.
- qayxc 4y ago
  I wonder the same thing. Many languages are far better at making precise statements due to stricter and more complex grammar. The reduction in ambiguity could help models perform better. Then there's also the problem of labelling training data. Most of the labelling and annotating is outsourced to countries with cheap labour and performed by non-native speakers, which leads to problems with mis-labelled training data.
  - visarga 4y ago
    DallE is trained on web scraped data, there is no manual labelling taking place except maybe to filter out some types of content. A few years ago we were training captioning models on manually labelled datasets (such as COCO captions), but the they were small and models were not too general.
ffhhj 4y ago
> reinforcing English as the global language Which might also result in new speakers modifying English to their cultures (like Blade Runner's Cityspeak), and Global-English speakers not understanding "secret" foreign communication, so they might create new languages for their own subcultures, then relegating English as the new Latin for technical knowledge (Latin was kept by the Catholic Church).
bearjaws 4y ago
Covid and remote work are going to make this even more true. At my new job I work with a Nigerian co-worker who lives in Nigeria, he speaks clear & proper English and earns an amazing salary for his area. If you live in a third world country, you could really benefit from remote work going forward and English will be a popular language to learn for that. That being said, I know some people will 'phone it in' and not speak as clearly, which will put them at a disadvantage.
- mfkp 4y ago
  The official language of Nigeria is English, so it's not much of a surprise that your co-worker speaks fluent English.
Workaccount2 4y ago
I envision the skill of giving good AI prompts to be as short lived as the skill of T9 texting. Probably even shorter.
- TillE 4y ago
  Right, I would expect the rise of a bunch of domain-specific tools within the next few years.

affgrff2 4y ago

But the same work is also removing the language barriers at the same time with really good translation tools. I rather guess being fluent in English will be not as important as it is now. Edit: the same work = transformer based language models

mtlmtlmtlmtl 4y ago

Also worth noting that the internet has massively accelerated the importance of English already. As an ESL speaker who grew up on the internet, Norwegian was more or less useless to me outside school and family. Most of my time was spent on the internet, reading and writing lots of English. Norwegian wikipedia is pretty much useless unless you don't know English. That's still true today for the vast majority of articles, but back then was universally the case. There were Norwegian forums, but with a population of just 4 million and change at the time, they were never as interesting or active as international/American forums and IRC channels. In fact I'd say Norwegian is only my native language in spoken form, whereas English feels more natural to me to write and read. Doesn't help that Norwegian has two very divergent written forms, either. I even write my private notes in English, even though I will be the only one reading them.

Ajedi32 4y ago

Perhaps, but another possibility is that the more advanced models all end up being polyglots. The state of the art in machine translation already uses a single model trained on multiple languages[1], which results in better translations between languages it doesn't have a lot of examples for. If the same principle applies to other types of models, then training them on every possible dataset available regardless of language might yield better results. That could result in models that are fluent in hundreds of languages. (I'd be curious as to whether DALL-E understands prompts in languages other than English, has anyone tried?) [1]: https://ai.googleblog.com/2019/10/exploring-massively-multilingual.html https://ai.googleblog.com/2019/10/exploring-massively-multil...

visarga 4y ago
BLOOM is the most recent model trained on multi-lingual data - 46 natural languages and 13 programming languages.
Ajedi32 4y ago
Found this prompt for DALL-E: https://labs.openai.com/s/WzRG0r1vMVvqi0MYBY489OZi https://labs.openai.com/s/WzRG0r1vMVvqi0MYBY489OZi It clearly understands at least basic Spanish. Not sure how it'd handle more complex phrases or other, more obscure languages.

planede 4y ago

I wonder if these models performed better if they were trained on an other language.

qayxc 4y ago
I wonder the same thing. Many languages are far better at making precise statements due to stricter and more complex grammar. The reduction in ambiguity could help models perform better. Then there's also the problem of labelling training data. Most of the labelling and annotating is outsourced to countries with cheap labour and performed by non-native speakers, which leads to problems with mis-labelled training data.
- visarga 4y ago
  DallE is trained on web scraped data, there is no manual labelling taking place except maybe to filter out some types of content. A few years ago we were training captioning models on manually labelled datasets (such as COCO captions), but the they were small and models were not too general.

ffhhj 4y ago

> reinforcing English as the global language Which might also result in new speakers modifying English to their cultures (like Blade Runner's Cityspeak), and Global-English speakers not understanding "secret" foreign communication, so they might create new languages for their own subcultures, then relegating English as the new Latin for technical knowledge (Latin was kept by the Catholic Church).

bearjaws 4y ago

Covid and remote work are going to make this even more true. At my new job I work with a Nigerian co-worker who lives in Nigeria, he speaks clear & proper English and earns an amazing salary for his area. If you live in a third world country, you could really benefit from remote work going forward and English will be a popular language to learn for that. That being said, I know some people will 'phone it in' and not speak as clearly, which will put them at a disadvantage.

mfkp 4y ago
The official language of Nigeria is English, so it's not much of a surprise that your co-worker speaks fluent English.

Workaccount2 4y ago

I envision the skill of giving good AI prompts to be as short lived as the skill of T9 texting. Probably even shorter.

TillE 4y ago
Right, I would expect the rise of a bunch of domain-specific tools within the next few years.