7 ms·

Natural language isn’t just English [pdf]

avmich 7y ago
It could be extremely useful to concentrate efforts on usage and details of one specific language rather than spread them over all different communication forms. The opposite could also be true.
zzo38computer 7y ago
It is correct, natural language isn't just English there is also many other kind too. But, the document is itself written in English, so we can guess that it is English. Still it doesn't help when you want to distinguish the natural language stuff dealing specifically English or in general (and if in general, you must be prepared to deal with stuff that is different from English, because they have their own features which are different from English).
briga 7y ago
Part of the problem seems to be logistical. English natural language training sets are probably just way more common than other languages. About the only language that has more training data is Chinese, and they seem to be doing fine in the NLP department. Aren't modern transformer networks able to deal with multiple languages? Seems like models powerful enough to do that should be fairly language-agnostic, assuming you can pull together enough training data for that language.
- zdragnar 7y ago
  My very limited understanding is that some interstitial language data is used to map between languages, but they are often somewhat "polluted" by the idioms or idiosyncrasies of whichever language you first start with to build it up. I seem to recall, for example, google translate having difficulty with the word "plane" between two non-english languages, accidentally confusing the word with two meanings from English. This was, iirc, because the internal representation of the words were originally generated by an English dataset, and so airplane in one non-english language became to level wood with a planer in the other non-english language (or something to that effect). Mandarin / Chinese has a comparatively simpler grammar (no verb conjugation) and no phonetic alphabet with arbitrarily varying phonetic rules, it is also chock full of idioms and homophobes. There is a lot that can be done, but the 80 / 20 rule is in full effect.
  - misterman0 7y ago
    >> idioms and homophobes Did you mean idiots?
    - the_duke 7y ago
      https://en.wikipedia.org/wiki/Homophone https://en.wikipedia.org/wiki/Homophone
      - Sharlin 7y ago
        I believe the GP was joking about great-GP's typoing of "homophones" as "homophobes" :)
  - yorwba 7y ago
    Another problem affecting Chinese NLP is that text doesn't come pre-segmented with spaces between words and segmentation errors can produce total nonsense. E.g. 我们很快会谈到这个的。[0] "We'll talk about this soon." Should be segmented like 我们(we)很(very)快(quick)会(will)谈(talk)到(get to)这个(this)的(emphasis)。Literally, "We will very quickly get to talk about this!" But 会谈 is also a noun meaning "negotiation", so the sentence could end up as "We are very quick negotiation gets to this!" which is still kind of comprehensible, but loses the future aspect, so it might as well be a comment about how quickly the negotiation has progressed. This kind of error affects all downstream tasks. Try doing named-entity recognition when names keep getting partially glued to surrounding words. Very annoying. [0] from https://tatoeba.org/eng/sentences/show/7781672 https://tatoeba.org/eng/sentences/show/7781672
    - duguxu 7y ago
      Another classical, interesting but evil example: 工信处女干事每月经过下属科室都要亲口交代24口交换机等技术性器件的安装工作 Literal translation of each character one by one: 工(industry)信(information)处(place)女(female)干(do)事(thing)每(every)月(month)经(through)过(pass)下(below)属(obey)科(division)室(room)都(always)要(would)亲(oneself)口(mouth)交(deliver/intersect)代(replace)24(24)口(mouth)交(deliver)换(exchange)机(machine)等(etc)技(ability)术(technique)性(nature/gender)器(device/organ)件(piece)的(of)安(install)装(costume)工(work)作(job) Correct segmentation: 工信处(ministry of industry and information)女(female)干事(secretary)每月(every month)经过(pass by)下属(subordinate)科室(department)都要(would always)亲口(from her own lips)交代(arrange)24口交换机(24-port switch)等(etc)技术性(technological)器件(device)的(of)安装(installation)工作(work) Every month when the female secretary of the ministry of industry and information passes by the subordinate department, she would always arrange in person the installation work of the 24-port switch and other technological devices. Vulgar segmentation: 工信处女(indoor female=virgin) 干事每月经(month through=menstruation) 过下属科室都要亲口交(mouth intersect=oral sex) 代24 口交(oral sex) 换机等技术性器(sex organ=genital) 件的安装工作 Obviously google translation passed the test!
natalyarostova 7y ago
Methods that work on English are an order of magnitude more important than others. There is a fair methodological point towards saying English=!natural language. But the economics of it simply make it the most important one to get right.
- rahulnair23 7y ago
  Fully agree on the economic merits of English here. I think the point is more than methodological though. It is one of equity and access. Broad swaths of society don't have the same access to the technology. My mother tongue Malayalam in the 1960's redefined its script to be better suited for typewriters. It is still non-trivial to typeset correctly. Like my Chinese colleagues, there are all sorts of constraints on how text is input. Machine translation beyond the "main" languages aren't too great. To see bias in action just try English->Turkish, a gender neutral language. "o multu. o mutsuz." translates to "he's happy. she is unhappy." Things built for English don't readily transfer. Its worth raising awareness around this as Emily does.
  - natalyarostova 7y ago
    I think people are aware, it’s just that English is clearly the most important. That’s just reality.
unishark 7y ago
A NLP researcher is supposed to describe how language is used, not insist on how it should be used.
- hibbelig 7y ago
  Emily's point is that NLP researchers tend to describe only how English is used, not how other languages are used.
  - unishark 7y ago
    That's like criticizing a biologist for only studying nematodes. Research is specialized by its nature. She does say a bit about directing research to other languages. But mostly she complains about researchers not identifying their research as English, including threats to heckle speakers who don't identify it as such, i.e. who don't use the spoken language in the way she prefers.
pjmlp 7y ago
Indeed, one of the reasons why I usually don't bother with natural language processing tools, as I want to use my own native language, instead of English or Brazilian Portuguese (if I am lucky enough for it to be supported).
elfexec 7y ago
Pages 4 and 5 ( "Languages of the world" ) of the pdf are duplicates. I agree that NL isn't just English, but the world of computer science, information processing, AI and even linguistics is heavily english-centric for a wide variety of reasons. And unless something drastic happens ( the end of pax americana and the fracturing of the world order ), everything will get even more english-centric. The internet is spreading all over the world. American culture is spreading all over the world. And so is american/english. I do wish CS and programming would be less english-centric. It doesn't make sense to me that people in china, russia, europe, india, etc are writing code in english. Why not translate computer languages to their native language? Given the nature of programming languages, it's so simple to do. Wouldn't it be simpler to translate C or python or java to chinese and have chinese students/programmer code in the chinese language ( mandarin )? Rather than having chinese students learn english and then learn to code in english?
- ben_w 7y ago
  What do you mean by “translate computer languages”? The reserved keywords, like ‘class’ or ‘if’? That might be easy, though I won’t be surprised if some natural languages will have multiple forms of those keywords depending on numbering or word-gender. The function and class names in the standard libraries, like UITableViewController or String.toUpperCase()? That’s harder, and would have to be manual to avoid the “water sheep” problem (hydraulic rams). Also: does Chinese have the concept of upper/lower case? Or do you mean the entire documentation, which is sometimes lacking useful details even when it’s only written in English? That’s not going to be possible automatically, and when it is, software developers will be obsolete because business speak could be translated into code without us.
  - int_19h 7y ago
    Translating programming languages was tried. Indeed, when ALGOL was originally designed, that was one of the reasons why its keywords were standalone "symbols", completely distinct from the letters they consist from. The idea was that a particular representation language could choose different representations for those "symbols" - and one obvious option was to use the translated words. Here's an example of an Algol-60 program in Russian representation: https://i.imgur.com/9edem3y.png https://i.imgur.com/9edem3y.png This was also done for Algol-68, and some Pascal dialects. But the practice has mostly died out once newsgroups and Internet became widely accessible. There's honestly very little upside to translating keywords and identifiers, because learning words is the easiest part of learning any language (grammar is much more complicated). And you don't actually have to learn the original meanings of words, either - you just need to know what they mean in programming context. In fact, not knowing the original meaning can be beneficial sometimes, because it can prevent poor understanding due to false equivalence or poor metaphors. But there's considerable downside in not being able to understand or reuse snippets from another language... TL;DR: we tried - it's not worth it. English FTW.
mamon 7y ago
I would jokingly say that English is halfway between natural languages and programming languages. I mean: English grammar is so simple and all sentences are so well structured that processing English is order of magnitude easier than, say, German.
- ben_w 7y ago
  Yeah right. Examples of well-recognised sarcastic phrases that would mess with simple sentiment analysis aside, I think the fact that it’s easy to think that English is simple and well-formed has probably held back serious development of chatbots etc. — the hard work of learning the real rules can be substituted with a simple grammatical approximation that gets a lot wrong and can’t be easily improved. Or, to put it another way: “Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo” is treated as a string of nouns, not [proper noun] [noun] [proper noun] [noun] [verb] [verb] [proper noun] [noun].
- gbear605 7y ago
  English is in fact not particularly simple, especially not if your only comparison is German. Both English and German can be modeled at approximately the same level of difficulty, with only a few minor differences. There is case assignment, but that’s simple compared to everything else (and English has deal with case too, if not to the same extent). The field of linguistics has been trying to fully model English syntax for the last 70 years and is still turning up new corner cases. And a lot of languages are in fact a lot simpler. To quote one of my bilingual friends after waking up from general anesthesia: > [I] couldn't put English in order so I just switched to Russian since I knew what syntactic roles I wanted, just not how to order words Latin, a supposedly more complex language, is another example of a language with much less strict syntax than English. And Google Translate is horrible at Latin. Mandarin, on the other hand, has a more complex grammar than English, but that doesn’t mean that it is more or less easy for NLP. After all, Google Translate is horrible at Mandarin.
the_decider 7y ago
This blog-post discusses the dangers of blindly applying English-optimized NLP models to other languages: https://primer.ai/blog/Russian-Natural-Language-Processing/ https://primer.ai/blog/Russian-Natural-Language-Processing/
- Veedrac 7y ago
  If anything this post disagrees with the claim that ‘English isn't generic for language’. It considers a naïve approach that fails for English, and shows that it fails for Russian in a similar way, and then compares it with an improvement also developed for English, which too helps Russian in a similar way. It is true that when Facebook trained their model “blindly”, this introduced meaningful bugs, so I agree with that claim from the post. It does not, however, give credence to the idea that techniques developed for English don't reliably work for other languages. Unfortunately the specific comparisons they make are suspect given how different the training sets were.

It could be extremely useful to concentrate efforts on usage and details of one specific language rather than spread them over all different communication forms. The opposite could also be true.

zzo38computer 7y ago

It is correct, natural language isn't just English there is also many other kind too. But, the document is itself written in English, so we can guess that it is English. Still it doesn't help when you want to distinguish the natural language stuff dealing specifically English or in general (and if in general, you must be prepared to deal with stuff that is different from English, because they have their own features which are different from English).

briga 7y ago

Part of the problem seems to be logistical. English natural language training sets are probably just way more common than other languages. About the only language that has more training data is Chinese, and they seem to be doing fine in the NLP department. Aren't modern transformer networks able to deal with multiple languages? Seems like models powerful enough to do that should be fairly language-agnostic, assuming you can pull together enough training data for that language.

zdragnar 7y ago
My very limited understanding is that some interstitial language data is used to map between languages, but they are often somewhat "polluted" by the idioms or idiosyncrasies of whichever language you first start with to build it up. I seem to recall, for example, google translate having difficulty with the word "plane" between two non-english languages, accidentally confusing the word with two meanings from English. This was, iirc, because the internal representation of the words were originally generated by an English dataset, and so airplane in one non-english language became to level wood with a planer in the other non-english language (or something to that effect). Mandarin / Chinese has a comparatively simpler grammar (no verb conjugation) and no phonetic alphabet with arbitrarily varying phonetic rules, it is also chock full of idioms and homophobes. There is a lot that can be done, but the 80 / 20 rule is in full effect.
- misterman0 7y ago
  >> idioms and homophobes Did you mean idiots?
  - the_duke 7y ago
    https://en.wikipedia.org/wiki/Homophone https://en.wikipedia.org/wiki/Homophone
    - Sharlin 7y ago
      I believe the GP was joking about great-GP's typoing of "homophones" as "homophobes" :)
- yorwba 7y ago
  Another problem affecting Chinese NLP is that text doesn't come pre-segmented with spaces between words and segmentation errors can produce total nonsense. E.g. 我们很快会谈到这个的。[0] "We'll talk about this soon." Should be segmented like 我们(we)很(very)快(quick)会(will)谈(talk)到(get to)这个(this)的(emphasis)。Literally, "We will very quickly get to talk about this!" But 会谈 is also a noun meaning "negotiation", so the sentence could end up as "We are very quick negotiation gets to this!" which is still kind of comprehensible, but loses the future aspect, so it might as well be a comment about how quickly the negotiation has progressed. This kind of error affects all downstream tasks. Try doing named-entity recognition when names keep getting partially glued to surrounding words. Very annoying. [0] from https://tatoeba.org/eng/sentences/show/7781672 https://tatoeba.org/eng/sentences/show/7781672
  - duguxu 7y ago
    Another classical, interesting but evil example: 工信处女干事每月经过下属科室都要亲口交代24口交换机等技术性器件的安装工作 Literal translation of each character one by one: 工(industry)信(information)处(place)女(female)干(do)事(thing)每(every)月(month)经(through)过(pass)下(below)属(obey)科(division)室(room)都(always)要(would)亲(oneself)口(mouth)交(deliver/intersect)代(replace)24(24)口(mouth)交(deliver)换(exchange)机(machine)等(etc)技(ability)术(technique)性(nature/gender)器(device/organ)件(piece)的(of)安(install)装(costume)工(work)作(job) Correct segmentation: 工信处(ministry of industry and information)女(female)干事(secretary)每月(every month)经过(pass by)下属(subordinate)科室(department)都要(would always)亲口(from her own lips)交代(arrange)24口交换机(24-port switch)等(etc)技术性(technological)器件(device)的(of)安装(installation)工作(work) Every month when the female secretary of the ministry of industry and information passes by the subordinate department, she would always arrange in person the installation work of the 24-port switch and other technological devices. Vulgar segmentation: 工信处女(indoor female=virgin) 干事每月经(month through=menstruation) 过下属科室都要亲口交(mouth intersect=oral sex) 代24 口交(oral sex) 换机等技术性器(sex organ=genital) 件的安装工作 Obviously google translation passed the test!

natalyarostova 7y ago

Methods that work on English are an order of magnitude more important than others. There is a fair methodological point towards saying English=!natural language. But the economics of it simply make it the most important one to get right.

rahulnair23 7y ago
Fully agree on the economic merits of English here. I think the point is more than methodological though. It is one of equity and access. Broad swaths of society don't have the same access to the technology. My mother tongue Malayalam in the 1960's redefined its script to be better suited for typewriters. It is still non-trivial to typeset correctly. Like my Chinese colleagues, there are all sorts of constraints on how text is input. Machine translation beyond the "main" languages aren't too great. To see bias in action just try English->Turkish, a gender neutral language. "o multu. o mutsuz." translates to "he's happy. she is unhappy." Things built for English don't readily transfer. Its worth raising awareness around this as Emily does.
- natalyarostova 7y ago
  I think people are aware, it’s just that English is clearly the most important. That’s just reality.

unishark 7y ago

A NLP researcher is supposed to describe how language is used, not insist on how it should be used.

hibbelig 7y ago
Emily's point is that NLP researchers tend to describe only how English is used, not how other languages are used.
- unishark 7y ago
  That's like criticizing a biologist for only studying nematodes. Research is specialized by its nature. She does say a bit about directing research to other languages. But mostly she complains about researchers not identifying their research as English, including threats to heckle speakers who don't identify it as such, i.e. who don't use the spoken language in the way she prefers.

pjmlp 7y ago

Indeed, one of the reasons why I usually don't bother with natural language processing tools, as I want to use my own native language, instead of English or Brazilian Portuguese (if I am lucky enough for it to be supported).

elfexec 7y ago

Pages 4 and 5 ( "Languages of the world" ) of the pdf are duplicates. I agree that NL isn't just English, but the world of computer science, information processing, AI and even linguistics is heavily english-centric for a wide variety of reasons. And unless something drastic happens ( the end of pax americana and the fracturing of the world order ), everything will get even more english-centric. The internet is spreading all over the world. American culture is spreading all over the world. And so is american/english. I do wish CS and programming would be less english-centric. It doesn't make sense to me that people in china, russia, europe, india, etc are writing code in english. Why not translate computer languages to their native language? Given the nature of programming languages, it's so simple to do. Wouldn't it be simpler to translate C or python or java to chinese and have chinese students/programmer code in the chinese language ( mandarin )? Rather than having chinese students learn english and then learn to code in english?

ben_w 7y ago
What do you mean by “translate computer languages”? The reserved keywords, like ‘class’ or ‘if’? That might be easy, though I won’t be surprised if some natural languages will have multiple forms of those keywords depending on numbering or word-gender. The function and class names in the standard libraries, like UITableViewController or String.toUpperCase()? That’s harder, and would have to be manual to avoid the “water sheep” problem (hydraulic rams). Also: does Chinese have the concept of upper/lower case? Or do you mean the entire documentation, which is sometimes lacking useful details even when it’s only written in English? That’s not going to be possible automatically, and when it is, software developers will be obsolete because business speak could be translated into code without us.
- int_19h 7y ago
  Translating programming languages was tried. Indeed, when ALGOL was originally designed, that was one of the reasons why its keywords were standalone "symbols", completely distinct from the letters they consist from. The idea was that a particular representation language could choose different representations for those "symbols" - and one obvious option was to use the translated words. Here's an example of an Algol-60 program in Russian representation: https://i.imgur.com/9edem3y.png https://i.imgur.com/9edem3y.png This was also done for Algol-68, and some Pascal dialects. But the practice has mostly died out once newsgroups and Internet became widely accessible. There's honestly very little upside to translating keywords and identifiers, because learning words is the easiest part of learning any language (grammar is much more complicated). And you don't actually have to learn the original meanings of words, either - you just need to know what they mean in programming context. In fact, not knowing the original meaning can be beneficial sometimes, because it can prevent poor understanding due to false equivalence or poor metaphors. But there's considerable downside in not being able to understand or reuse snippets from another language... TL;DR: we tried - it's not worth it. English FTW.

mamon 7y ago

I would jokingly say that English is halfway between natural languages and programming languages. I mean: English grammar is so simple and all sentences are so well structured that processing English is order of magnitude easier than, say, German.

ben_w 7y ago
Yeah right. Examples of well-recognised sarcastic phrases that would mess with simple sentiment analysis aside, I think the fact that it’s easy to think that English is simple and well-formed has probably held back serious development of chatbots etc. — the hard work of learning the real rules can be substituted with a simple grammatical approximation that gets a lot wrong and can’t be easily improved. Or, to put it another way: “Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo” is treated as a string of nouns, not [proper noun] [noun] [proper noun] [noun] [verb] [verb] [proper noun] [noun].
gbear605 7y ago
English is in fact not particularly simple, especially not if your only comparison is German. Both English and German can be modeled at approximately the same level of difficulty, with only a few minor differences. There is case assignment, but that’s simple compared to everything else (and English has deal with case too, if not to the same extent). The field of linguistics has been trying to fully model English syntax for the last 70 years and is still turning up new corner cases. And a lot of languages are in fact a lot simpler. To quote one of my bilingual friends after waking up from general anesthesia: > [I] couldn't put English in order so I just switched to Russian since I knew what syntactic roles I wanted, just not how to order words Latin, a supposedly more complex language, is another example of a language with much less strict syntax than English. And Google Translate is horrible at Latin. Mandarin, on the other hand, has a more complex grammar than English, but that doesn’t mean that it is more or less easy for NLP. After all, Google Translate is horrible at Mandarin.

the_decider 7y ago

This blog-post discusses the dangers of blindly applying English-optimized NLP models to other languages: https://primer.ai/blog/Russian-Natural-Language-Processing/ https://primer.ai/blog/Russian-Natural-Language-Processing/

Veedrac 7y ago
If anything this post disagrees with the claim that ‘English isn't generic for language’. It considers a naïve approach that fails for English, and shows that it fails for Russian in a similar way, and then compares it with an improvement also developed for English, which too helps Russian in a similar way. It is true that when Facebook trained their model “blindly”, this introduced meaningful bugs, so I agree with that claim from the post. It does not, however, give credence to the idea that techniques developed for English don't reliably work for other languages. Unfortunately the specific comparisons they make are suspect given how different the training sets were.