5 ms·

Will be interesting to see how it performs in the wild. Here's a sample sentence that I've never seen any automatic translator get right: "My cousin and her w

by GuiA 9y ago

Will be interesting to see how it performs in the wild. Here's a sample sentence that I've never seen any automatic translator get right: "My cousin and her wife" Any human would infer from the context that my cousin is a woman, in a same sex marriage. Yet Google, who also uses fancy deep learning, gives us, for Spanish: "Mi primo y su esposa" This should be "Mi prima y su esposa". And that's just for Spanish, a language relatively close to English. With more convoluted examples and more distant languages, it still breaks down pretty fast. See also: examples of gender bias when using deep learning to translate from languages with gender neutral pronouns: https://twitter.com/amuellerml/status/799658925326995456 https://twitter.com/amuellerml/status/799658925326995456 https://twitter.com/nobody_indepth/status/799700696572526592 https://twitter.com/nobody_indepth/status/799700696572526592

tmsldd 9y ago
using google translation to German gives "meine Cousine und ihre Frau".. as you cited it might work just because English/German are close languages.. But, yeah, DL is just a fancy (and incomplete) mapping.
- justicezyx 9y ago
  Is there anything isn't fancy mapping? Use that term to down play the effectiveness of the approach seems quite surprising. Consider human needs years of study to do the same thing, which usually also makes mistakes. Even for the example, without much analysis, it's very easy to miss the part.
  - tmsldd 9y ago
    Don't get me wrong... but If it makes you feel better, I use DL for most of my research / projects. Well, after 7 years dealing with DL I couldn't really find a better single word to describe it.
    - visarga 9y ago
      How's "Fancy context dependent matching"? More dignified, right /s In this case the context dependence part wasn't up to par, so it was just fancy mapping.
deleted 9y ago
[deleted]
toomanybeersies 9y ago
I assume this is a case of implicit bias from the training materials used.
gok 9y ago
This is an excellent example where phrase-based statistical machine translation is troublesome; the language model score of "his wife" is going to be dramatically larger "her wife" in any world where there are a lot more hereto married people. Neural MT should have a much better shot at getting it.
schoen 9y ago
I've been looking a little bit at anaphora resolution in AI and there's the trouble that your phrase could appear in many contexts where Google's translation would be completely correct! If "cousin" is the antecedent of "her" (which is the only possibility if this phrase occurs in isolation), then Google's translation is clearly wrong. But if there is a preceding or following sentence which mentions a third person, Google's translation can be correct because "her" can then refer to that person. For example "my cousin and her wife think that Sarah has good taste in ice cream": here one likely anaphora resolution is "my cousin and Sarah's wife think that Sarah has good taste in ice cream". Or "when Sarah got married, she invited my cousin to the wedding; my cousin and her wife turned out to have gone to college together" (again "my cousin and Sarah's wife turned out to have gone to college together"). Anaphora resolution in general is one of the hardest problems for machine translation because it appears to require so much knowledge about the world to do it as well as human beings do. But also, different resolutions can be correct (or maximum-probability) in different contexts depending on the additional information! For instance, there's the Winograd Schema structure where a single pronoun would be interpreted as referring to different people depending on the surrounding context (but not grammar). Winograd's classic example was The city council members refused the demonstrators a permit because they feared violence. The city council members refused the demonstrators a permit because they advocated violence. Disturbingly for machine translation, in the former sentence "they" refers to "the city council members", while in the latter sentence "they" refers to "the demonstrators", even though the syntax of the two sentences is identical! This, in turn, means that if a translation task required knowing the antecedent of "they" in "The city council members refused the demonstrators a permit because they", the translation task would have no unique solution because the antecedent is ambiguous. Formally this is also true of every reference to, for example, family members when one language marks gender and another doesn't, even if there is a likely resolution offered by the local context, as in your sentence. There is no unique translation available. Finding the one intended by the speaker will require more context, while even finding the one that other speakers find most probable with limited contexts is sometimes among the most challenges AI problems today.
erik14th 9y ago
Works from english to portuguese in google, maybe because they have more data? Considering this example alone, the logical difference between english/spanish and english/portuguese are pretty much the same: gender is made explicit in the pronoun in spanish/portuguese(primo/prima) in contrast with english that relies on an adjective(her). About ambiguity, biases, well, what the "machine" learns are pretty much biases, meaning it's biased by the data it was trained on.
hasenj 9y ago
Actually as a human I did not immediately infer the lesbian relationship part.
- deleted 9y ago
  [deleted]

by GuiA 9y ago

tmsldd 9y ago

using google translation to German gives "meine Cousine und ihre Frau".. as you cited it might work just because English/German are close languages.. But, yeah, DL is just a fancy (and incomplete) mapping.

justicezyx 9y ago
Is there anything isn't fancy mapping? Use that term to down play the effectiveness of the approach seems quite surprising. Consider human needs years of study to do the same thing, which usually also makes mistakes. Even for the example, without much analysis, it's very easy to miss the part.
- tmsldd 9y ago
  Don't get me wrong... but If it makes you feel better, I use DL for most of my research / projects. Well, after 7 years dealing with DL I couldn't really find a better single word to describe it.
  - visarga 9y ago
    How's "Fancy context dependent matching"? More dignified, right /s In this case the context dependence part wasn't up to par, so it was just fancy mapping.

deleted 9y ago

[deleted]

toomanybeersies 9y ago

I assume this is a case of implicit bias from the training materials used.

gok 9y ago

This is an excellent example where phrase-based statistical machine translation is troublesome; the language model score of "his wife" is going to be dramatically larger "her wife" in any world where there are a lot more hereto married people. Neural MT should have a much better shot at getting it.

schoen 9y ago

I've been looking a little bit at anaphora resolution in AI and there's the trouble that your phrase could appear in many contexts where Google's translation would be completely correct! If "cousin" is the antecedent of "her" (which is the only possibility if this phrase occurs in isolation), then Google's translation is clearly wrong. But if there is a preceding or following sentence which mentions a third person, Google's translation can be correct because "her" can then refer to that person. For example "my cousin and her wife think that Sarah has good taste in ice cream": here one likely anaphora resolution is "my cousin and Sarah's wife think that Sarah has good taste in ice cream". Or "when Sarah got married, she invited my cousin to the wedding; my cousin and her wife turned out to have gone to college together" (again "my cousin and Sarah's wife turned out to have gone to college together"). Anaphora resolution in general is one of the hardest problems for machine translation because it appears to require so much knowledge about the world to do it as well as human beings do. But also, different resolutions can be correct (or maximum-probability) in different contexts depending on the additional information! For instance, there's the Winograd Schema structure where a single pronoun would be interpreted as referring to different people depending on the surrounding context (but not grammar). Winograd's classic example was The city council members refused the demonstrators a permit because they feared violence. The city council members refused the demonstrators a permit because they advocated violence. Disturbingly for machine translation, in the former sentence "they" refers to "the city council members", while in the latter sentence "they" refers to "the demonstrators", even though the syntax of the two sentences is identical! This, in turn, means that if a translation task required knowing the antecedent of "they" in "The city council members refused the demonstrators a permit because they", the translation task would have no unique solution because the antecedent is ambiguous. Formally this is also true of every reference to, for example, family members when one language marks gender and another doesn't, even if there is a likely resolution offered by the local context, as in your sentence. There is no unique translation available. Finding the one intended by the speaker will require more context, while even finding the one that other speakers find most probable with limited contexts is sometimes among the most challenges AI problems today.

erik14th 9y ago

Works from english to portuguese in google, maybe because they have more data? Considering this example alone, the logical difference between english/spanish and english/portuguese are pretty much the same: gender is made explicit in the pronoun in spanish/portuguese(primo/prima) in contrast with english that relies on an adjective(her). About ambiguity, biases, well, what the "machine" learns are pretty much biases, meaning it's biased by the data it was trained on.

hasenj 9y ago

Actually as a human I did not immediately infer the lesbian relationship part.

deleted 9y ago
[deleted]