3 ms·

AI translation falls flat on its face with gendered languages, when it is given text in English and has to translate it into French, German, or Polish. How is t

AI translation falls flat on its face with gendered languages, when it is given text in English and has to translate it into French, German, or Polish. How is that even a consideration for a service that's supposed to be teaching people languages?

frabcus 3y ago
Google Translate and DeepL are pretty bad at this, that's true. As far as I can tell they translate a sentence at a time. Using an LLM and giving more context, it can infer gender from signals earlier in the text. This based on testing we did at the last place I worked. What kind of cases do you have gender failing, with which tools?
- madsbuch 3y ago
  > ... it can infer gender from signals earlier in the text Or even later in the text. Which is one of the great thing about transformers.
madsbuch 3y ago
Can you give an example of this? In my experience (learning Polish) ChatGPT can usually derive the gender when the subject is a common name: Sure, here are the translations of your sentences into Polish: "You are beautiful Natalia" translates to "Jesteś piękna Natalia." "You are beautiful Piotr" translates to "Jesteś piękny Piotr." piękn- is correctly genered.
- rideontime 3y ago
  "gendered languages" are languages where every noun has a gender, not just people. Polish is one of them, so I'm not sure where the confusion lies.
  - madsbuch 3y ago
    There is no confusion. I am just challenging that machine translation is incapable of inferring genders. Danish is also a gendered language. The genders are difficult to learn because there rarely are any rules to the genders. Ie, you can not derive it from the word, but only from the context it is used in or idiomatically. This is an area neural networks perform well in. Sure, here are the translations of your sentences into Polish: "This is a beautiful book" translates to "To jest piękna książka." "This is a beautiful table" translates to "To jest piękny stół."
    - tyfon 3y ago
      Speaking of Danish (and Norwegian), ChatGPT has quite large issues separating the two. Often it will insert random Danish words into the text when I write the prompt in Norwegian. The languages are quite similar when written, but it does this even when I specify that it should be in Norwegian.
- david_allison 3y ago
  [0] is a comprehensive list of issues with current machine translation [0] https://www.fairslator.com/fairslator-whitepaper.pdf https://www.fairslator.com/fairslator-whitepaper.pdf [PDF, 2022]
  - madsbuch 3y ago
    This publication is from february 2022 and can probably mostly be invalidated by modern software. First try on a translation: "Je suis étudiant(e). J'aime lire des livres et je suis très joli(e) en robe." (Note: "étudiant" is masculine, and "étudiante" is feminine. Similarly, "joli" is masculine, and "jolie" is feminine, so you can choose based on the context.) (A human translator would probably have inferred feminine by that fact that "I" looks good in a dress, ChatGPT has less bias) Second try: "Je suis étudiante. J'adore les longues promenades en forêt et lire des choses intéressantes. En particulier, j'aime lire Shakespeare. Oh, je me considère aussi comme une femme." ChatGPT infers from the last part of the sentence that "I" identify as a woman and corrects gendering in the entire sentence.
    - crotchfire 3y ago
      Dude you're coping pretty hard if you're trying to say that something less than two years old isn't modern.
      - madsbuch 3y ago
        did you read through the report? did you see the support I posted? That a report is less than 2 years old and in a fancy looking PDF does not make it correct. you'd be coping pretty hard to believe that.
- surfingdino 3y ago
  Only when there is a common female name. Not when there isn't one.
  - madsbuch 3y ago
    You can see the other examples posted in the sibling threads. ChatGPT handles gendered nouns and correctly deriving genders for articles from context.
savanaly 3y ago
I'd be really interested in an example. Preferably a link to the actual conversation with GPT4 on ChatGPT or equivalent on another service, but even just typing out the sentence here with the translation you think is correct vs the translation you expect to receive would be helpful.
- surfingdino 3y ago
  But that's not a realistic scenario for a learner who doesn't know what the translation ought to look like, because he/she is still learning the language.

frabcus 3y ago

Google Translate and DeepL are pretty bad at this, that's true. As far as I can tell they translate a sentence at a time. Using an LLM and giving more context, it can infer gender from signals earlier in the text. This based on testing we did at the last place I worked. What kind of cases do you have gender failing, with which tools?

madsbuch 3y ago
> ... it can infer gender from signals earlier in the text Or even later in the text. Which is one of the great thing about transformers.

madsbuch 3y ago

Can you give an example of this? In my experience (learning Polish) ChatGPT can usually derive the gender when the subject is a common name: Sure, here are the translations of your sentences into Polish: "You are beautiful Natalia" translates to "Jesteś piękna Natalia." "You are beautiful Piotr" translates to "Jesteś piękny Piotr." piękn- is correctly genered.

rideontime 3y ago
"gendered languages" are languages where every noun has a gender, not just people. Polish is one of them, so I'm not sure where the confusion lies.
- madsbuch 3y ago
  There is no confusion. I am just challenging that machine translation is incapable of inferring genders. Danish is also a gendered language. The genders are difficult to learn because there rarely are any rules to the genders. Ie, you can not derive it from the word, but only from the context it is used in or idiomatically. This is an area neural networks perform well in. Sure, here are the translations of your sentences into Polish: "This is a beautiful book" translates to "To jest piękna książka." "This is a beautiful table" translates to "To jest piękny stół."
  - tyfon 3y ago
    Speaking of Danish (and Norwegian), ChatGPT has quite large issues separating the two. Often it will insert random Danish words into the text when I write the prompt in Norwegian. The languages are quite similar when written, but it does this even when I specify that it should be in Norwegian.
david_allison 3y ago
[0] is a comprehensive list of issues with current machine translation [0] https://www.fairslator.com/fairslator-whitepaper.pdf https://www.fairslator.com/fairslator-whitepaper.pdf [PDF, 2022]
- madsbuch 3y ago
  This publication is from february 2022 and can probably mostly be invalidated by modern software. First try on a translation: "Je suis étudiant(e). J'aime lire des livres et je suis très joli(e) en robe." (Note: "étudiant" is masculine, and "étudiante" is feminine. Similarly, "joli" is masculine, and "jolie" is feminine, so you can choose based on the context.) (A human translator would probably have inferred feminine by that fact that "I" looks good in a dress, ChatGPT has less bias) Second try: "Je suis étudiante. J'adore les longues promenades en forêt et lire des choses intéressantes. En particulier, j'aime lire Shakespeare. Oh, je me considère aussi comme une femme." ChatGPT infers from the last part of the sentence that "I" identify as a woman and corrects gendering in the entire sentence.
  - crotchfire 3y ago
    Dude you're coping pretty hard if you're trying to say that something less than two years old isn't modern.
    - madsbuch 3y ago
      did you read through the report? did you see the support I posted? That a report is less than 2 years old and in a fancy looking PDF does not make it correct. you'd be coping pretty hard to believe that.
surfingdino 3y ago
Only when there is a common female name. Not when there isn't one.
- madsbuch 3y ago
  You can see the other examples posted in the sibling threads. ChatGPT handles gendered nouns and correctly deriving genders for articles from context.

savanaly 3y ago

I'd be really interested in an example. Preferably a link to the actual conversation with GPT4 on ChatGPT or equivalent on another service, but even just typing out the sentence here with the translation you think is correct vs the translation you expect to receive would be helpful.

surfingdino 3y ago
But that's not a realistic scenario for a learner who doesn't know what the translation ought to look like, because he/she is still learning the language.