4 ms·

Hungarian has no gendered pronouns, but English has them. The text has been translated from Hungarian to English. What would be the "correct" (or "non-biased")

Hungarian has no gendered pronouns, but English has them. The text has been translated from Hungarian to English. What would be the "correct" (or "non-biased") way for Google Translate to handle this scenario? Should all pronouns in the translated text be either a "he", a "she" or a "they"? Wouldn't that still be considered "biased"?

hprotagonist 6y ago
“he”, “they”, or “one” can be neuter. Each has its pros and cons, because words both denote and connote things. “he” might read as sexist or old-fashioned; “they” does double duty for plural and singular; “one” practically drips with starched-collar stiffness. “correct translation” is a matter of taste. You’re creating a new work, after all, so there’s some authorial contribution.
- csmpltn 6y ago
  > “correct translation” is a matter of taste So, bias no matter what then?
  - hprotagonist 6y ago
    Bias no matter what. It’s unavoidable; languages are ambiguous and you are always, always going to leak out of band information to your audience. (by word choice, accent, font, punctuation, ... )
    - csmpltn 6y ago
      Makes sense to me. I wonder what's the point behind this entire Reddit thread then. More fuel for the fire?
      - hprotagonist 6y ago
        Well, just because there's always _a_ bias doesn't mean you can't influence which one your machine translation system favors. I'm reminded of one of the early "oh, wow" moments with word2vec (a word embedding scheme), where the statement "King - Man + Woman" evaluates to "Queen". This is not a measure that the system understands anything, just that there are really strong statistical correlations that are preserved in the embedding that let you see how close vectors are to one another. Necessarily, this sort of vector similarity or word choice probability is a function of (at least) your training data. If you want to stop over-weighting pronoun choice for adjectives like "pretty" and "weak" or "emotional" to be feminine, be more choosy about your training set and don't just snarf up reddit willy-nilly and go brrrrr.
        csmpltn 6y ago
        > be more choosy about your training set and don't just snarf up reddit willy-nilly and go brrrrr That's a much more interesting (and complex) conversation. Siphoning raw Reddit/Wikipedia is the closest you'll get to an honest snapshot of our internet-based society (for all of its tremendous flaws). One could make the case that we don't need an "honest snapshot of our internet-based society" to build a Translation app. But that's debatable... Being "more choosy" will inherently introduce a different set of biases. Perhaps those biases align more closely with our core beliefs and values - but they serve as bias nontheless.
        hprotagonist 6y ago
        No disagreement here! One thing that will absolutely raise some people's hackles is pointing out that "intentionally modulating biases in translation techniques" already has a name, and it is "being prescriptive about language usage". I've found that generally, tech/progressive/machine learning ethics folks tend to believe that prescriptivism is the domain of, for example, stuffy old people like Strunk and White, how dare they tell me what is "proper" -- and then turn right around and do just the same with a straight face but don't seem to really realize that that's what they're doing. At any rate, designers of such systems do need to face head on the fact that machine translation systems at scale both 1. need to reflect usage in the world in a descriptive enough way as to be useful and honest and provide coherent approximations of translation, even sometimes if it's reflecting perhaps unpleasant truths about ourselves we might not like to think about and 2. need to realize that their participation and choices of embeddings, etc., contribute to the corpus of texts in the world as well as reflect them, so their prescriptive choices are also real and will have feedback effects.
coffeefirst 6y ago
Yes. And because of that, the AI guesses based on the relations it sees most often. The HN title "Google translate is biased" is misleading, but it's a fascinating example of how AI can encode bias. It probably doesn't matter if I'm using Google Translate to make hotel reservations when traveling. It matters a lot if you're going to use AI to scan resumes.
- mempko 6y ago
  It doesn't just encode bias but perpetuates it. So yes, the AI is biased.
  - fakedang 6y ago
    It's not the AI's inherent bias as such though. It's the bias of the dataset which has more cases of women cooking and men being professors in Hungarian texts.
    - Jasper_ 6y ago
      So AI is taking over the world, and we should just accept that it has severe limitations of just parroting the training data like a Markov chain? This is the algorithm we think can solve everything?
      - csmpltn 6y ago
        > So AI is taking over the world, and we should just accept that it has severe limitations of just parroting the training data like a Markov chain? This is the algorithm we think can solve everything? "No", to both questions. These systems are a work-in-progress, constantly improving. They have value, even though they are far from perfect.
    - BoiledCabbage 6y ago
      > It's not the AI's inherent bias as such though. It's the bias of the dataset... An engineer builds a chat protocol and app on top of UDP. Chat messages between users frequently never arrive and are lost. When questioned about it, the engineer says "it's not my app that causing the message loss it's UDP that's causing it." The engineer selected the communication protocol. To avoid packet loss, I build retry logic into my app, or build on top of TCP.
riffraff 6y ago
google translate actually handles this fine if you use a single sentence, showing both interpretations, e.g. "ö szép" shows "Translations are gender-specific. LEARN MORE" and both versions "he is beautiful" "she is beautiful" In the past this wasn't the case, AFAIR. I guess they couldn't just patch a workaround for everything. I'm also unsure if they should: let's say you want to translate a bit from the gospel "He broke the bread... gave it to His disciples... and He said..". If you're translating from hungarian, would you have this translated as "he/she" ? It seems more correct to use the original "He" since you can infer from context what it is about. At the same time, it's clear there is always bias in the training material, and probably "recover original text" is useless.
slibhb 6y ago
> Should all pronouns in the translated text be either a "he", a "she" or a "they"? Wouldn't that still be considered "biased"? The proffered solution is to use singular "they" of course. Trouble is, people don't talk like that and so from a language-is-as-it-used perspective (descriptivist), always using singular "they" makes your translation worse. I would offer a solution like this: allow the person using the translation program to specify pronoun rules.
- csmpltn 6y ago
  > The proffered solution is to use singular "they" of course That's subjective. > allow the person using the translation program to specify pronoun rules "Translation is the communication of the meaning of a source–language text by means of an equivalent target-language text" (https://en.wikipedia.org/wiki/Translation https://en.wikipedia.org/wiki/Translation) Allowing the end-user to customize the translation according to their own personal biases introduces an entirely different set of issues. We should no longer call it a "translation" at that point.
  - slibhb 6y ago
    > That's subjective. I don't follow. There is an argument out there that we should stop using he/she and default to they (unless informed otherwise). Am I wrong about that? I didn't say I agree with it, just that it is a proposed solution. > Allowing the end-user to customize the translation according to their own personal biases introduces an entirely different set of issues... Yes, and allowing people to speak introduces this exact same issue...
    - csmpltn 6y ago
      > Yes, and allowing people to speak introduces this exact same issue... It's not the purpose of a translation service - to let people customize the translation such that it fits their own personal biases and sits more comfortably with their world views.
      - 6y ago
bloak 6y ago
I think a helpful human translator might use singular "they" in cases where the sex can probably not be known (for example, if it's a reference to a hypothetical person) and there is little danger of confusion with plural "they", and "he/she" in other cases (for example if the person requesting the translation might know the sex and be able to fill it in for themselves). Those criteria also align roughly with the acceptability of singular "they" in modern British English: a sentence like "everyone should mind their own business" is totally normal and standard and always has been; a sentence like "the unidentified miscreant left their cup on the table" will seem a bit strange to some people; a sentence like "Chris ate their lunch" will seem strange to a lot of people. I wouldn't expect Google Translate to be capable of such cleverness so a simpler solution would be needed.
commandlinefan 6y ago
It seems to me that Google translate is just playing the odds. It doesn't see "he is beautiful" very often (if at all), so it has two possible outputs: "she is beatiful" or "he is beautiful" and goes with the most common one, as it not only should, but must.

hprotagonist 6y ago

“he”, “they”, or “one” can be neuter. Each has its pros and cons, because words both denote and connote things. “he” might read as sexist or old-fashioned; “they” does double duty for plural and singular; “one” practically drips with starched-collar stiffness. “correct translation” is a matter of taste. You’re creating a new work, after all, so there’s some authorial contribution.

csmpltn 6y ago
> “correct translation” is a matter of taste So, bias no matter what then?
- hprotagonist 6y ago
  Bias no matter what. It’s unavoidable; languages are ambiguous and you are always, always going to leak out of band information to your audience. (by word choice, accent, font, punctuation, ... )
  - csmpltn 6y ago
    Makes sense to me. I wonder what's the point behind this entire Reddit thread then. More fuel for the fire?
    - hprotagonist 6y ago
      Well, just because there's always _a_ bias doesn't mean you can't influence which one your machine translation system favors. I'm reminded of one of the early "oh, wow" moments with word2vec (a word embedding scheme), where the statement "King - Man + Woman" evaluates to "Queen". This is not a measure that the system understands anything, just that there are really strong statistical correlations that are preserved in the embedding that let you see how close vectors are to one another. Necessarily, this sort of vector similarity or word choice probability is a function of (at least) your training data. If you want to stop over-weighting pronoun choice for adjectives like "pretty" and "weak" or "emotional" to be feminine, be more choosy about your training set and don't just snarf up reddit willy-nilly and go brrrrr.
      - csmpltn 6y ago
        > be more choosy about your training set and don't just snarf up reddit willy-nilly and go brrrrr That's a much more interesting (and complex) conversation. Siphoning raw Reddit/Wikipedia is the closest you'll get to an honest snapshot of our internet-based society (for all of its tremendous flaws). One could make the case that we don't need an "honest snapshot of our internet-based society" to build a Translation app. But that's debatable... Being "more choosy" will inherently introduce a different set of biases. Perhaps those biases align more closely with our core beliefs and values - but they serve as bias nontheless.
        hprotagonist 6y ago
        No disagreement here! One thing that will absolutely raise some people's hackles is pointing out that "intentionally modulating biases in translation techniques" already has a name, and it is "being prescriptive about language usage". I've found that generally, tech/progressive/machine learning ethics folks tend to believe that prescriptivism is the domain of, for example, stuffy old people like Strunk and White, how dare they tell me what is "proper" -- and then turn right around and do just the same with a straight face but don't seem to really realize that that's what they're doing. At any rate, designers of such systems do need to face head on the fact that machine translation systems at scale both 1. need to reflect usage in the world in a descriptive enough way as to be useful and honest and provide coherent approximations of translation, even sometimes if it's reflecting perhaps unpleasant truths about ourselves we might not like to think about and 2. need to realize that their participation and choices of embeddings, etc., contribute to the corpus of texts in the world as well as reflect them, so their prescriptive choices are also real and will have feedback effects.

coffeefirst 6y ago

Yes. And because of that, the AI guesses based on the relations it sees most often. The HN title "Google translate is biased" is misleading, but it's a fascinating example of how AI can encode bias. It probably doesn't matter if I'm using Google Translate to make hotel reservations when traveling. It matters a lot if you're going to use AI to scan resumes.

mempko 6y ago
It doesn't just encode bias but perpetuates it. So yes, the AI is biased.
- fakedang 6y ago
  It's not the AI's inherent bias as such though. It's the bias of the dataset which has more cases of women cooking and men being professors in Hungarian texts.
  - Jasper_ 6y ago
    So AI is taking over the world, and we should just accept that it has severe limitations of just parroting the training data like a Markov chain? This is the algorithm we think can solve everything?
    - csmpltn 6y ago
      > So AI is taking over the world, and we should just accept that it has severe limitations of just parroting the training data like a Markov chain? This is the algorithm we think can solve everything? "No", to both questions. These systems are a work-in-progress, constantly improving. They have value, even though they are far from perfect.
  - BoiledCabbage 6y ago
    > It's not the AI's inherent bias as such though. It's the bias of the dataset... An engineer builds a chat protocol and app on top of UDP. Chat messages between users frequently never arrive and are lost. When questioned about it, the engineer says "it's not my app that causing the message loss it's UDP that's causing it." The engineer selected the communication protocol. To avoid packet loss, I build retry logic into my app, or build on top of TCP.

riffraff 6y ago

google translate actually handles this fine if you use a single sentence, showing both interpretations, e.g. "ö szép" shows "Translations are gender-specific. LEARN MORE" and both versions "he is beautiful" "she is beautiful" In the past this wasn't the case, AFAIR. I guess they couldn't just patch a workaround for everything. I'm also unsure if they should: let's say you want to translate a bit from the gospel "He broke the bread... gave it to His disciples... and He said..". If you're translating from hungarian, would you have this translated as "he/she" ? It seems more correct to use the original "He" since you can infer from context what it is about. At the same time, it's clear there is always bias in the training material, and probably "recover original text" is useless.

slibhb 6y ago

> Should all pronouns in the translated text be either a "he", a "she" or a "they"? Wouldn't that still be considered "biased"? The proffered solution is to use singular "they" of course. Trouble is, people don't talk like that and so from a language-is-as-it-used perspective (descriptivist), always using singular "they" makes your translation worse. I would offer a solution like this: allow the person using the translation program to specify pronoun rules.

csmpltn 6y ago
> The proffered solution is to use singular "they" of course That's subjective. > allow the person using the translation program to specify pronoun rules "Translation is the communication of the meaning of a source–language text by means of an equivalent target-language text" (https://en.wikipedia.org/wiki/Translation https://en.wikipedia.org/wiki/Translation) Allowing the end-user to customize the translation according to their own personal biases introduces an entirely different set of issues. We should no longer call it a "translation" at that point.
- slibhb 6y ago
  > That's subjective. I don't follow. There is an argument out there that we should stop using he/she and default to they (unless informed otherwise). Am I wrong about that? I didn't say I agree with it, just that it is a proposed solution. > Allowing the end-user to customize the translation according to their own personal biases introduces an entirely different set of issues... Yes, and allowing people to speak introduces this exact same issue...
  - csmpltn 6y ago
    > Yes, and allowing people to speak introduces this exact same issue... It's not the purpose of a translation service - to let people customize the translation such that it fits their own personal biases and sits more comfortably with their world views.
    - 6y ago

bloak 6y ago

I think a helpful human translator might use singular "they" in cases where the sex can probably not be known (for example, if it's a reference to a hypothetical person) and there is little danger of confusion with plural "they", and "he/she" in other cases (for example if the person requesting the translation might know the sex and be able to fill it in for themselves). Those criteria also align roughly with the acceptability of singular "they" in modern British English: a sentence like "everyone should mind their own business" is totally normal and standard and always has been; a sentence like "the unidentified miscreant left their cup on the table" will seem a bit strange to some people; a sentence like "Chris ate their lunch" will seem strange to a lot of people. I wouldn't expect Google Translate to be capable of such cleverness so a simpler solution would be needed.

commandlinefan 6y ago

It seems to me that Google translate is just playing the odds. It doesn't see "he is beautiful" very often (if at all), so it has two possible outputs: "she is beatiful" or "he is beautiful" and goes with the most common one, as it not only should, but must.