4 ms·

> do you have any idea of what things machine translation still consistently gets wrong? My biggest pet peeve with MT is the incredibly brain-dead choice of us

by qayxc 6y ago

> do you have any idea of what things machine translation still consistently gets wrong? My biggest pet peeve with MT is the incredibly brain-dead choice of using English as an intermediate language. This effectively prevents MT from ever becoming useful for any A→B translation where neither A nor B are English. Example: put "пружи́на" into Google translate (or Bing translate, doesn't matter)and translate to German. You'll get "Frühling" as the suggested translation. This is due to the fact that internally "пружи́на" is translated to English first, resulting in "spring". This is then translated to German with the statistically highest probability being "Frühling", even though "Feder" would be the only correct match ("Frühling" would be "весна" in Russian). If you now say "But wait! This should clear itself up once you add some context, right?" you'd be wrong. Try "Die Sprungfeder kam gerade rechtzeitig." ("The coil spring arrived just in time,") and you'll get "Весна пришла как раз вовремя." ("Springtime came just in time"). It's ridiculous! EDIT: happens with DeepL as well; only Yandex gets it right (presumably because they actually use an actual German/Russian-model.

dmurray 6y ago
It's not brain-dead, it's a result of limited training data. For N languages you need to train N models if you go through English as an intermediate language, but N^2 if you don't use an intermediate language model. OK, computer time is cheap these days, but more importantly, for each language pair you need D high-quality documents translated in each of your two languages. There are far fewer pieces of writing that exist in both German and Russian than in both German and English or both Russian and English. There is cool-looking research on zero-shot language translation where the model learns its own internal "intermediate language". But you can be sure that if it produced reliably better results with the data available, the likes of Google would already be using it.
- qayxc 6y ago
  > It's not brain-dead, it's a result of limited training data. That's quite literally a cheap excuse. It'snot lack of training data (remember, you still need translations in the target language anyway!), it's a result of Anglo-Saxon cultural hegemony. > There are far fewer pieces of writing that exist in both German and Russian than in both German and English or both Russian and English. Right. And that's why Russia-based Yandex gets excellent results for German/Russian translation, I see... > But you can be sure that if it produced reliably better results with the data available, the likes of Google would already be using it. Nah, you're being way too optimistic there. Google is a business and as such, they don't chase SOTA outside of research (which doubles as advertisement). Good enough is the name of the game and it's simply infinitely cheaper to just train and maintain N-models (i.e. English/X and X/English) than to do the same for all combinations of X/Y-pairs (which would be N² models). So as long as there is no incentive to do better, they simply won't, because they rely on the fact that English is the de-facto Lingua Franca and people translate to and from English more often than directly between other languages. This would be an actual chance for companies from other countries (especially European ones) to step in and do exactly that, but again - "good enough" and reliance on English stops that from happening anytime soon. It's a missed opportunity especially with low-resource languages as demonstrated by the recent "Scottish" Wikipedia fiasco...
MiroF 6y ago
> This is due to the fact that internally "пружи́на" is translated to English first, resulting in "spring" Do you have a source that that is actually what is happening behind the scenes? I work in this field and would be surprised if that is what Google is doing. My guess is this phenomenon is more due to a model that was trained on an Russian->English corpora first and then trained on Russian->German due to low resource, or something like that. Someone like Yandex is going to have better translations because (shocker) they have larger russian corpora than google.
- qayxc 6y ago
  > Do you have a source that that is actually what is happening behind the scenes? Indirectly yes, I do: https://codesachin.wordpress.com/2017/01/18/understanding-the-new-google-translate/ https://codesachin.wordpress.com/2017/01/18/understanding-th... > What this basically means is: If during training you provide it examples of English->Japanese & English->Korean translations, GNMT automatically does Japanese->Korean reasonably well! In fact, this is the biggest achievement of GNMT as a project. It's exactly what Google have been doing for years now: train on X-to-English and English-to-Y only to get X-to-Y for free. Since the intermediate language only ever sees English as a source or target language, ambiguities like "spring" literally get lost in translation.
  - MiroF 6y ago
    Just gave that a read. 1. It is pretty out of date (ML is a fast moving field) - I doubt Google is using LSTMs for translation in 2020. > Since the intermediate language only ever sees English as a source or target language, ambiguities like "spring" literally get lost in translation. 2. This article is trying to dumb down what Google is doing - but you're right that this is why ambiguities get lost in translation, due to pre-training on a different language pair. That said, there isn't literally a process of "translate into english" and then "translate english into german". These models are trained on Russian-German corpora, but because there is little resources for that, they are supplementing with Russian-English and English-German.
    - 6y ago

by qayxc 6y ago

dmurray 6y ago

It's not brain-dead, it's a result of limited training data. For N languages you need to train N models if you go through English as an intermediate language, but N^2 if you don't use an intermediate language model. OK, computer time is cheap these days, but more importantly, for each language pair you need D high-quality documents translated in each of your two languages. There are far fewer pieces of writing that exist in both German and Russian than in both German and English or both Russian and English. There is cool-looking research on zero-shot language translation where the model learns its own internal "intermediate language". But you can be sure that if it produced reliably better results with the data available, the likes of Google would already be using it.

qayxc 6y ago
> It's not brain-dead, it's a result of limited training data. That's quite literally a cheap excuse. It'snot lack of training data (remember, you still need translations in the target language anyway!), it's a result of Anglo-Saxon cultural hegemony. > There are far fewer pieces of writing that exist in both German and Russian than in both German and English or both Russian and English. Right. And that's why Russia-based Yandex gets excellent results for German/Russian translation, I see... > But you can be sure that if it produced reliably better results with the data available, the likes of Google would already be using it. Nah, you're being way too optimistic there. Google is a business and as such, they don't chase SOTA outside of research (which doubles as advertisement). Good enough is the name of the game and it's simply infinitely cheaper to just train and maintain N-models (i.e. English/X and X/English) than to do the same for all combinations of X/Y-pairs (which would be N² models). So as long as there is no incentive to do better, they simply won't, because they rely on the fact that English is the de-facto Lingua Franca and people translate to and from English more often than directly between other languages. This would be an actual chance for companies from other countries (especially European ones) to step in and do exactly that, but again - "good enough" and reliance on English stops that from happening anytime soon. It's a missed opportunity especially with low-resource languages as demonstrated by the recent "Scottish" Wikipedia fiasco...

MiroF 6y ago

> This is due to the fact that internally "пружи́на" is translated to English first, resulting in "spring" Do you have a source that that is actually what is happening behind the scenes? I work in this field and would be surprised if that is what Google is doing. My guess is this phenomenon is more due to a model that was trained on an Russian->English corpora first and then trained on Russian->German due to low resource, or something like that. Someone like Yandex is going to have better translations because (shocker) they have larger russian corpora than google.

qayxc 6y ago
> Do you have a source that that is actually what is happening behind the scenes? Indirectly yes, I do: https://codesachin.wordpress.com/2017/01/18/understanding-the-new-google-translate/ https://codesachin.wordpress.com/2017/01/18/understanding-th... > What this basically means is: If during training you provide it examples of English->Japanese & English->Korean translations, GNMT automatically does Japanese->Korean reasonably well! In fact, this is the biggest achievement of GNMT as a project. It's exactly what Google have been doing for years now: train on X-to-English and English-to-Y only to get X-to-Y for free. Since the intermediate language only ever sees English as a source or target language, ambiguities like "spring" literally get lost in translation.
- MiroF 6y ago
  Just gave that a read. 1. It is pretty out of date (ML is a fast moving field) - I doubt Google is using LSTMs for translation in 2020. > Since the intermediate language only ever sees English as a source or target language, ambiguities like "spring" literally get lost in translation. 2. This article is trying to dumb down what Google is doing - but you're right that this is why ambiguities get lost in translation, due to pre-training on a different language pair. That said, there isn't literally a process of "translate into english" and then "translate english into german". These models are trained on Russian-German corpora, but because there is little resources for that, they are supplementing with Russian-English and English-German.
  - 6y ago