4 ms·

I agree. What I meant was that Google's blog post and the New York Times article make it sound like machine translation is a solved problem.

by ma2rten 10y ago

nl 10y ago
Which Google blog post made it sound like that? "A Neural Network for Machine Translation, at Production Scale"[1] says: Machine translation is by no means solved. GNMT can still make significant errors that a human translator would never make, like dropping words and mistranslating proper names or rare terms, and translating sentences in isolation rather than considering the context of the paragraph or page. There is still a lot of work we can do to serve our users better. However, GNMT represents a significant milestone. We would like to celebrate it with the many researchers and engineers—both within Google and the wider community—who have contributed to this direction of research in the past few years. That seems fair to me. "Found in translation: More accurate, fluent sentences in Google Translate"[2] says: Today’s step towards Neural Machine Translation is a significant milestone for Google Translate, but there’s always more work to do and we’ll continue to learn over time. We’ll also continue to rely on Translate Community, where language loving multilingual speakers can help share their language by contributing and reviewing translations. "Zero-Shot Translation with Google’s Multilingual Neural Machine Translation System"[3] says: In September, we announced that Google Translate is switching to a new system called Google Neural Machine Translation (GNMT), an end-to-end learning framework that learns from millions of examples, and provided significant improvements in translation quality. [1] https://research.googleblog.com/2016/09/a-neural-network-for-machine.html https://research.googleblog.com/2016/09/a-neural-network-for... [2] https://blog.google/products/translate/found-translation-more-accurate-fluent-sentences-google-translate/ https://blog.google/products/translate/found-translation-mor... [3] https://research.googleblog.com/2016/11/zero-shot-translation-with-googles.html https://research.googleblog.com/2016/11/zero-shot-translatio...
- ma2rten 10y ago
  If you look at the first chart in [1] and the cherry-picked example, it's easy for a casual observer to to come to the conclusion that MT is a solved problem, especially for "English > Spanish" and "French > English".
  - nl 10y ago
    There is a clear gap between the performance of the system and "perfect translation". I don't think Google has been misleading in any way here - the system is pretty amazing, and this is the research blog so it is reasonable to expect people to look at it at least slightly.
    - ma2rten 10y ago
      But there is almost no gap between "neural" and "human translation".
      - nl 10y ago
        Compared to a single human, yes. It really is that good for those language pairs(!) This isn't cherry picking - you can try it yourself. Randomly selecting the 2nd paragraph from https://www.hrw.org/fr/news/2017/01/02/israel/palestine-des-responsables-israeliens-ont-encourage-les-forces-de-lordre https://www.hrw.org/fr/news/2017/01/02/israel/palestine-des-... Human Rights Watch a documenté de nombreuses déclarations faites depuis octobre 2015 par des personnalités politiques israéliennes de haut rang, dont les ministres de la Police et de la Défense, appelant la police et les forces armées à tirer sur des individus suspectés d'attentats pour les tuer, avant même de déterminer si le recours à la force létale est, ou non, absolument nécessaire pour protéger des vies. Choose which is automatically translated: Human Rights Watch has documented numerous statements since October 2015, by senior Israeli politicians, including the police minister and defense minister, calling on police and soldiers to shoot to kill suspected attackers, irrespective of whether lethal force is actually strictly necessary to protect life. vs Human Rights Watch has documented numerous statements made since October 2015 by high-ranking Israeli politicians, including police and defense ministers, calling on police and armed forces to shoot at suspected terrorists even before deciding whether or not the use of lethal force is absolutely necessary to protect lives. One of these in manually translated by an expert, one is by Google Translate (clicking on the "English" link on the page linked above will show the expert translation). You can find other document collections to test here: http://www.miis.edu/academics/library/find/guides/translating/intlorgs http://www.miis.edu/academics/library/find/guides/translatin...

I agree. What I meant was that Google's blog post and the New York Times article make it sound like machine translation is a solved problem.

by ma2rten 10y ago

I agree. What I meant was that Google's blog post and the New York Times article make it sound like machine translation is a solved problem.

nl 10y ago

Which Google blog post made it sound like that? "A Neural Network for Machine Translation, at Production Scale"[1] says: Machine translation is by no means solved. GNMT can still make significant errors that a human translator would never make, like dropping words and mistranslating proper names or rare terms, and translating sentences in isolation rather than considering the context of the paragraph or page. There is still a lot of work we can do to serve our users better. However, GNMT represents a significant milestone. We would like to celebrate it with the many researchers and engineers—both within Google and the wider community—who have contributed to this direction of research in the past few years. That seems fair to me. "Found in translation: More accurate, fluent sentences in Google Translate"[2] says: Today’s step towards Neural Machine Translation is a significant milestone for Google Translate, but there’s always more work to do and we’ll continue to learn over time. We’ll also continue to rely on Translate Community, where language loving multilingual speakers can help share their language by contributing and reviewing translations. "Zero-Shot Translation with Google’s Multilingual Neural Machine Translation System"[3] says: In September, we announced that Google Translate is switching to a new system called Google Neural Machine Translation (GNMT), an end-to-end learning framework that learns from millions of examples, and provided significant improvements in translation quality. [1] https://research.googleblog.com/2016/09/a-neural-network-for-machine.html https://research.googleblog.com/2016/09/a-neural-network-for... [2] https://blog.google/products/translate/found-translation-more-accurate-fluent-sentences-google-translate/ https://blog.google/products/translate/found-translation-mor... [3] https://research.googleblog.com/2016/11/zero-shot-translation-with-googles.html https://research.googleblog.com/2016/11/zero-shot-translatio...

ma2rten 10y ago
If you look at the first chart in [1] and the cherry-picked example, it's easy for a casual observer to to come to the conclusion that MT is a solved problem, especially for "English > Spanish" and "French > English".
- nl 10y ago
  There is a clear gap between the performance of the system and "perfect translation". I don't think Google has been misleading in any way here - the system is pretty amazing, and this is the research blog so it is reasonable to expect people to look at it at least slightly.
  - ma2rten 10y ago
    But there is almost no gap between "neural" and "human translation".
    - nl 10y ago
      Compared to a single human, yes. It really is that good for those language pairs(!) This isn't cherry picking - you can try it yourself. Randomly selecting the 2nd paragraph from https://www.hrw.org/fr/news/2017/01/02/israel/palestine-des-responsables-israeliens-ont-encourage-les-forces-de-lordre https://www.hrw.org/fr/news/2017/01/02/israel/palestine-des-... Human Rights Watch a documenté de nombreuses déclarations faites depuis octobre 2015 par des personnalités politiques israéliennes de haut rang, dont les ministres de la Police et de la Défense, appelant la police et les forces armées à tirer sur des individus suspectés d'attentats pour les tuer, avant même de déterminer si le recours à la force létale est, ou non, absolument nécessaire pour protéger des vies. Choose which is automatically translated: Human Rights Watch has documented numerous statements since October 2015, by senior Israeli politicians, including the police minister and defense minister, calling on police and soldiers to shoot to kill suspected attackers, irrespective of whether lethal force is actually strictly necessary to protect life. vs Human Rights Watch has documented numerous statements made since October 2015 by high-ranking Israeli politicians, including police and defense ministers, calling on police and armed forces to shoot at suspected terrorists even before deciding whether or not the use of lethal force is absolutely necessary to protect lives. One of these in manually translated by an expert, one is by Google Translate (clicking on the "English" link on the page linked above will show the expert translation). You can find other document collections to test here: http://www.miis.edu/academics/library/find/guides/translating/intlorgs http://www.miis.edu/academics/library/find/guides/translatin...