4 ms·
If you look at the first chart in [1] and the cherry-picked example, it's easy for a casual observer to to come to the conclusion that MT is a solved problem,
by ma2rten 10y ago
If you look at the first chart in [1] and the cherry-picked example, it's easy for a casual observer to to come to the conclusion that MT is a solved problem, especially for "English > Spanish" and "French > English".
- nl 10y agoThere is a clear gap between the performance of the system and "perfect translation". I don't think Google has been misleading in any way here - the system is pretty amazing, and this is the research blog so it is reasonable to expect people to look at it at least slightly.
- ma2rten 10y agoBut there is almost no gap between "neural" and "human translation".
- nl 10y agoCompared to a single human, yes. It really is that good for those language pairs(!) This isn't cherry picking - you can try it yourself. Randomly selecting the 2nd paragraph from https://www.hrw.org/fr/news/2017/01/02/israel/palestine-des-responsables-israeliens-ont-encourage-les-forces-de-lordre https://www.hrw.org/fr/news/2017/01/02/israel/palestine-des-... Human Rights Watch a documenté de nombreuses déclarations faites depuis octobre 2015 par des personnalités politiques israéliennes de haut rang, dont les ministres de la Police et de la Défense, appelant la police et les forces armées à tirer sur des individus suspectés d'attentats pour les tuer, avant même de déterminer si le recours à la force létale est, ou non, absolument nécessaire pour protéger des vies. Choose which is automatically translated: Human Rights Watch has documented numerous statements since October 2015, by senior Israeli politicians, including the police minister and defense minister, calling on police and soldiers to shoot to kill suspected attackers, irrespective of whether lethal force is actually strictly necessary to protect life. vs Human Rights Watch has documented numerous statements made since October 2015 by high-ranking Israeli politicians, including police and defense ministers, calling on police and armed forces to shoot at suspected terrorists even before deciding whether or not the use of lethal force is absolutely necessary to protect lives. One of these in manually translated by an expert, one is by Google Translate (clicking on the "English" link on the page linked above will show the expert translation). You can find other document collections to test here: http://www.miis.edu/academics/library/find/guides/translating/intlorgs http://www.miis.edu/academics/library/find/guides/translatin...
- ma2rten 10y agoEvery translation is done by a single human... I would assume they used professional translators, otherwise it's not a fair comparison. To honest it works better than I thought for French > English, but I when I tried some random French websites (not news) the results were not as good as this example and nowhere near human translations.
- nl 10y agoEvery translation is done by a single human No, the "perfect score" is an "averaged" translation by multiple humans, measured by BLEU score.