11 ms·
Google’s Neural Machine Translation System
- vnglst 10y ago"Using a human side-by-side evaluation on a set of isolated simple sentences, it reduces translation errors by an average of 60% compared to Google's phrase-based production system." Does this mean that after checking the translation by a human it becomes 60% better than the phrase-based production system? (by which they presumably mean Google Translate...?). That seems rather disappointing.
- kuschku 10y ago60% better than Google translate still is at best elementary school level fluency in a language. It’s a needed improvement, but still far from good. It’s crazy how bad Google translate is, try it with any German text, and you’ll get 90% ununderstandable garbage out. EDIT: Seriously, downvotes for this? Try this: https://translate.google.com/translate?sl=auto&tl=en&js=y&prev=_t&hl=en&ie=UTF-8&u=http://www.spiegel.de/politik/ausland/bratislava-eu-verteidigungsminister-wollen-enger-zusammenarbeiten-a-1114151.html https://translate.google.com/translate?sl=auto&tl=en&js=y&pr... What’s " If you look at Ursula von der Leyen and Jean-Yves Le Drian before the meeting of defense ministers in Bratislava, one might think that Berlin and Paris would never prefer liked." supposed to mean? Or "Individual states can not prevent the European Council may decide by a qualified majority, the SSZ."
- lorenzhs 10y agoI think you're getting downvoted because you picked one of the hardest language pairs there is for machine translation. German grammar is really hard to parse, e.g. verbs being split with one part at the beginning and the other at the end of the sentence. Generally, figuring out which part of the sentence refers to which other part is quite hard. Google Translate works a lot better with many other language pairs, so people might be downvoting because your opinion sounds exaggerated and doesn't line up with their own experience. It's also quite dismissive of the submission, which is never nice. The guidelines ask us to "avoid gratuitous negativity". I often find your comments to be quite negative, often unnecessarily so.
- kuschku 10y agoIt seems like I forgot the actual point of my argument – "It’s a needed improvement, but still far from good". > I often find your comments to be quite negative, often unnecessarily so. Well, if it’s a relief, I’m equally negative about stuff I create myself.
- DonaldFisk 10y agoTranslations from French to English seem to be a lot better. Why is that?
- pigscantfly 10y agoThe French/English language pair was the problem machine translation research started with and has seen the most work. There are also greater similarities than most language pairs between those two due to the Norman conquest in 1066.
- DonaldFisk 10y agoI accept your first point, but I disagree with your second - 1066 and all that notwithstanding, English is still very much a Germanic language, and grammatically very different from French.
- d_burfoot 10y agoThis paper is a good illustration of what I see as a critical shortcoming of modern NLP work: it is all about math and algorithms, there is nothing about actual language. The paper has lots of information about: neural network architectures, parameter update equations, learning rules, and inference algorithms. There is nothing about: part of speech categories, relative clauses, morphology, affixes or compound words, the theta criterion, the content/function distinction, verb tenses, agreement, or anything else related to actual linguistic phenomena. To me this seems a bit like the Aristotelian thinkers who tried to reason about physics based on pure mathematical analysis, without any empirical work.
- lohankin 10y agohttps://en.wikipedia.org/wiki/Laputa https://en.wikipedia.org/wiki/Laputa
- vog 10y agoCare to elaborate?
- ktta 10y agoThere is a problem that arises in this case that isn't present in the analogy you gave. The physics they were using pure mathematical analysis was logical. Language, and how humans always change the way they use language is largely illogical. You're right in saying that we can't really get anywhere trying to analyse language piece by piece. But pure mathematical analysis is not getting us great results either.
- tlb 10y agoThat's the best part of this style of machine translation: the code knows nothing about parts of speech or other categories that linguists try to force-fit language into. Instead, it learns the structure of both languages from training data (sentence pairs in source and target languages).
- Houshalter 10y agoBecause this isn't linguistics work. It's AI work. We are bypassing the impossibly hard problem of natural language processing. Instead we build AIs that can do the hard work for us, and much faster and better than any human linguist ever could.
- Houshalter 10y agoI hate when people judge machine translation progress by looking at Google translate. Google translate is really old and is no longer state of the art. But the best neural network based systems like this are much too expensive to use in production, at least for free. They say their new neural network ASICs will make it more practical, at least. Anyway the scale of these neural nets is quite incredible. Google is getting far ahead of what any individual researcher with a few consumer GPUs can do.
- habitue 10y agoBut not, apparently, ahead of what you can do with 8 consumer GPUs
- Houshalter 10y ago>On WMT En→Fr, it takes around 6 days to train a basic model using 96 NVIDIA K80 GPUs. Then they use custom ASICs to actually run the thing in production.
- aab0 10y agoA $3k Tesla K80 GPU is not 'consumer', nor do most consumers (or most researchers or small businesses) have $24k to drop on a set of GPUs alone.
- habitue 10y agoAh, sorry I missed that they were tesla gpus
- dgacmu 10y agoYou're right - a $3k k80 is generally inferior for most DNN applications than a $1k Titan X. The primary reasons that big companies use K80s have to do with achieving high computational density, and licensing issues, more than the performance of a single GPU. Sticking 8 Titan X boards in a machine is a bummer job if you want to pack them closely together. But for academic researchers, a quad Titan X box is pretty solid and quite affordable.
- runesoerensen 10y agoRelated blog post: https://research.googleblog.com/2016/09/a-neural-network-for-machine.html https://research.googleblog.com/2016/09/a-neural-network-for...
- andrewtbham 10y ago"In addition to releasing this research paper today, we are announcing the launch of GNMT in production on a notoriously difficult language pair: Chinese to English.... we will be working to roll out GNMT to many more of these over the coming months." interesting news for all the translation startups... like yc's unbabel.
- readams 10y agoI've been attempting to use this system for a while today to converse with a native Chinese speaker and it seems to be better but still very far from human-level translation. Maybe they've been using especially bad human translators in their comparisons?
- DonaldFisk 10y agoIf its performance is dependent on proximity of words in the sentences of the two languages, its output for Chinese would be better than you'd otherwise expect. Chinese and English have similar word orders (though the underlying grammar's different).
- est 10y agoDid you try short conversational dialogue or long paragraphs? AFAIK, most neural training model are based on formal written text instead of verbal communications.
- WhitneyLand 10y agoDumb question: Why doesn't Google Translate hardcode perfect translations for 1,000,000 of the most popular requests? So many times it screws up really basic, common phrases and I'm always wondering how is that bar that low?
- kafkaesq 10y agoNot a dumb question at all. Actually a very fundamental question.