7 ms·

Zero-Shot Translation with Multilingual Neural Machine Translation System

218 points by wwilson 10y ago

honkhonkpants 10y ago
Pretty impressive, but even more amazingly their paper is in a single-column format that I can actually read on my computer, instead of pretending that I am reading printed and bound conference proceedings. Truly a giant leap for the field.
spynxic 10y ago
This post seems to over-exaggerate a commonly known mathematical property. Suppose I have languages X, Y, and Z. My machine currently knows how to translate between X->Y and X->Z. The goal is to turn Y into Z without direct training. The process would be to translate Y into X and X into Z.. effectively Y->Z. This isn't really transfer learning as much as it is logical induction...Or am I missing something?
- railorsi 10y ago
  Could we say logical induction is the principle behind transfer learning then?
  - spynxic 10y ago
    Emergent induction via high dimensional structure composition.. maybe. I haven't read any whitepages about the underlying processes P.S. note the hand-waviness when I use the word "emergent"
- eridius 10y ago
  Presumably what you're missing is the fact that this is translating from Y->Z in a single step, rather than doing two separate translations.
  - spynxic 10y ago
    Then the high dimensional representation of the language that allows such a translation to be possible is profound
  - mattkrause 10y ago
    They're certainly not doing English to Portuguese by way of Spanish or anything like that. However, you could almost read the paper as "We can translate something from a natural language into a high-dimensional representation, then turn that high-dimensional representation back into (another, possibly different) natural language. "
    - empath75 10y ago
      It seems similar to automatically captioning images.
      - mattkrause 10y ago
        Very much so! The two tasks are surprisingly interchangable. I once worked on a project where we used a statistical MT approach to "translate" between image features and captions--and I don't think we were the only ones trying such things. In a pleasing bit of symmetry, the attentional network used here looks like it was initially developed for image captioning.
- mattkrause 10y ago
  Well.... It is one of the oldest ideas in machine translation. In the late 1950s, Richens proposed an MT system that used an intermediate representation called an "interlingua." For each language, the system has two components, one that maps a natural language onto the interlingua, and a second that converts the interlingua back into a natural language. Use different source and destination pairs and--bam! you've got a multi-way translator. However, this only works if a) the interlingua is rich enough to capture the semantics of the to-be-translated text and b) the conversions between natural languages and interlingua also preserves those semantics. In practice, neither has worked pretty well. In the 1960s, the interlingua was typically hand-crafted and rules were either hand written, or later, induced, to convert natural languages to/from it. Since people have been working on this for 60 years, you can probably image how well that works[]. The clever bit here is that you don't really need to do that. If you have enough data, the LSTMs can learn it on their own (see Figure 2, for example, where semantically-related sentences from multiple languages end up in the same neighborhoods). [] Actually, somewhat better than you'd think. It certainly wouldn't have done Pushkin any justice, but it was surprisingly decent on weather reports (etc).
  - csydas 10y ago
    I was going to comment on the article and idea of multilingual NN's in general since, as I'm learning Russian currently, I'm having a hard time seeing how you can get a good automated translation without some editorializing or awareness of what's going on in both sentences. Your comment on NN's "... not doing Pushkin justice" stood out since it's something I've been wondering how NN could handle a language like Russian that traditionally doesn't use the verb "to be" in sentences, much like the E-Prime idea that exists in English for clarity of description. Just trying to re-think in terms of a Russian sentence instead of an English sentence has also proven difficult, and I'd be curious how you can automate it on NN's well. My colleagues (russian natives) are all able to spot auto-translated messages from me nearly instantly since it messes up pretty rudimentary grammar rules or just doesn't phrase the sentence like you would in native Russian.
- nl 10y ago
  No one has managed to get induction working reliably for natural language tasks in any way that is generalizable enough to rely on. So that is new.
hota_mazi 10y ago
Fascinating. Maybe the next step will be to extract the tokenized interlingua language that's emerged in the neural network and map it to real words, and blam, we reinvent Esperanto!
- eveningcoffee 10y ago
  Esperanto is a lovely idea but any language needs an active support to stay current. If the language does not take inputs from culture, science and philosophy, it starts to degrade.
  - renox 10y ago
    ? Nothing prevent adding new words to Esperanto, the most difficult part is reaching a consensus but that's also the case for 'normal' languages: there are many new French word introduced by the academie Française which aren't much used (courriel instead of mail, pourriel instead of spam, etc)..
    - eveningcoffee 10y ago
      It is not about language capability to accept new words but about language users motivation to do so.
- klodolph 10y ago
  It would probably not look anything like Esperanto, which is a charliefoxtrot of pidgin Spanish with some Turkish orthography thrown in. In all seriousness, however, different languages often have radically different concepts encoded into them. For example, in English sentences you might be forced to explicitly give subjects to the actors in your sentences, assign genders to your pronouns, and choose whether actions take place in the present or future, whether they are ongoing in general or happening at this moment, etc. In Japanese you might be forced to choose how to express your relationship with the person you are speaking to, the person you are speaking about, and decide whether a causal relationship is "if and only if" or just plain "if then". There's long been a hypothesis about whether there is some kind of "universal grammar" (Chomsky) but modern linguists do not really entertain that idea.
glandium 10y ago
The mentioned Japanese->English->Korean combination is one of the worst possible things to do. Both Korean and Japanese are very different from English, while similar to each other to some extent. Direct translation from one to the other would actually have much better results than translating back to Korean the (likely broken) English you get from Japanese. Edit: I do realize they're not talking about successive translations, but that's essentially how the training ends up happening, isn't it? A better example, IMHO, would have been three very different languages, like English, Japanese and Russian.
- xbmcuser 10y ago
  It is not translating from Japanese>English>korean it learned how to translate Japanese<>English Korean<>English now it knows how to translate Japanese<>Korean. In laymen terms it understands Korean and Japanese so can translate between them without needing examples of Japanese<>Korean translations.
- paulsutter 10y ago
  I'm guessing the downvotes here are based on your wording, "worst possible thing to do", and not your core point that Korean and Japanese are grammatically very similar. You're right that the test would be even more interesting with three very different languages. But it's still pretty impressive with Korean, English, and Japanese as shown, despite the similarity.
Smerity 10y ago
If people are interested in the underlying architecture of Google's Neural Machine Translation (GNMT) system, I wrote an article that builds it up piece by piece. While it's intended for people who are likely to implement GNMT or similar architectures, the article is descriptive enough that it should be possible to follow along even if you're not well versed in deep learning. http://smerity.com/articles/2016/google_nmt_arch.html http://smerity.com/articles/2016/google_nmt_arch.html The GNMT architecture is used almost as is for the zero-shot MT experiments. We're likely to see the GNMT architecture used extensively by Google for a variety of projects as they spent a deal of time and effort ensuring it is scalable to quite large datasets. Training a neural machine translation system with a single language pair is difficult - training it with multiple, especially all using the same set of weights, is insanely challenging! As an example, the GNMT architecture was used as the basis of "Generating Long and Diverse Responses with Neural Conversation Models", which trains on the entirety of Reddit (1.7 billion messages) as well as other various datasets. https://openreview.net/forum?id=HJDdiT9gl https://openreview.net/forum?id=HJDdiT9gl
- gabrielgoh 10y ago
  i see ray kurtzwell managed to sneak his name in as the last author. heh
- alexbeloi 10y ago
  Can you explain the odd looking connections to the decoder that appear in V2 and onwards? It's not entirely obvious to me from the description how the attention is applied. In the first part of the decoder rollout, the diagram connects the (concatenated?) output of the encoder into the middle(?) of the first output of the decoder rollout. In what sense? For the second rollout of the decoder the diagram shows 3 arrows going into the LSTM cell, what is that (left-facing) arrow on the right side doing?
sgentle 10y ago
This reminds me of Searle's Chinese Room Argument[0]: imagine you have a particularly dreary job where you sit in a room filled with boxes of symbols written on paper. Every now and again someone comes in and hands you some new symbols. You look through your rulebook and, depending on what it says, hand them some symbols back. It turns out that these rules actually implement a conversational program in Chinese. And if you can implement those rules and not understand Chinese, why would you think a computer program, implementing its own rules, could understand anything? The common "Systems Reply" response to this is that you're looking at the wrong layer of abstraction. The computer hardware (or the person in the room) doesn't understand Chinese, the computer plus the rules plus the data forms a system that understands Chinese. Searle's answer to this is that, well, what if you memorised the rules and the database? You might know all the rules, you might be able to follow them, but you wouldn't understand Chinese. What I think is fascinating about this is that it's vulnerable to Bayesian Judo[1]: if you have a strong belief that computers aren't capable of true understanding because of the Chinese Room Argument, then building an actual Chinese Room-style computer and having it show understanding should be a fairly strong blow to that belief. Now, it's easy to quibble about what true understanding actually means, but one version (used by Searle's answer) is this: "[..] he would not know the meaning of the Chinese word for hamburger. He still cannot get semantics from syntax." But this news is exactly that! A computer translation of the same semantic concept from one syntax to another without ever having been taught the rules connecting them. In other words, this is semantics from syntax implemented by nothing but a computer, a database, and a set of rules. So, by the reverse Chinese Room Argument, I would say this system exhibits a kind of understanding. Not a very sophisticated kind, mind you, but something that should still spook you if you believe computers are categorically incapable of thinking like us. [0] http://plato.stanford.edu/entries/chinese-room/ http://plato.stanford.edu/entries/chinese-room/ [1] http://lesswrong.com/lw/i5/bayesian_judo/ http://lesswrong.com/lw/i5/bayesian_judo/
- timr 10y ago
  "But this news is exactly that! A computer translation of the same semantic concept from one syntax to another without ever having been taught the rules connecting them." By that standard, statistical translation approaches were "understanding" a long time ago. The new thing here isn't that systems aren't being taught "the rules" (that wasn't happening in statistical MT either), the new thing is that there's a different kind of classifier in the "middle" now, which is representing a hidden state. This classifier is more flexible in a lot of ways, but also more of a black box, and takes a lot more effort to train without overfitting. It's cool that you can translate between language pairs that have never been explicitly trained, but let's not overstate the meaning of it. The blog post makes this rather breathless speculation: "Within a single group, we see a sentence with the same meaning but from three different languages. This means the network must be encoding something about the semantics of the sentence rather than simply memorizing phrase-to-phrase translations. We interpret this as a sign of existence of an interlingua in the network." This is...a fun story, but not much else. First off, you can make dimensionality reduction plots that "show" a lot of things. Even ignoring that issue, in translations of short sentences involving specific concepts (i.e. the example about the stratosphere), is it really surprising that you'd find clusters? The words in that sentence are probably unique enough that they'd form a distinct cluster in mappings from any translation system. Folks get caught up in the "neural" part of neural networks, and assume that some magical quasi-human thought is happening. If the tech were called "highly parameterized reconfigurable weighted networks of logistic classifiers", there'd be less loopy speculation.
ChuckMcM 10y ago
Nice piece of work and counts as "implementing Star Trek in the present". Now I just need a nice pair of noise cancelling over the ear headphones that let me hear english spoken no matter where I am :-)
- agildehaus 10y ago
  Stick a fish in your ear. Might get lucky.
vurpo 10y ago
I wonder how much memory this translation via an intermediate representation of a sentence takes. It seems like representing the semantic meaning of a sentence in a language-independent way would take a huge amount of data.
- ximeng 10y ago
  E.g. Japanese is 50MB to download for offline usage in Google Translate's iPhone app.
deleted 10y ago
[deleted]
YeGoblynQueenne 10y ago
>> We call this “zero-shot” translation, shown by the yellow dotted lines in the animation. To the best of our knowledge, this is the first time this type of transfer learning has worked in Machine Translation. I think it was last year when a friend was telling me how Google translates the Greek word for "swallow" (the bird) to French. Back then, the translation was the French word for "to swallow" (the verb). The bird and the action don't even sound remotely alike in Greek and neither are they spelled alike (the bird is "χελιδόνι" the action is "καταπίνω"; google trans. will at least give their correct pronounciation). My friend figured Google can't find enough examples between the two languages, so it goes via English ... were the two words are homonyms. I think that was last year, and certainly before September. So I gave it a try again today, and this is still what I get: Greek French χελιδόνι avaler chelidóni If you omit the accent on the "o" you don't even get the mistranslation- you get only the phonetic transcription of the Greek word in latin characters. Obviously the important thing here is not the one word that google translate gets wrong, but the fact that it doesn't really look like this "new" system is all that new, or that it does anything all that different from the previous one, or indeed that it improves things at all.
YeGoblynQueenne 10y ago
Google, and also Microsoft btw, absolutely need to be called out on this. They keep claiming that their translation systems work well, because they have reasonably good results between some language pairs, like English/French or English/Spanish, that are a) close linguistically b) have a lot of examples of translated documents and, more importantly, c) have many speakers who might use Google translate. For languages where none of the above holds, however, the results continue to be completely ridiculous, no matter what "new" technique Google (or MS) advertises. Since those languages are not spoken by as many people as English or Spanish etc, however, it's very hard for the user to figure out how attrociously bad their automatic translations are. Here's an example from my native Greek; this is a bit of news text from yesterday [1]: Λανθασμένη χαρακτηρίζει ο Κύπριος κυβερνητικός εκπρόσωπος, Νίκος Χριστοδουλίδης, την προσέγγιση, να μπαίνουν στο «ίδιο καλάθι» η Ελλάδα με την Τουρκία σε σχέση με το κυπριακό. And here's Google's translation: Incorrect characterizes Cypriot government spokesman Nikos Christodoulides, the approach to be put in "one basket" by Greece and Turkey in relation to Cyprus. So, the Cypriot government spokesman (well done) is put in a basket by Greece and Turkey (wait wut). Hey, maybe the guy wanted to be put in two baskets? [2] That's very typical of the way Google translates between Greek and English. For Google, it's Neural Networks leading us to a bright future where language barriers are eliminated thanks to Scienz! For Greeks, it's comedy gold. And it's the same for Russians, Polish, Finns, Swedes, Indians, Chinese, Hungarians... Still, Google keeps including those languages in the count of languages it "covers", because it's good advertisement and who can really dispute them anyway? _____________________ [1] http://www.kathimerini.gr/884845/article/epikairothta/politikh/xristodoylidhs-gia-kypriako-den-yphr3e-akraia-8esh-apo-thn-a8hna http://www.kathimerini.gr/884845/article/epikairothta/politi... [2] What's being said is more like: "The Cypriot government spokesman said that it's a mistake to treat Greece and Turkey in the same manner with regards to Cyprus".

218 points by wwilson 10y ago

Pretty impressive, but even more amazingly their paper is in a single-column format that I can actually read on my computer, instead of pretending that I am reading printed and bound conference proceedings. Truly a giant leap for the field.

spynxic 10y ago

This post seems to over-exaggerate a commonly known mathematical property. Suppose I have languages X, Y, and Z. My machine currently knows how to translate between X->Y and X->Z. The goal is to turn Y into Z without direct training. The process would be to translate Y into X and X into Z.. effectively Y->Z. This isn't really transfer learning as much as it is logical induction...Or am I missing something?

railorsi 10y ago
Could we say logical induction is the principle behind transfer learning then?
- spynxic 10y ago
  Emergent induction via high dimensional structure composition.. maybe. I haven't read any whitepages about the underlying processes P.S. note the hand-waviness when I use the word "emergent"
eridius 10y ago
Presumably what you're missing is the fact that this is translating from Y->Z in a single step, rather than doing two separate translations.
- spynxic 10y ago
  Then the high dimensional representation of the language that allows such a translation to be possible is profound
- mattkrause 10y ago
  They're certainly not doing English to Portuguese by way of Spanish or anything like that. However, you could almost read the paper as "We can translate something from a natural language into a high-dimensional representation, then turn that high-dimensional representation back into (another, possibly different) natural language. "
  - empath75 10y ago
    It seems similar to automatically captioning images.
    - mattkrause 10y ago
      Very much so! The two tasks are surprisingly interchangable. I once worked on a project where we used a statistical MT approach to "translate" between image features and captions--and I don't think we were the only ones trying such things. In a pleasing bit of symmetry, the attentional network used here looks like it was initially developed for image captioning.
mattkrause 10y ago
Well.... It is one of the oldest ideas in machine translation. In the late 1950s, Richens proposed an MT system that used an intermediate representation called an "interlingua." For each language, the system has two components, one that maps a natural language onto the interlingua, and a second that converts the interlingua back into a natural language. Use different source and destination pairs and--bam! you've got a multi-way translator. However, this only works if a) the interlingua is rich enough to capture the semantics of the to-be-translated text and b) the conversions between natural languages and interlingua also preserves those semantics. In practice, neither has worked pretty well. In the 1960s, the interlingua was typically hand-crafted and rules were either hand written, or later, induced, to convert natural languages to/from it. Since people have been working on this for 60 years, you can probably image how well that works[]. The clever bit here is that you don't really need to do that. If you have enough data, the LSTMs can learn it on their own (see Figure 2, for example, where semantically-related sentences from multiple languages end up in the same neighborhoods). [] Actually, somewhat better than you'd think. It certainly wouldn't have done Pushkin any justice, but it was surprisingly decent on weather reports (etc).
- csydas 10y ago
  I was going to comment on the article and idea of multilingual NN's in general since, as I'm learning Russian currently, I'm having a hard time seeing how you can get a good automated translation without some editorializing or awareness of what's going on in both sentences. Your comment on NN's "... not doing Pushkin justice" stood out since it's something I've been wondering how NN could handle a language like Russian that traditionally doesn't use the verb "to be" in sentences, much like the E-Prime idea that exists in English for clarity of description. Just trying to re-think in terms of a Russian sentence instead of an English sentence has also proven difficult, and I'd be curious how you can automate it on NN's well. My colleagues (russian natives) are all able to spot auto-translated messages from me nearly instantly since it messes up pretty rudimentary grammar rules or just doesn't phrase the sentence like you would in native Russian.
nl 10y ago
No one has managed to get induction working reliably for natural language tasks in any way that is generalizable enough to rely on. So that is new.

hota_mazi 10y ago

Fascinating. Maybe the next step will be to extract the tokenized interlingua language that's emerged in the neural network and map it to real words, and blam, we reinvent Esperanto!

eveningcoffee 10y ago
Esperanto is a lovely idea but any language needs an active support to stay current. If the language does not take inputs from culture, science and philosophy, it starts to degrade.
- renox 10y ago
  ? Nothing prevent adding new words to Esperanto, the most difficult part is reaching a consensus but that's also the case for 'normal' languages: there are many new French word introduced by the academie Française which aren't much used (courriel instead of mail, pourriel instead of spam, etc)..
  - eveningcoffee 10y ago
    It is not about language capability to accept new words but about language users motivation to do so.
klodolph 10y ago
It would probably not look anything like Esperanto, which is a charliefoxtrot of pidgin Spanish with some Turkish orthography thrown in. In all seriousness, however, different languages often have radically different concepts encoded into them. For example, in English sentences you might be forced to explicitly give subjects to the actors in your sentences, assign genders to your pronouns, and choose whether actions take place in the present or future, whether they are ongoing in general or happening at this moment, etc. In Japanese you might be forced to choose how to express your relationship with the person you are speaking to, the person you are speaking about, and decide whether a causal relationship is "if and only if" or just plain "if then". There's long been a hypothesis about whether there is some kind of "universal grammar" (Chomsky) but modern linguists do not really entertain that idea.

glandium 10y ago

The mentioned Japanese->English->Korean combination is one of the worst possible things to do. Both Korean and Japanese are very different from English, while similar to each other to some extent. Direct translation from one to the other would actually have much better results than translating back to Korean the (likely broken) English you get from Japanese. Edit: I do realize they're not talking about successive translations, but that's essentially how the training ends up happening, isn't it? A better example, IMHO, would have been three very different languages, like English, Japanese and Russian.

xbmcuser 10y ago
It is not translating from Japanese>English>korean it learned how to translate Japanese<>English Korean<>English now it knows how to translate Japanese<>Korean. In laymen terms it understands Korean and Japanese so can translate between them without needing examples of Japanese<>Korean translations.
paulsutter 10y ago
I'm guessing the downvotes here are based on your wording, "worst possible thing to do", and not your core point that Korean and Japanese are grammatically very similar. You're right that the test would be even more interesting with three very different languages. But it's still pretty impressive with Korean, English, and Japanese as shown, despite the similarity.

Smerity 10y ago

If people are interested in the underlying architecture of Google's Neural Machine Translation (GNMT) system, I wrote an article that builds it up piece by piece. While it's intended for people who are likely to implement GNMT or similar architectures, the article is descriptive enough that it should be possible to follow along even if you're not well versed in deep learning. http://smerity.com/articles/2016/google_nmt_arch.html http://smerity.com/articles/2016/google_nmt_arch.html The GNMT architecture is used almost as is for the zero-shot MT experiments. We're likely to see the GNMT architecture used extensively by Google for a variety of projects as they spent a deal of time and effort ensuring it is scalable to quite large datasets. Training a neural machine translation system with a single language pair is difficult - training it with multiple, especially all using the same set of weights, is insanely challenging! As an example, the GNMT architecture was used as the basis of "Generating Long and Diverse Responses with Neural Conversation Models", which trains on the entirety of Reddit (1.7 billion messages) as well as other various datasets. https://openreview.net/forum?id=HJDdiT9gl https://openreview.net/forum?id=HJDdiT9gl

gabrielgoh 10y ago
i see ray kurtzwell managed to sneak his name in as the last author. heh
alexbeloi 10y ago
Can you explain the odd looking connections to the decoder that appear in V2 and onwards? It's not entirely obvious to me from the description how the attention is applied. In the first part of the decoder rollout, the diagram connects the (concatenated?) output of the encoder into the middle(?) of the first output of the decoder rollout. In what sense? For the second rollout of the decoder the diagram shows 3 arrows going into the LSTM cell, what is that (left-facing) arrow on the right side doing?

sgentle 10y ago

This reminds me of Searle's Chinese Room Argument[0]: imagine you have a particularly dreary job where you sit in a room filled with boxes of symbols written on paper. Every now and again someone comes in and hands you some new symbols. You look through your rulebook and, depending on what it says, hand them some symbols back. It turns out that these rules actually implement a conversational program in Chinese. And if you can implement those rules and not understand Chinese, why would you think a computer program, implementing its own rules, could understand anything? The common "Systems Reply" response to this is that you're looking at the wrong layer of abstraction. The computer hardware (or the person in the room) doesn't understand Chinese, the computer plus the rules plus the data forms a system that understands Chinese. Searle's answer to this is that, well, what if you memorised the rules and the database? You might know all the rules, you might be able to follow them, but you wouldn't understand Chinese. What I think is fascinating about this is that it's vulnerable to Bayesian Judo[1]: if you have a strong belief that computers aren't capable of true understanding because of the Chinese Room Argument, then building an actual Chinese Room-style computer and having it show understanding should be a fairly strong blow to that belief. Now, it's easy to quibble about what true understanding actually means, but one version (used by Searle's answer) is this: "[..] he would not know the meaning of the Chinese word for hamburger. He still cannot get semantics from syntax." But this news is exactly that! A computer translation of the same semantic concept from one syntax to another without ever having been taught the rules connecting them. In other words, this is semantics from syntax implemented by nothing but a computer, a database, and a set of rules. So, by the reverse Chinese Room Argument, I would say this system exhibits a kind of understanding. Not a very sophisticated kind, mind you, but something that should still spook you if you believe computers are categorically incapable of thinking like us. [0] http://plato.stanford.edu/entries/chinese-room/ http://plato.stanford.edu/entries/chinese-room/ [1] http://lesswrong.com/lw/i5/bayesian_judo/ http://lesswrong.com/lw/i5/bayesian_judo/

timr 10y ago
"But this news is exactly that! A computer translation of the same semantic concept from one syntax to another without ever having been taught the rules connecting them." By that standard, statistical translation approaches were "understanding" a long time ago. The new thing here isn't that systems aren't being taught "the rules" (that wasn't happening in statistical MT either), the new thing is that there's a different kind of classifier in the "middle" now, which is representing a hidden state. This classifier is more flexible in a lot of ways, but also more of a black box, and takes a lot more effort to train without overfitting. It's cool that you can translate between language pairs that have never been explicitly trained, but let's not overstate the meaning of it. The blog post makes this rather breathless speculation: "Within a single group, we see a sentence with the same meaning but from three different languages. This means the network must be encoding something about the semantics of the sentence rather than simply memorizing phrase-to-phrase translations. We interpret this as a sign of existence of an interlingua in the network." This is...a fun story, but not much else. First off, you can make dimensionality reduction plots that "show" a lot of things. Even ignoring that issue, in translations of short sentences involving specific concepts (i.e. the example about the stratosphere), is it really surprising that you'd find clusters? The words in that sentence are probably unique enough that they'd form a distinct cluster in mappings from any translation system. Folks get caught up in the "neural" part of neural networks, and assume that some magical quasi-human thought is happening. If the tech were called "highly parameterized reconfigurable weighted networks of logistic classifiers", there'd be less loopy speculation.

ChuckMcM 10y ago

Nice piece of work and counts as "implementing Star Trek in the present". Now I just need a nice pair of noise cancelling over the ear headphones that let me hear english spoken no matter where I am :-)

agildehaus 10y ago
Stick a fish in your ear. Might get lucky.

vurpo 10y ago

I wonder how much memory this translation via an intermediate representation of a sentence takes. It seems like representing the semantic meaning of a sentence in a language-independent way would take a huge amount of data.

ximeng 10y ago
E.g. Japanese is 50MB to download for offline usage in Google Translate's iPhone app.

deleted 10y ago

[deleted]

YeGoblynQueenne 10y ago

>> We call this “zero-shot” translation, shown by the yellow dotted lines in the animation. To the best of our knowledge, this is the first time this type of transfer learning has worked in Machine Translation. I think it was last year when a friend was telling me how Google translates the Greek word for "swallow" (the bird) to French. Back then, the translation was the French word for "to swallow" (the verb). The bird and the action don't even sound remotely alike in Greek and neither are they spelled alike (the bird is "χελιδόνι" the action is "καταπίνω"; google trans. will at least give their correct pronounciation). My friend figured Google can't find enough examples between the two languages, so it goes via English ... were the two words are homonyms. I think that was last year, and certainly before September. So I gave it a try again today, and this is still what I get: Greek French χελιδόνι avaler chelidóni If you omit the accent on the "o" you don't even get the mistranslation- you get only the phonetic transcription of the Greek word in latin characters. Obviously the important thing here is not the one word that google translate gets wrong, but the fact that it doesn't really look like this "new" system is all that new, or that it does anything all that different from the previous one, or indeed that it improves things at all.

YeGoblynQueenne 10y ago

Google, and also Microsoft btw, absolutely need to be called out on this. They keep claiming that their translation systems work well, because they have reasonably good results between some language pairs, like English/French or English/Spanish, that are a) close linguistically b) have a lot of examples of translated documents and, more importantly, c) have many speakers who might use Google translate. For languages where none of the above holds, however, the results continue to be completely ridiculous, no matter what "new" technique Google (or MS) advertises. Since those languages are not spoken by as many people as English or Spanish etc, however, it's very hard for the user to figure out how attrociously bad their automatic translations are. Here's an example from my native Greek; this is a bit of news text from yesterday [1]: Λανθασμένη χαρακτηρίζει ο Κύπριος κυβερνητικός εκπρόσωπος, Νίκος Χριστοδουλίδης, την προσέγγιση, να μπαίνουν στο «ίδιο καλάθι» η Ελλάδα με την Τουρκία σε σχέση με το κυπριακό. And here's Google's translation: Incorrect characterizes Cypriot government spokesman Nikos Christodoulides, the approach to be put in "one basket" by Greece and Turkey in relation to Cyprus. So, the Cypriot government spokesman (well done) is put in a basket by Greece and Turkey (wait wut). Hey, maybe the guy wanted to be put in two baskets? [2] That's very typical of the way Google translates between Greek and English. For Google, it's Neural Networks leading us to a bright future where language barriers are eliminated thanks to Scienz! For Greeks, it's comedy gold. And it's the same for Russians, Polish, Finns, Swedes, Indians, Chinese, Hungarians... Still, Google keeps including those languages in the count of languages it "covers", because it's good advertisement and who can really dispute them anyway? _____________________ [1] http://www.kathimerini.gr/884845/article/epikairothta/politikh/xristodoylidhs-gia-kypriako-den-yphr3e-akraia-8esh-apo-thn-a8hna http://www.kathimerini.gr/884845/article/epikairothta/politi... [2] What's being said is more like: "The Cypriot government spokesman said that it's a mistake to treat Greece and Turkey in the same manner with regards to Cyprus".