6 ms·

Bilingual speaker here: the German translation produced was very poor quality compared to the English one; the German speaker was enunciating excessively clearl

Bilingual speaker here: the German translation produced was very poor quality compared to the English one; the German speaker was enunciating excessively clearly, sitting in a sound studio and sticking to short, simple phrases with clear grammar. From that point of view the demo was misleading - we heard an English speaker talking naturally and heard clear responses, but the other side of the conversation was quite broken, although possible to understand. If both sides were speaking in real world settings the demo would have been honest but far less impressive to casual watchers. Despite this misdirection I am still impressed with the amount it did manage to translate from the long, naturally-spoken English sentences.

simias 12y ago
I'm always a bit wary of those demonstrations, obviously the dialogue was rehearsed. It's hard to know how much has been hidden under the carpet for the sake of the presentation. For all we know the entirety of the machine translation could have been on playback. I'm wiling to give them some credit and assume it isn't the case though but it's pretty obvious they didn't choose the worst case scenario for a skype conversation. I hope they'll make a public beta soon, so that we can all try and see how this works in practice in the real world.
- DennisAleynikov 12y ago
  While it isn't microsoft's version, here's something I made at a hacakthon which daisy chains google's services to make something very similar. It's an extension of Google Hangouts and uses and at the moment only works in Chrome, although I am working on improving it and making it less reliant on Google's technology. http://goo.gl/3wC6b7 http://goo.gl/3wC6b7
- seanmcdirmid 12y ago
  I know the speech recognition researcher who worked on this project. It is all very real, and these demo situations don't always represent ideal conditions. I tried an early version of the speech-to-text component (1 or 2 years ago?) myself using an off-the-shelf microphone with a model that was completely generic (none of me speaking in the corpus). It worked surprisingly very well.
  - mtrimpe 12y ago
    If you know the researcher, do you also know the patent situation here perhaps? If I'm correct this is all using the deep neural nets matching on phonemes and I'm really wondering how I could use this if I wanted to. Is it something they license? Is it something that's perhaps not patented and would end up in CMU sphinx eventually?
skriticos2 12y ago
I agree, the German output was horrible. The German speaker spoke like a German lesson 101 speaker. Very simple sentences and she over-pronounced everything. Nobody speaks like that.
- knowaveragejoe 12y ago
  So, the TTS was terrible or the translated text itself was terrible? In the case of the latter, it's entirely understandable and correctable.
  - deleted 12y ago
    [deleted]
  - stewbrew 12y ago
    The German speaker, the woman. She spoke as if she were talking to somebody who started to learn German yesterday.
alister 12y ago
> the German translation produced was very poor quality compared to the English one They would have selected the best possible language pair for this demo, so we should expect it to have done very poorly if they picked, say, English<->Mandarin. What stands out is that they didn't pick Spanish as the other language. It's pretty universally agreed that Spanish[1] is the easiest language for English speakers to learn, and Portuguese is in the same ballpark[2], but German is significantly harder[3]. (And Russian, Chinese, and Arabic would be way to the right on an exponential graph.) I'm guessing that machine translation of English<->German, for some reason, must be easier than English<->Spanish. [1] There's a fairly authoritative study on this which I can't find it immediately. [2] Just from personal experience I find that English<->Portuguese with Google Translate is astonishingly good (in either direction): http://brazilsense.com/index.php?title=Getting_by_with_just_English_and_without_Portuguese#Google_Translate_for_2-way_conversation http://brazilsense.com/index.php?title=Getting_by_with_just_... [3] The difficulty of German vs Spanish is confirmed by an NSA (!) document that says that "Next to Vietnamese, German may be the most difficult for English-speaking students to learn for German has a difficult syntactical feature, the discontinuity of the predicate, which the others lack. Among French, Italian, and Spanish, there also seems to be only a slight difference in difficulty. It appears that these three are the easiest languages for English-speaking students to learn": http://www.nsa.gov/public_info/_files/cryptologic_spectrum/foreign_language.pdf http://www.nsa.gov/public_info/_files/cryptologic_spectrum/f...
- _s 12y ago
  Completely guessing - but I think the strict structure of German makes it easier for machines to translate? English and Spanish on the other hand break more of their rules than they stick to, so it makes it more "human" / "natural" and easier to relate to?
- frobozz 12y ago
  Your NSA quote is a little out of context. The article you link to states that German is the fourth-hardest language from a list of five comparatively easy languages. > Among Vietnamese, German, French, Italian and Spanish, Vietnamese may be the most difficult... Next to Vietnamese, German may be the most difficult This doesn't amount to being significantly harder, particularly in light of the statement (that you quoted) that there is just one feature that makes it harder than the other three. Speaking from personal experience (native English speaker, no prior difference in exposure to the two languages, simultaneous study of the two, similar teacher quality and curriculum), I found French harder than German. Although my single anecdote doesn't prove German to be easier, surely it suggests that the one I found easier couldn't be significantly harder.
infinitone 12y ago
The problem with solving this problem is that the entire dev team and anyone that demos it has to fluently know all the languages being used. I highly doubt even they all know german, spanish, mandarin to such a high caliber that they are able to detect common language vs robot translated garble. So what do you get in the end? This unimpressive video to native speakers and skeptical one to non-natives. The fact that at the start of the demo, she asks him if he knows german and he said no should be a red flag for the impending bad demo.
ekidd 12y ago
Why moconnor's observation matters, based on my personal experience learning French: - It's relatively easy to learn enough French to carry on a slow, clearly-enunciated conversation with a highly-cooperative speaker in a quiet room. Call it 350 hours of study, or B1 on the CEFRL scale: http://en.wikipedia.org/wiki/Common_European_Framework_of_Reference_for_Languages http://en.wikipedia.org/wiki/Common_European_Framework_of_Re... - If you want to watch the French dub of [i]The Game of Thrones[/i] in a noisy room, and actually follow the plot twists, it's whole different game. Call it 1,000 to 2,000 hours of study and exposure, perhaps more for some people. If you want to listen to standup comedy, it's usually even worse. In other words, it's surprisingly easy to establish basic communication, but you can break your heart trying to get really good. Going by this experience, and by lots of experiments with Google Translate, I would predict that machine translation will suffer from similar challenges: It will be easy to get the point across if everybody cooperates and speaks clearly. But it will be a long time before you can speak idiomatically and casually, at full speed, and pretend the translation system isn't there.

simias 12y ago

I'm always a bit wary of those demonstrations, obviously the dialogue was rehearsed. It's hard to know how much has been hidden under the carpet for the sake of the presentation. For all we know the entirety of the machine translation could have been on playback. I'm wiling to give them some credit and assume it isn't the case though but it's pretty obvious they didn't choose the worst case scenario for a skype conversation. I hope they'll make a public beta soon, so that we can all try and see how this works in practice in the real world.

DennisAleynikov 12y ago
While it isn't microsoft's version, here's something I made at a hacakthon which daisy chains google's services to make something very similar. It's an extension of Google Hangouts and uses and at the moment only works in Chrome, although I am working on improving it and making it less reliant on Google's technology. http://goo.gl/3wC6b7 http://goo.gl/3wC6b7
seanmcdirmid 12y ago
I know the speech recognition researcher who worked on this project. It is all very real, and these demo situations don't always represent ideal conditions. I tried an early version of the speech-to-text component (1 or 2 years ago?) myself using an off-the-shelf microphone with a model that was completely generic (none of me speaking in the corpus). It worked surprisingly very well.
- mtrimpe 12y ago
  If you know the researcher, do you also know the patent situation here perhaps? If I'm correct this is all using the deep neural nets matching on phonemes and I'm really wondering how I could use this if I wanted to. Is it something they license? Is it something that's perhaps not patented and would end up in CMU sphinx eventually?

skriticos2 12y ago

I agree, the German output was horrible. The German speaker spoke like a German lesson 101 speaker. Very simple sentences and she over-pronounced everything. Nobody speaks like that.

knowaveragejoe 12y ago
So, the TTS was terrible or the translated text itself was terrible? In the case of the latter, it's entirely understandable and correctable.
- deleted 12y ago
  [deleted]
- stewbrew 12y ago
  The German speaker, the woman. She spoke as if she were talking to somebody who started to learn German yesterday.

alister 12y ago

> the German translation produced was very poor quality compared to the English one They would have selected the best possible language pair for this demo, so we should expect it to have done very poorly if they picked, say, English<->Mandarin. What stands out is that they didn't pick Spanish as the other language. It's pretty universally agreed that Spanish[1] is the easiest language for English speakers to learn, and Portuguese is in the same ballpark[2], but German is significantly harder[3]. (And Russian, Chinese, and Arabic would be way to the right on an exponential graph.) I'm guessing that machine translation of English<->German, for some reason, must be easier than English<->Spanish. [1] There's a fairly authoritative study on this which I can't find it immediately. [2] Just from personal experience I find that English<->Portuguese with Google Translate is astonishingly good (in either direction): http://brazilsense.com/index.php?title=Getting_by_with_just_English_and_without_Portuguese#Google_Translate_for_2-way_conversation http://brazilsense.com/index.php?title=Getting_by_with_just_... [3] The difficulty of German vs Spanish is confirmed by an NSA (!) document that says that "Next to Vietnamese, German may be the most difficult for English-speaking students to learn for German has a difficult syntactical feature, the discontinuity of the predicate, which the others lack. Among French, Italian, and Spanish, there also seems to be only a slight difference in difficulty. It appears that these three are the easiest languages for English-speaking students to learn": http://www.nsa.gov/public_info/_files/cryptologic_spectrum/foreign_language.pdf http://www.nsa.gov/public_info/_files/cryptologic_spectrum/f...

_s 12y ago
Completely guessing - but I think the strict structure of German makes it easier for machines to translate? English and Spanish on the other hand break more of their rules than they stick to, so it makes it more "human" / "natural" and easier to relate to?
frobozz 12y ago
Your NSA quote is a little out of context. The article you link to states that German is the fourth-hardest language from a list of five comparatively easy languages. > Among Vietnamese, German, French, Italian and Spanish, Vietnamese may be the most difficult... Next to Vietnamese, German may be the most difficult This doesn't amount to being significantly harder, particularly in light of the statement (that you quoted) that there is just one feature that makes it harder than the other three. Speaking from personal experience (native English speaker, no prior difference in exposure to the two languages, simultaneous study of the two, similar teacher quality and curriculum), I found French harder than German. Although my single anecdote doesn't prove German to be easier, surely it suggests that the one I found easier couldn't be significantly harder.

infinitone 12y ago

The problem with solving this problem is that the entire dev team and anyone that demos it has to fluently know all the languages being used. I highly doubt even they all know german, spanish, mandarin to such a high caliber that they are able to detect common language vs robot translated garble. So what do you get in the end? This unimpressive video to native speakers and skeptical one to non-natives. The fact that at the start of the demo, she asks him if he knows german and he said no should be a red flag for the impending bad demo.

ekidd 12y ago

Why moconnor's observation matters, based on my personal experience learning French: - It's relatively easy to learn enough French to carry on a slow, clearly-enunciated conversation with a highly-cooperative speaker in a quiet room. Call it 350 hours of study, or B1 on the CEFRL scale: http://en.wikipedia.org/wiki/Common_European_Framework_of_Reference_for_Languages http://en.wikipedia.org/wiki/Common_European_Framework_of_Re... - If you want to watch the French dub of [i]The Game of Thrones[/i] in a noisy room, and actually follow the plot twists, it's whole different game. Call it 1,000 to 2,000 hours of study and exposure, perhaps more for some people. If you want to listen to standup comedy, it's usually even worse. In other words, it's surprisingly easy to establish basic communication, but you can break your heart trying to get really good. Going by this experience, and by lots of experiments with Google Translate, I would predict that machine translation will suffer from similar challenges: It will be easy to get the point across if everybody cooperates and speaks clearly. But it will be a long time before you can speak idiomatically and casually, at full speed, and pretend the translation system isn't there.