7 ms·

Microsoft demos breakthrough in real-time translated conversations

159 points by ghosh 12y ago

WorkingClassDev 12y ago
Fair play to Microsoft. Thats some impressive tech.
- Schwolop 12y ago
  Indeed. I wonder if they'll also want to call it Babel Fish? http://en.wikipedia.org/wiki/Yahoo!_Babel_Fish http://en.wikipedia.org/wiki/Yahoo!_Babel_Fish just confused me more, and I've now no idea who actually owns the trademark rights to that term. (It was AltaVista, bought by Yahoo!, spun-off into Bing Translate, which was partially bought by Microsoft, and ... I'm lost... )
  - frik 12y ago
    Yahoo/Altavista Bablefish and Google Translation (til 2007) were based on SYSTRAN software: http://en.wikipedia.org/wiki/Systran http://en.wikipedia.org/wiki/Systran (a desktop version for offline usage is even available) Now, both Bing Translator and Google Translation use a technology called statistical machine translation: http://en.wikipedia.org/wiki/Statistical_machine_translation http://en.wikipedia.org/wiki/Statistical_machine_translation
mcmire 12y ago
"Who thought you could have fun with the Germans?" Really, Kara? Anyway, obviously there were a couple of mistakes by the speech recognition, and they were taking care to speak very clearly, but still, I'm impressed. The future looks bright.
- m_mueller 12y ago
  I hate her live appearances with a passion (can't speak of her articles). How can someone that annoying get the best interviews in the industry? As an example, the landmark Bill Gates / Steve Jobs sitdown - all she did was annoy with sudo sexual innuendo, completely trivial questions and played up drama.
  - platz 12y ago
    I was surprised that Musk interviewed with them.
DennisAleynikov 12y ago
Back in April, at a hackathon sponsored in part by Microsoft, me and my friends built something we called Unilingo which was a working model of this idea! At the time we wondered why no other video chat applications did anything like this, but now it seems Microsoft stepped up to the plate and added translation to Skype!
- deleted 12y ago
  [deleted]
rlu 12y ago
Reminds me of this from a few years ago: https://www.youtube.com/watch?v=Nu-nlQqFCKg https://www.youtube.com/watch?v=Nu-nlQqFCKg I feel like they should merge the two. The subtitles are a nice touch, but if it could capture the voice of the speaker it would be truly magical.
nivla 12y ago
Kudos to Microsoft Research. It is very impressive especially given that it was an actual long distance live demo. Although that funny Blooper from Vista's speech recognition [1] will always remain stuck in my mind. I am guessing this is going to be the part of our future where language barrier slowly becomes the thing of the past. The most intriguing part of this demo was where Satya says that Machine learning technology gets better at previous languages as new languages are introduced. That in itself is an accidental revolution. [1] http://youtu.be/kX8oYoYy2Gc?t=43s http://youtu.be/kX8oYoYy2Gc?t=43s
eyeareque 12y ago
I remember seeing an article a few years back about msft filing a patent for automatic language translation for instant message. Maybe that is why we haven't seen this done by anyone else?
- stinos 12y ago
  Possibly - but maybe also because it apparently requires years and years of research to get to something that works properly. Not a lot of companies have the amounts of money and staff for that.
- chrischen 12y ago
  QQ (Tencent) Mobile app does automatic translation int its mobile messaging app. You tap on a message and it will translate it.
moconnor 12y ago
Bilingual speaker here: the German translation produced was very poor quality compared to the English one; the German speaker was enunciating excessively clearly, sitting in a sound studio and sticking to short, simple phrases with clear grammar. From that point of view the demo was misleading - we heard an English speaker talking naturally and heard clear responses, but the other side of the conversation was quite broken, although possible to understand. If both sides were speaking in real world settings the demo would have been honest but far less impressive to casual watchers. Despite this misdirection I am still impressed with the amount it did manage to translate from the long, naturally-spoken English sentences.
- simias 12y ago
  I'm always a bit wary of those demonstrations, obviously the dialogue was rehearsed. It's hard to know how much has been hidden under the carpet for the sake of the presentation. For all we know the entirety of the machine translation could have been on playback. I'm wiling to give them some credit and assume it isn't the case though but it's pretty obvious they didn't choose the worst case scenario for a skype conversation. I hope they'll make a public beta soon, so that we can all try and see how this works in practice in the real world.
  - DennisAleynikov 12y ago
    While it isn't microsoft's version, here's something I made at a hacakthon which daisy chains google's services to make something very similar. It's an extension of Google Hangouts and uses and at the moment only works in Chrome, although I am working on improving it and making it less reliant on Google's technology. http://goo.gl/3wC6b7 http://goo.gl/3wC6b7
  - seanmcdirmid 12y ago
    I know the speech recognition researcher who worked on this project. It is all very real, and these demo situations don't always represent ideal conditions. I tried an early version of the speech-to-text component (1 or 2 years ago?) myself using an off-the-shelf microphone with a model that was completely generic (none of me speaking in the corpus). It worked surprisingly very well.
vayarajesh 12y ago
Few years back when Google was working on Google Wave project (the new email) they also had done a similar translation.. but i think it was only at text and document level. Microsoft has come up with some real problem solving solutions specially when the world is more Globally connected. Kudos to Microsoft research.
CookWithMe 12y ago
As someone who is a native German speaker, I wasn't impressed. The translation to English was much better - because she spoke very clearly, slowly, and with simple grammar. The translation was no better than, say Google Translate. A mashup of Siri + Google Translate wouldn't have been any worse, and both technologies exist for some years. This is, unfortunately, hardly a breakthrough. Rather a 24h API hackathon project ;)
- DennisAleynikov 12y ago
  And one that my friends and I put together as an extension for Google Hangouts back in April, during a 36 hour hackathon in LA. Microsoft was one of the sponsors too.
- chrischen 12y ago
  Actually the Android google translate app already does this.. and probably better. If you tilt the app sideways it will alternate between the two languages and voice input.
- 3rd3 12y ago
  The beginning of the conversation made me crack up. It's almost like a Loriot sketch.
- pmelendez 12y ago
  " A mashup of Siri + Google Translate wouldn't have been any worse, and both technologies exist for some years." I have seen those already and they don't work nearly as well as this one. I guess the lady was trying really hard to do not screw up the demo and that might actually reduce the wow effect for German speakers.
- TazeTSchnitzel 12y ago
  I've done German at high school for three years or so. The contrast between how she had to speak German (ridiculously slowly as if trying to teach someone pronunciation) and how the English speaker spoke (pretty much normally) must say something about the confidence in their German speech recognition.
bsaul 12y ago
I know two sentences of german, and spotted one blattant mistake right from the first sentence : "wie gehts dir" and not "wie dir's geht". Not impressed as well.
aw3c2 12y ago
Could someone share a direct link to the video or a mirror? It is not available without Flash.
Piskvorrr 12y ago
ObXKCD: http://xkcd.com/678/ http://xkcd.com/678/ In other words, this has come a long way since "Dear aunt, let's set so double the killer delete select all", but I've seen enough "automated translation breakthroughs" to remain highly sceptical.
ape4 12y ago
Human interpreters (like at the UN) will be speaking and listening at the same time since the speaker doesn't wait for them. I was hoping for that.
hellbreakslose 12y ago
The way this is going, would take an eternity to communicate!
- iagooar 12y ago
  But it is: a) less expensive than getting a human translator b) faster than finding a human translator c) a good support for people who speak a little bit of the other's language to provide an additional help for understanding each other d) still far from perfect
IBalic 12y ago
This was actually really impressing
benjaminva 12y ago
I wonder why people are so excited about this. It's voice recognition coupled with an automated translator and a text-to-speech tool. Sure it all has to work fast enough but the English-to-German translation is horrible and probably much worse than a Google translate of the same sentence. For me as a native German speaker I would have been embarrassed being a Microsoft employee if they advertise something like that as "revolutionary" - it's not!
frik 12y ago
I remember some IBM TV-ads with a similar realtime IBM ViaVoice speech technology, more than 10 years ago. It was about a girl that answers a call in english, talking to an arabic speaking client of her father.
nsxwolf 12y ago
Anyone bothered by Nadella's comment that "...quite frankly none of us know exactly why" the system improves at one language when it learns another? What's going on there? Someone must know, no?
- Torn 12y ago
  Assuming it's related to putting more data into their machine learning model
- msaroufim 12y ago
  I believe Nadella was referring to a more general sentiment in the machine learning community about our lack of theoretical understanding of deep neural nets. I'd suspect that the network still learns some higher level features that exist across languages which is why it could become better in English after learning German.
deleted 12y ago
[deleted]
- deleted 12y ago
  [deleted]

159 points by ghosh 12y ago

WorkingClassDev 12y ago

Fair play to Microsoft. Thats some impressive tech.

Schwolop 12y ago
Indeed. I wonder if they'll also want to call it Babel Fish? http://en.wikipedia.org/wiki/Yahoo!_Babel_Fish http://en.wikipedia.org/wiki/Yahoo!_Babel_Fish just confused me more, and I've now no idea who actually owns the trademark rights to that term. (It was AltaVista, bought by Yahoo!, spun-off into Bing Translate, which was partially bought by Microsoft, and ... I'm lost... )
- frik 12y ago
  Yahoo/Altavista Bablefish and Google Translation (til 2007) were based on SYSTRAN software: http://en.wikipedia.org/wiki/Systran http://en.wikipedia.org/wiki/Systran (a desktop version for offline usage is even available) Now, both Bing Translator and Google Translation use a technology called statistical machine translation: http://en.wikipedia.org/wiki/Statistical_machine_translation http://en.wikipedia.org/wiki/Statistical_machine_translation

mcmire 12y ago

"Who thought you could have fun with the Germans?" Really, Kara? Anyway, obviously there were a couple of mistakes by the speech recognition, and they were taking care to speak very clearly, but still, I'm impressed. The future looks bright.

m_mueller 12y ago
I hate her live appearances with a passion (can't speak of her articles). How can someone that annoying get the best interviews in the industry? As an example, the landmark Bill Gates / Steve Jobs sitdown - all she did was annoy with sudo sexual innuendo, completely trivial questions and played up drama.
- platz 12y ago
  I was surprised that Musk interviewed with them.

DennisAleynikov 12y ago

Back in April, at a hackathon sponsored in part by Microsoft, me and my friends built something we called Unilingo which was a working model of this idea! At the time we wondered why no other video chat applications did anything like this, but now it seems Microsoft stepped up to the plate and added translation to Skype!

deleted 12y ago
[deleted]

rlu 12y ago

Reminds me of this from a few years ago: https://www.youtube.com/watch?v=Nu-nlQqFCKg https://www.youtube.com/watch?v=Nu-nlQqFCKg I feel like they should merge the two. The subtitles are a nice touch, but if it could capture the voice of the speaker it would be truly magical.

nivla 12y ago

Kudos to Microsoft Research. It is very impressive especially given that it was an actual long distance live demo. Although that funny Blooper from Vista's speech recognition [1] will always remain stuck in my mind. I am guessing this is going to be the part of our future where language barrier slowly becomes the thing of the past. The most intriguing part of this demo was where Satya says that Machine learning technology gets better at previous languages as new languages are introduced. That in itself is an accidental revolution. [1] http://youtu.be/kX8oYoYy2Gc?t=43s http://youtu.be/kX8oYoYy2Gc?t=43s

eyeareque 12y ago

I remember seeing an article a few years back about msft filing a patent for automatic language translation for instant message. Maybe that is why we haven't seen this done by anyone else?

stinos 12y ago
Possibly - but maybe also because it apparently requires years and years of research to get to something that works properly. Not a lot of companies have the amounts of money and staff for that.
chrischen 12y ago
QQ (Tencent) Mobile app does automatic translation int its mobile messaging app. You tap on a message and it will translate it.

moconnor 12y ago

Bilingual speaker here: the German translation produced was very poor quality compared to the English one; the German speaker was enunciating excessively clearly, sitting in a sound studio and sticking to short, simple phrases with clear grammar. From that point of view the demo was misleading - we heard an English speaker talking naturally and heard clear responses, but the other side of the conversation was quite broken, although possible to understand. If both sides were speaking in real world settings the demo would have been honest but far less impressive to casual watchers. Despite this misdirection I am still impressed with the amount it did manage to translate from the long, naturally-spoken English sentences.

simias 12y ago
I'm always a bit wary of those demonstrations, obviously the dialogue was rehearsed. It's hard to know how much has been hidden under the carpet for the sake of the presentation. For all we know the entirety of the machine translation could have been on playback. I'm wiling to give them some credit and assume it isn't the case though but it's pretty obvious they didn't choose the worst case scenario for a skype conversation. I hope they'll make a public beta soon, so that we can all try and see how this works in practice in the real world.
- DennisAleynikov 12y ago
  While it isn't microsoft's version, here's something I made at a hacakthon which daisy chains google's services to make something very similar. It's an extension of Google Hangouts and uses and at the moment only works in Chrome, although I am working on improving it and making it less reliant on Google's technology. http://goo.gl/3wC6b7 http://goo.gl/3wC6b7
- seanmcdirmid 12y ago
  I know the speech recognition researcher who worked on this project. It is all very real, and these demo situations don't always represent ideal conditions. I tried an early version of the speech-to-text component (1 or 2 years ago?) myself using an off-the-shelf microphone with a model that was completely generic (none of me speaking in the corpus). It worked surprisingly very well.

vayarajesh 12y ago

Few years back when Google was working on Google Wave project (the new email) they also had done a similar translation.. but i think it was only at text and document level. Microsoft has come up with some real problem solving solutions specially when the world is more Globally connected. Kudos to Microsoft research.

CookWithMe 12y ago

As someone who is a native German speaker, I wasn't impressed. The translation to English was much better - because she spoke very clearly, slowly, and with simple grammar. The translation was no better than, say Google Translate. A mashup of Siri + Google Translate wouldn't have been any worse, and both technologies exist for some years. This is, unfortunately, hardly a breakthrough. Rather a 24h API hackathon project ;)

DennisAleynikov 12y ago
And one that my friends and I put together as an extension for Google Hangouts back in April, during a 36 hour hackathon in LA. Microsoft was one of the sponsors too.
chrischen 12y ago
Actually the Android google translate app already does this.. and probably better. If you tilt the app sideways it will alternate between the two languages and voice input.
3rd3 12y ago
The beginning of the conversation made me crack up. It's almost like a Loriot sketch.
pmelendez 12y ago
" A mashup of Siri + Google Translate wouldn't have been any worse, and both technologies exist for some years." I have seen those already and they don't work nearly as well as this one. I guess the lady was trying really hard to do not screw up the demo and that might actually reduce the wow effect for German speakers.
TazeTSchnitzel 12y ago
I've done German at high school for three years or so. The contrast between how she had to speak German (ridiculously slowly as if trying to teach someone pronunciation) and how the English speaker spoke (pretty much normally) must say something about the confidence in their German speech recognition.

bsaul 12y ago

I know two sentences of german, and spotted one blattant mistake right from the first sentence : "wie gehts dir" and not "wie dir's geht". Not impressed as well.

aw3c2 12y ago

Could someone share a direct link to the video or a mirror? It is not available without Flash.

Piskvorrr 12y ago

ObXKCD: http://xkcd.com/678/ http://xkcd.com/678/ In other words, this has come a long way since "Dear aunt, let's set so double the killer delete select all", but I've seen enough "automated translation breakthroughs" to remain highly sceptical.

ape4 12y ago

Human interpreters (like at the UN) will be speaking and listening at the same time since the speaker doesn't wait for them. I was hoping for that.

hellbreakslose 12y ago

The way this is going, would take an eternity to communicate!

iagooar 12y ago
But it is: a) less expensive than getting a human translator b) faster than finding a human translator c) a good support for people who speak a little bit of the other's language to provide an additional help for understanding each other d) still far from perfect

IBalic 12y ago

This was actually really impressing

benjaminva 12y ago

I wonder why people are so excited about this. It's voice recognition coupled with an automated translator and a text-to-speech tool. Sure it all has to work fast enough but the English-to-German translation is horrible and probably much worse than a Google translate of the same sentence. For me as a native German speaker I would have been embarrassed being a Microsoft employee if they advertise something like that as "revolutionary" - it's not!

frik 12y ago

I remember some IBM TV-ads with a similar realtime IBM ViaVoice speech technology, more than 10 years ago. It was about a girl that answers a call in english, talking to an arabic speaking client of her father.

nsxwolf 12y ago

Anyone bothered by Nadella's comment that "...quite frankly none of us know exactly why" the system improves at one language when it learns another? What's going on there? Someone must know, no?

Torn 12y ago
Assuming it's related to putting more data into their machine learning model
msaroufim 12y ago
I believe Nadella was referring to a more general sentiment in the machine learning community about our lack of theoretical understanding of deep neural nets. I'd suspect that the network still learns some higher level features that exist across languages which is why it could become better in English after learning German.

deleted 12y ago

[deleted]

deleted 12y ago
[deleted]