Even if Google Translate worked well for text (and it doesn't), translating text may be easier than translating the spoken word. When we speak, more things are

Even if Google Translate worked well for text (and it doesn't), translating text may be easier than translating the spoken word. When we speak, more things are left unsaid, because we can pick them up from nonverbal cues or from context. We also don't produce nice, grammatically-correct, complete sentences, we produce fragments instead, and make mistakes, and go back and correct things. And we slur things in ways that make it difficult for voice recognition to understand us. We also use intonation, timing, and so on in speech, which a voice recognition-translation-text to speech roundtrip probably destroys. We use less standardised, colloquial language which can be highly specific to groups of people, both in vocabulary and grammar. Real-time translation of speech is difficult even for interpreters. They have to be aware of the context (in some cases specifically prepared beforehand, depending on the topic) and make assumptions based on human experience. You'd need to make a computer that can do that, too. I suspect that a babelfish with current technology will work fine if you're listening to a prepared speech from a political leader, and poorly in actual social situations if people speak normally.