5 ms·

I understand what you're saying, but there's no subtlety or dual meaning in my examples - Google Translate is just taking in a word with a single, unambiguous m

I understand what you're saying, but there's no subtlety or dual meaning in my examples - Google Translate is just taking in a word with a single, unambiguous meaning, and returning a wholly unrelated word (or in the latter case gibberish). In other words they're cases where GT is less useful than a plain lookup database - which makes comparisons to human translators seem rather beside the point!

YeGoblynQueenne 9y ago
Fair point. My hunch is that there is in fact some subtle ambiguity associated with your example words, that is not immediately apparent for human spearkers of Japanese and English, simply because they have all sorts of "filters" that wring out the right meaning even for single words, filters that are missing from simple dumb statistical translation. However, it's often very hard to rev-eng the errors committed by statistical machine translation (and, generally, language processing) systems because they're basically just vectors of numbers that are completely meaningless to us. Which also makes it very hard to identify patterns of errors and correct them. Another thing that's very wrong with machine translation is the measures of "goodness" of translations. This is usually something like the BLEU score, that work by comparing the words in machine translation to those in human translation. Those are very problematic in all sorts of ways, not least the inability to select a "good" candidate translation among the many possible. In other words, it's not just that gathering statistics over words in context is not a very good way to model translations. On top of that, the results of such translations are very difficult to evaluate objectively. In general, machine translation is a very vaguely defined problem- which lets GT claim to be really, really good at it even when it's easy to see it really, really isn't.

YeGoblynQueenne 9y ago

Fair point. My hunch is that there is in fact some subtle ambiguity associated with your example words, that is not immediately apparent for human spearkers of Japanese and English, simply because they have all sorts of "filters" that wring out the right meaning even for single words, filters that are missing from simple dumb statistical translation. However, it's often very hard to rev-eng the errors committed by statistical machine translation (and, generally, language processing) systems because they're basically just vectors of numbers that are completely meaningless to us. Which also makes it very hard to identify patterns of errors and correct them. Another thing that's very wrong with machine translation is the measures of "goodness" of translations. This is usually something like the BLEU score, that work by comparing the words in machine translation to those in human translation. Those are very problematic in all sorts of ways, not least the inability to select a "good" candidate translation among the many possible. In other words, it's not just that gathering statistics over words in context is not a very good way to model translations. On top of that, the results of such translations are very difficult to evaluate objectively. In general, machine translation is a very vaguely defined problem- which lets GT claim to be really, really good at it even when it's easy to see it really, really isn't.