7 ms·
In a way, it's literally nonsense, in the, uh, sense that it can't makey any sense of what it's translating and is just choosing some output string at random (i
by YeGoblynQueenne 9y ago
In a way, it's literally nonsense, in the, uh, sense that it can't makey any sense of what it's translating and is just choosing some output string at random (i.e. with probability drawn from some distribution it has derived from its training examples).
Of course that's not what I mean above. Yes, the string I present is an example of the nonsense that comes out of purely statistical machine translation. It's a simple example and a very simple sentence, but that sort of noun-pronoun disagreement can completely distort the meaning of more complex sentences, not to mention longer passages where there are references to multiple persons across sentence boundaries.
Here's another example (original text by myself):
Πήγαμε με τη Βασιλική να δούμε το Μήτσο στο νοσοκομείο. Είχε σπάσει το πόδι του
και του βάλανε λάμες. Η Βασιλική, που είναι λίγο καψούρα μαζί του, του πήρε κάτι
σοκολάτες και λουλούδια, σα χαζό κοριτσάκι από καμμιά ρομαντική σειρά. Εγώ πήγα
για συμπαράσταση λαέ. Το χειρότερο ξέρεις ποιό είναι; Όχι μόνο ήτανε εκεί η μάνα
του, ο πατέρας του κι η αδερφή του, αλλά ήτανε κι η Μαρία, που τά 'χανε πέρσι.
Καταστροφή!
Google translates:
We went with Vassiliki to see Mitsos in the hospital. He had broken his leg
and he put blades on him. Basilica, who is a little bit of mascara with him,
got something chocolates and flowers, like a stupid little girl from any
romantic series. I went for support. Worst you know who it is? Not only was
the mother there his father and his sister, but she was Maria, who lost her
last year. Destruction!
My intended meaning:
Me and Vassiliki went to see Mitsos at the hospital. He had broken his leg
and he had to have metal plates inserted. Vassiliki, who has a bit of a crush
on him, got him some chocolates and flowers, like a silly little girl from some
rom-com. I just went along for solidarity. And you know what the worse part is?
His mother and father and his sister were there and not just them, but also Maria
his ex from last year. Disaster!
Examples like this reinforce my intuition that, in translation, it's impossible to get correct results while starting from the wrong premises. If you think about maths, for example. I'm pretty sure that you can train a statistical model (a translation model, actually) to try and generate results of various mathematical expressions. It's easy to see that such a model would often get the wrong results, because of a lack of any understanding of mathematics. In fact, you could very accurately say that it would produce a lot of nonsense. And even if it got the right answer- what would that prove? It would still be obvious that it doesn't know how to solve maths problems, just guess at their more likely solution.
It's also easy to see that we don't use statistical models to solve maths problems, because we don't need them: we know the rules of arithmetic etc that lead to the correct solutions. With translation, we don't know what those rules are (if they can even be represented as rules) so we go for statistical models instead- but that is only because we can't do anything better. Statistical machine translations are fundamentally flawed. They might be "good enough" for some situations, but they are effectively designed to fail, by not even trying to solve the actual problem of translation.
And so what they produce is nonsense- and it is nonsense even when it looks like it sorta, kinda makes sense. It's nonsense because it's basically, just random.
- mannykannot 9y agoI used to know a couple of people working on rules-based translation, but that approach seems to have been sidelined by the early successes of statistical methods. I hope that work has continued in the background, not only for its own sake, but because statistical methods may have reached a point of diminishing returns. The lack of understanding in statistical machine translation is a counter-argument I use whenever someone claims that its apparent success is a sign that the AI singularity is imminent, but I have to admit that the recent progress in statistical AI has led me to wonder if, just maybe, these methods are closing in on the fundamental methods of understanding. Perhaps John von Neumann's famous quip "Young man, in mathematics you don't understand things. You just get used to them" is actually generally true.
- mseebach 9y agoThis seems like an example of letting the perfect be the enemy of the good. Translation without any loss of veracity is extremely difficult, even for human translators, and failure to clear that bar doesn't constitute nonsense. If instead we consider that I don't know a single word of greek, nor even read the alphabet, I can put that text into Google translate, and with a minimum of contextual knowledge of Vassiliki and perhaps Maria, I can get perhaps 80% of the meaning of the paragraph with zero effort. Same deal for the sentence with the teacher; the number of situations where the gender of the teacher is the pivotal point of meaning in the sentence would be very low. That's huge, it's leaps and bounds beyond what I could even begin to achieve without months of training before. I have literally read and made basic sense of signs, posters and menus in greek without even being able to read the alphabet. How is this not substantial progress on "the actual problem of translation"?
- YeGoblynQueenne 9y agoLet's focus a bit at some of the translationd of sentences in my passage: He had broken his leg and he put blades on him. Basilica, who is a little bit of mascara with him, got something chocolates and flowers. Worst you know who it is? she was Maria, who lost her last year. Destruction! All that's absolute gibberish, obviously. The only reson you are able to make some sense out of it is that you have a human-level understanding of language and that you can place the nonsense strings generated by GT into an appropriate context that can give you some information about their intended meaning. Most however are too garbled to make any sense of them- the second string is the most striking one. This is not "good enough" (as opposed to perfect). That's just terribly bad. If you think it works for you that's only because you're so good at understanding language by dint of your being human (well- I assume). Of course human translators get it badly wrong. I've read human translations that are not much better than that nonsense above, to be honest. But a reasonably competent human translator will never make the kind of elementary mistakes that automatic translation does. And those elementary mistakes will always escalate to the point that the meaning of longer passages will deteriorate to gibberish. As to being able to understand basic signs, posters and menus in Greek- you can do that with a good travel guide also. And the travel guide will not claim to be a breakthrough Artificial Intelligence system, like Google claims for GT.