3 ms·
>> What matters for the purpose of human communication is the latter, not the former. Success at communication trumps your grammatical errors (...) But grammat
by YeGoblynQueenne 9y ago
>> What matters for the purpose of human communication is the latter, not the
former. Success at communication trumps your grammatical errors (...)
But grammatical errors cause communication to fail. Because they introduce
(measurable) noise to the signal, which obviously makes it harder to extract
information from it.
Btw, grammaticality is another uncontroversial measure of the performance of
NLP systems that are required to generate language (like machine translation
does). There is some theory behind the idea that well-formed sentences are
inherently more intelligible, that will take a lot more to overturn than "it
works for me".
To draw yet another analogy- homeopathy "works" for some people: they feel
better when they take its concoctions. That doesn't make homeopathy
objectively useful.
>> in the context of computer science, a heuristic is a technique where you take
an optimization problem (e.g. "finding the minimum of something", "the
shortest route to somewhere", etc) which is computationally untractable and
provide a "good enough" answer which can be computed.
Yes, looking at machine translation as a heuristic is a good point. Like I say
in other comments, the problem of machine translation has no good definition.
That's because it's very difficult to define what is a "good translation" in a
formal and principled manner. Hence, we're left with heuristics like "how
close this automatic translation is to this human translation" (where the
human translation is whatever happens to be available, rather than something
chosen for its quality or other attributes).
This is actually exactly what machine translation systems do. They are made to
optimise a measure of error over examples of human translation. So GT is
actually designed to do what you say is not important, produce grammatically
and syntactically correct language, like its examples are.
In that, it fails. It may achieve your goal of "communication" but that is not
really a heuristic, it's more of a quality that you, personally (and obviously
other people too) assign some value to. But it's far from a universally
accepted measure of machine translation quality. You might say that
"communication" is a business goal whereas matching human translations is an
engineering goal.
As a P.S., I'm not "claiming" to be working on a PhD. The information is in my
profile with links to my linkedin and github :)
- the_af 9y ago> But grammatical errors cause communication to fail. Because they introduce (measurable) noise to the signal, which obviously makes it harder to extract information from it. But when communication doesn't fail, grammatical errors don't matter that much. Because communication success is its own evidence (cue your "but how do you know [...]?" -- please don't. We've been down that road and your argument is unconvincing and solipsistic). You say grammar correctness is another uncontroversial measure of NLP, but nobody was arguing against this. Of course it is. You're shifting the goalpost. Grammar matters. You can tell when something is ungrammatical but whether this affects communication is another issue. It's also a matter of degree: something can be so ungrammatical it conveys no meaning; and something can be ungrammatical enough to sound "bad", but still convey its intended meaning. > Hence, we're left with heuristics like "how close this automatic translation is to this human translation" That's one heuristic, sure, and a pretty good one! Not the only one, though. In any case, you missed the point that your Math example was not very good, because it ignored that in a heuristic, what you proposed as a bad "translation" could actually be a good one. In the realm of human language translations, it's never a case of 4+4=8. > As a P.S., I'm not "claiming" to be working on a PhD. The information is in my profile with links to my linkedin and github :) My mistake then! No offense intended. PS: one more thing: I've looked at your comments history and I actually agree with your skepticism about AI. I think it's overhyped, especially by people with no background and no formal understanding about it -- like Eliezer, who you rightfully mock in your profile, and many other Prophets of the AI Apocalypse (or AI Utopia, depending on their mindset). So consider me a skeptic as well. We're more in agreement that it seems! I just don't think the kind of translation we're talking about is anywhere close to General AI or "understanding" of any form, and I can get excited about a "limited breakthrough" even though it's not actually about AI :)
- YeGoblynQueenne 9y agoWell, alright then. Let's agree to disagree so I won't say anything else. This thread's text is already pushing very aggressively towards the right side of my browser window anyway. >> No offense intended. ... and no offense taken, absolutely. We'll probably get the chance to argue again in the future, anyway, given that you're interested in the same subjects as I am :)