4 ms·
Apart from the external dangers described (social, environmental), which I'm sure many will disagree with on multiple grounds, the article in general raises som
by tsimionescu 6y ago
Apart from the external dangers described (social, environmental), which I'm sure many will disagree with on multiple grounds, the article in general raises some very good points about the internal dangers these models pose to the field of NLP itself:
> The problem is, if one side of the communication does not have meaning, then the comprehension of the
implicit meaning is an illusion arising from our singular human
understanding of language (independent of the model). Contrary to how it may seem when we observe its output, an LM is a system
for haphazardly stitching together sequences of linguistic forms
it has observed in its vast training data, according to probabilistic
information about how they combine, but without any reference to
meaning: a stochastic parrot.
> However, from
the perspective of work on language technology, it is far from clear
that all of the effort being put into using large LMs to ‘beat’ tasks
designed to test natural language understanding, and all of the
effort to create new such tasks, once the existing ones have been
bulldozed by the LMs, brings us any closer to long-term goals of
general language understanding systems. If a large LM, endowed
with hundreds of billions of parameters and trained on a very large
dataset, can manipulate linguistic form well enough to cheat its
way through tests meant to require language understanding, have
we learned anything of value about how to build machine language
understanding or have we been led down the garden path?
- blackbear_ 6y agoDesigning benchmarks and metrics to test for deep understanding is really hard! Even our education system is nowhere close to a good solution. It would probably warrant an entire research field, maybe using NLP systems to evaluate how good benchmarks are at evaluating understanding.
- opwieurposiu 6y agoIn a recent conversation regarding colors, my three year old explained to me that clear:color as silent:sound. I thought this showed a pretty good grasp of meaning between these concepts. I have no idea if he came up with this "meaning" himself or if he learned it from some input corpus(likely the netflix kids show corpus). Later he related that because noises become quieter when they are far away, colors become more transparent when they are far away. This one I am pretty sure he came up with himself, as it is conflating "small angular size" with "transparent". My point is it not so easy to tell when humans are behaving as stochastic parrots, or when they produce language based on meaning. Indeed, this may be a distinction without a difference.
- tsimionescu 6y agoI think this is a perfect example of the difference, in fact. The first, corect, example could be either, but the second example shows a kind of mistake that couldn't really be made by a language model. Your son used (faulty) inference to deduce (1) silence:sound as clear:color, (2) distance => silence, therefore (3) distance => transparency. Premises 1 and 2 may be in the language training corpus, but conclusion 3 is definitely not, but still he produced this sentence. Another example is that human language use is clearly related to non-verbal situations. You can show me two fruit and I can say which is bigger without any language prompting. It may be true that our language faculty is in fact acting similarly to these LMs, but the overall human mind must somehow include a component that translates from our senses to language and back to motor control. There may or may not be other components as well, but this part is definitely required, and highly non-trivial.
- ineedasername 6y agoYou may be underestimating the possibility that your son has actual x-ray vision but just hasn't learned to fully control it yet.