3 ms·
The big issue is that it literally knows nothing - there is no reference to a model of the real world such as humans use when thinking about the real world. It
by gibsonf1 5y ago
The big issue is that it literally knows nothing - there is no reference to a model of the real world such as humans use when thinking about the real world. It is a very advanced pattern matching parrot, and in using words like a parrot, knows nothing about what those words mean.
- PaulHoule 5y agoExactly, with "language in language out" it can pass as a neurotypical (passing as a neurotypical doesn't mean you get the right answer, it means if you get a wrong answer it is a neurotypical-passing wrong answer.) Actual "understanding" means mapping language to something such as an action (I tell you to get me the plush bear and you get me the plush bear,) precise computer code, etc.
- macrolocal 5y agoI'm inclined to agree, but positing that "the meaning of a word is its use in a language" is a perfectly respectable philosophical position. In this sense, GPT3 empirically bolsters Wittgenstein.
- PaulHoule 5y agoGiven that Chomksy's paradigm (and all the others) have failed so completely at automated language understanding I'm much more inclined to believe that the "language instinct" is a peripheral for an animal and if you want language understanding you need the animal or a simulation of the animal. Thus I reject Wittgenstein.
- macrolocal 5y agoThat's why it's so amazing that language models have gotten so far with just proximity statistics! Btw, this paper https://arxiv.org/abs/2106.07890 https://arxiv.org/abs/2106.07890 outlines a promising approach to mapping the semantics encoded in these models. Understanding their limits could make your rejection of Wittgenstein all the more precise.