3 ms·
I’m not convinced that these are particularly useful tests of very large language models like GPT-4. These models have been trained on enormous amounts of text
by amluto 4y ago
I’m not convinced that these are particularly useful tests of very large language models like GPT-4. These models have been trained on enormous amounts of text, including theory of mind tests. And the example scenario in the paper is very simple and likely related to things that the models in question have seen before by a simple substitution.
So I would actually be a bit surprised if the models failed this type of test regardless of whether they have any understanding of theory of mind.
Note that I’m making no claim about how intelligent or understanding of humans or neurotypical these models are. My claim is that it’s very hard to tell given how incredibly knowledgeable the models are.
- nopassrecover 4y agoIt's interesting how quickly the Chinese Room has moved from thought experiment to reality https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
- amluto 4y agoFrom the description in the Wikipedia article, we’ve already reached the point where it’s quite interesting. An nVidia card and an iPhone are pretty clearly not intelligent (although the iPhone tries a bit harder). But these same devices, running the right software and given the right parameters, do something that seems impressively close to intelligence. And I suspect that my brain, with its software somehow removed, would not seem very intelligent. On the other hand, a human baby will spontaneously generate language and intelligence, and I haven’t heard of a language model doing that. (The experiment has, sadly, been done. https://en.m.wikipedia.org/wiki/Nicaraguan_Sign_Language https://en.m.wikipedia.org/wiki/Nicaraguan_Sign_Language )
- satvikpendem 4y agoAs someone that studied philosophy as well as computer science in college and had to hear many times people asking why I study philosophy at all, this is why. Philosophy often asks questions that are not yet reached by current technology but will be soon enough.