4 ms·

I think an entity might be unable to mean things (or some things) but still able to understand what they mean. Kinda-trivial example: I cannot say and mean "I h

by gjm11 2y ago

I think an entity might be unable to mean things (or some things) but still able to understand what they mean. Kinda-trivial example: I cannot say and mean "I have a pain in the sixth finger on my right hand" because like most people I have only five fingers on my right hand, but I can understand reasonably well what it means. Using indexicals like this is a bit of a cheat, but you did the same in your very first example so I don't feel too bad about it. It's not so clear whether this can happen without them, but arguably it could; e.g., you might take some view along the lines of "statements are only meaningful when they occur within some sort of community" and then an entity isolated from such communities couldn't say-and-mean things, but might still understand what things mean. Anyway: you say (if I understand you right) that if those concepts diverge the only one you actually care about is agents meaning things. I think that's a mistake, because a lot of the questions we have reason to care about with today's AIs are not about that. Will AIs be able to do all our jobs better and cheaper than we can? That's about their external behaviour and how it relates to the world. Will AIs gain vast power and use it in ways that are very bad for us? Ditto. Will AIs enable new technological innovations that make us all much better off? Ditto. No one will be saying as the killbots destroy their cities "well, this isn't so bad; at least the machines don't really know what it is they're doing". No one will be saying as they enjoy the fruits of Fully Automated Luxury Gay Space Communism "you know, this whole thing feels empty because the machines that make all this possible don't really understand, they just behave as if they do". If a melon+string+ball arrangement is a faithful enough model of the solar system and it somehow enables me to send spaceships to Uranus, or to discover that orbits are elliptical when I hadn't known it before, or something, then that's a thing of great value. Your comment about children imitating adults sounds as if you haven't actually taken in the conditions I proposed, because children imitating what their parents say cannot in fact have the property I called "richness". If I talk to a child and they have learned to say some things about stars by listening to their parents, it will not help them when I ask them about something they haven't heard their parents say. (One can imagine a situation where the child "passes" this test by just relaying everything I say to the parent and then imitating what they say back to me. But the point there isn't that the child is imitating, it's that the child is not really part of the conversation at all, I'm just talking to the parent. And it is clear that nothing like that is happening with AI systems.) You may imagine that you make your argument more convincing by finishing it up with "anyone to whom this is not obvious is obviously religiously obsessed with some converse belief", but to me at least the opposite is the case.

mjburgess 2y ago
Well we need to get into what exactly gives rise for the ability to mean something -- it isnt as simple as being able to state something truthfully -- which is why I mentioned the imagination. It is more common that people have the ability to mean X in virtue of being able to imagine what it would be to say X and mean it. ie., the semantics of natural language are grounded in possibilities, and apprehending possibilities is the function of the imagination. I was trying to simplify matters enough to make it clear that if an LLM says, "I have a pen in my hand" it isn't even lying. I agree with you that the right test for proper language acquisition is modal: how would the system respond in situation S1..Sn. However the present mania for computational statistics has reduced this question to 'what is a y for a given x' as-if the relevant counterfactual was a permutation to the input to a pregiven function. The relevant counterfactuals are changes to the non-lignustic environments that language serves to describe. How is it that the parents continue to obtain this 'richness' and 'robustness' (ie., performance across changing environments) ? It is by themselves having the capacity to acquire and use meanings in relevant environments. This is something the children lack, and so do LLMs. For the children to imitate the parents, and the LLM to function as the community of speakers -- those speakers must narrate at length in a manner which can be imitated. If a parent looks at the sky and sees a roketship they can be asked "did you see that spaceship?!" -- but the very young chidlren cannot. They do not know what those words mean, and werent looking at the sky, their whole attention is on trying to immitate the sounds they hear. Likewise an LLM is limited in modelling non-lingustic shifts via waiting on enough new text being written on these shifts to be retrained on -- there is much reason to expect that no where near enough is written on almost all changes to our environment to enable this. The parents arent going to repeat, "there is a rocket ship in the sky" over-and-over just so the children can hear it. The parents dont need to: they can see. They do not need langauge to be responsive to lingusitic interrogation. The route LLMs use to obtain their performance is constructing a distribution over historical linguistic records of non-linguistic change, and sampling from this distribution. The mechanism we call 'intelligence' that employs meaning acquires such shifts by being-in-the-world to notice, engage, imagine, interrogate, create, etc. them. This is where I am making the strong empirical claim: sampling from a distribution over historical language use is 'not enough'. It's fragile and shallow -- though its shallowness is masked by the false (Turing-esq masquerade) that we have to interact with the system thru a single bamboozling I/O boundary: a prompt. Via such an extreme narrowing of how this supposed linguistic agent is free to employ meaning, its engineers can rig the situation so that its fragility isn't as apparent. But in the end, it will be so. The test for whether a system is using the meanings of words is indeed, modal: change the non-ligusitic environment (ie., the meanings) and the lanuage ought change. For LLMs this does not happen: they are very very very indirectly responsive to such shifts... because their mechanism of recording them is imitation.

by gjm11 2y ago

mjburgess 2y ago

Well we need to get into what exactly gives rise for the ability to mean something -- it isnt as simple as being able to state something truthfully -- which is why I mentioned the imagination. It is more common that people have the ability to mean X in virtue of being able to imagine what it would be to say X and mean it. ie., the semantics of natural language are grounded in possibilities, and apprehending possibilities is the function of the imagination. I was trying to simplify matters enough to make it clear that if an LLM says, "I have a pen in my hand" it isn't even lying. I agree with you that the right test for proper language acquisition is modal: how would the system respond in situation S1..Sn. However the present mania for computational statistics has reduced this question to 'what is a y for a given x' as-if the relevant counterfactual was a permutation to the input to a pregiven function. The relevant counterfactuals are changes to the non-lignustic environments that language serves to describe. How is it that the parents continue to obtain this 'richness' and 'robustness' (ie., performance across changing environments) ? It is by themselves having the capacity to acquire and use meanings in relevant environments. This is something the children lack, and so do LLMs. For the children to imitate the parents, and the LLM to function as the community of speakers -- those speakers must narrate at length in a manner which can be imitated. If a parent looks at the sky and sees a roketship they can be asked "did you see that spaceship?!" -- but the very young chidlren cannot. They do not know what those words mean, and werent looking at the sky, their whole attention is on trying to immitate the sounds they hear. Likewise an LLM is limited in modelling non-lingustic shifts via waiting on enough new text being written on these shifts to be retrained on -- there is much reason to expect that no where near enough is written on almost all changes to our environment to enable this. The parents arent going to repeat, "there is a rocket ship in the sky" over-and-over just so the children can hear it. The parents dont need to: they can see. They do not need langauge to be responsive to lingusitic interrogation. The route LLMs use to obtain their performance is constructing a distribution over historical linguistic records of non-linguistic change, and sampling from this distribution. The mechanism we call 'intelligence' that employs meaning acquires such shifts by being-in-the-world to notice, engage, imagine, interrogate, create, etc. them. This is where I am making the strong empirical claim: sampling from a distribution over historical language use is 'not enough'. It's fragile and shallow -- though its shallowness is masked by the false (Turing-esq masquerade) that we have to interact with the system thru a single bamboozling I/O boundary: a prompt. Via such an extreme narrowing of how this supposed linguistic agent is free to employ meaning, its engineers can rig the situation so that its fragility isn't as apparent. But in the end, it will be so. The test for whether a system is using the meanings of words is indeed, modal: change the non-ligusitic environment (ie., the meanings) and the lanuage ought change. For LLMs this does not happen: they are very very very indirectly responsive to such shifts... because their mechanism of recording them is imitation.