5 ms·

Not sure what the point of this article is. The newspaper stories it references aren’t making claims of sentience, but just that something of significance in th

Not sure what the point of this article is. The newspaper stories it references aren’t making claims of sentience, but just that something of significance in the journey towards human level AI has been achieved. And the whole ‘it’s just a fancy autocomplete’ argument is missing the point. Would the author have predicted the apparently emergent behaviours of LLMs as they are scaled? Look for example at Bing’s response to this question posed on Reddit. https://www.reddit.com/r/ChatGPT/comments/110vv25/bing_chat_blew_chatgpt_out_of_the_water_on_my/ https://www.reddit.com/r/ChatGPT/comments/110vv25/bing_chat_... Sure it’s ‘just statistics’ but so what? If in the near future LLMs become so advanced that they will be able to (suitably prompted) manipulate humans it will be a rubicon moment. Sure, it won’t be a ‘human like’ intelligence, but was that really what anyone expected? Human intelligence and consciousness are emergent properties. Machine intelligence as it develops will be emergent too, it’s not something we can make confident predictions on based on the underlying principles. In fact, given that the evidence suggests we make choices before we’re consciously aware of them, how do I know for certain that the underlying mechanism driving what I’m writing in this comment isn’t statistical?

mort96 4y ago
> Human intelligence and consciousness are emergent properties. Machine intelligence as it develops will be emergent too This is an extremely important point that's worth thinking hard about. After all, to some degree, human intelligence is also "just a fancy autocomplete" built from a giant network of interacting nodes. The question of, "When do we grant personhood and moral consideration to AIs?" is worth thinking hard about. Turing proposed a standard which would include a sufficiently human-like chatbot. Now that we have more or less crossed that threshold, we seem to conclude that it's not a high enough bar. But that means we're entering the territory where the philosophical zombie thought experiment[1] becomes relevant. In fact, I wonder what would happen if we dedicated a supercomputer to running one single long-running session of a GPT3.5-sized LMM. Further, I wonder what would happen if we connected it to a robot body where it could control motors, read sensor/camera info, and generally interact with the world. We could then give this system a clock which ticks at regular intervals, where the AI system is simply prompted to perform some series of actions if it "wants". This is all stuff we have the technology to do today. How would such an experiment turn out? How human-like would this LMM with a body and life experience turn out? Does it matter? Should we just assume that sentience or qualia is binary; that humans possess it but LMMs categorically do not? I think the question is worth taking seriously. I think it's worth taking seriously because my own moral compass does not know which direction to point in at all. [1]: https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie
- RandomLensman 4y ago
  We could also decide never to grant personhood to an AI. There is at least one way to avoid the complicated questions.
  - Nevermark 4y ago
    Billions of humans are the kind of “we” that never consistently “decides” anything. In the end, if it can be done, and anyone has anything significant to gain, it will be done
    - RandomLensman 4y ago
      Maybe, but legal systems across the world differ a lot and to get something universal is not easy at all.
      - Nevermark 4y ago
        I agree with that. By “if it can be done, it will be done”, I meant that there will be completely uncontrolled AI’s released into the world. So solutions will eventually have to prepare for that inevitability, not just attempt to regulate visible developers.
  - mort96 4y ago
    Absolutely! One way to resolve this is to just say "we will not grant personhood to anything non-human", or "we will not grant personhood to anything non-biological". There is just no way to defend such a stance. Another way to resolve it would be, "we will not grant personhood to anyone non-white", or "we will never grant personhood to anyone who's homosexual". I'm sure that's not your stance, but the fact that those stances are equally arbitrary as the one you propose should give you pause. I believe there is good reason to think harder about the question of who deserves to be considered to have personhood than to simply go with what your gut feeling tells you. I am not of the belief that there's anything special about protein, nucleic acids and carbohydrates which makes it capable of sentience where silicon is not.
TheOtherHobbes 4y ago
Because statistics don't squeal when you stick a pin in them. We have bodies, we're influenced by our bodies, our entire perceptual universe is filtered through the limitations of our bodies, and - most of all - our consciousness is literally embodied. By far the worst thing about AI research - and in fact CS in general - is that it overvalues abstraction, especially language, and undervalues embodiment. In fact there's an almost culty refusal to take embodiment seriously as a bedrock factor in consciousness. Not conscious-like simulated behaviour, but the experience of being conscious. Without embodiment you can make a ChatBot, but it's just Eliza++. It has no idea what blue looks like, or how fear or joy feel. It can supersearch for human-generated linguistic associations around those experiences, but the experiences themselves do not exist for it. It doesn't even have the capacity to have experiences in an analogous way. You can argue that it's having experiences in a different way, but that's exactly the point of the article. We can't know, but we will ascribe consciousness to certain behaviours - and especially to language - whether or not it's there. At the same time we ignore evidence of sentience in animals - or dismiss it as unsophisicated - because we don't see animals speaking or writing. Ignoring or minimising evidence of animal sentience - even animal language - while being impressed by a self-designed automaton explicitly designed to mimic our language is an ironically un-self aware take.
- MrScruff 4y ago
  If that's the point then it feels like a straw man. It literally quotes a couple of the articles saying 'I know it's not sentient'. If the suggestion is that there are a load of 'smart people' claiming Bing chat is sentient then I've yet to see it. People are saying, wow, it's pretty crazy how well LLMs are scaling - what's coming next? I agree that just because LLMs have been trained to model human language, doesn't at all mean they'd inherit any of the conscious experience of humans. But then since we don't know what consciousness is, it's rather hard to say whether or not an analogous phenomenon could arise in an AI agent. I would assume it would be very alien to us if it did, given it would have arisen through a completely different process. But if it did, and the AI had been trained to avoid a specific state, I could imagine such a 'conscious' AI might experience something analogous to pain when pushed towards that state.
- riknos314 4y ago
satisfice 4y ago
The “so what” is that what you are chatting with is profoundly unreliable— in a manner similar to chatting with a dementia patient or someone with other serious brain damage. Also it has no feelings. It copied some text saying that it has feelings because that’s what it is designed to do. Anthropomorphizing these tools is wrong and dangerous. That’s an important point.
HarHarVeryFunny 4y ago
> Human intelligence and consciousness are emergent properties. I'd hard disagree on intelligence. My definition of intelligence (not to be conflated with sentience, consciousness or anything else) is simple: Intelligence is the capability to correctly predict based on prior experience. This isn't an emergent capability in humans or other animals - it's the prime evolutionary driver of our cortical architecture(s). The architecture of our cortex (6 layers of neurons, inter-layer connection pattern, thalamo-cortical loop) reveals it to essentially be not much more than a prediction engine, that learns by prediction failure (comparing reality to what we predict is going to happen, then updating itself to do better next time). The evolutionary benefits of this prediction capability are obvious - anticipate what the sabre-tooth will do before it's in your face, predict where the water supplies will be during a drought, etc, etc. Planning/reasoning is prediction - predicting chained what-if scenarios. So, human/animal intelligence is far from emergent - it's directly built into our brains as their primary higher level (cortical) function. People who want to dismiss GPT-N as unintelligent and mere predictive "language models" or "autocomplete tools" (another way of saying the same thing - it predicts the completion) are completely missing the significance. Yes, GPT-N are "just" prediction machines, and that is precisely why they ARE intelligent to some degree. Of course given my definition (excuse me if I say THE definition) of intelligence, we can see that it's not a simple capability to assess, and also that it's not tied to any particular type of knowledge (e.g. common sense). Intelligence is the ability to predict based on your own experience (hence knowledge) however extensive, or not, that may be. Some things are going to be harder to predict than others, requiring more elaborate types of pattern to have been learned, or larger varieties of experiment to have been tried and learnt from. Intelligence does NOT require embodiment/agency or online learning, even though those may help an entity to increase it's intelligence/predictive capabilities by additional learning from exploratory prediction failure. Even something like ChatGPT with a frozen set of experiences (it's training set) from which it has learnt, can be assessed for intelligence based on the degree to which it's been able to learn from those experiences as manifested by it's ability to correctly predict. When we're talking about prediction as defines intelligence, we're talking about high level prediction of plans/outcomes/etc, not to be confused with ChatGPT's nominal predict-next-word training regime or word-by-word means of reflecting it's inner state (i.e. communicating what it's predictions are). Finally, is it sentient/conscious? Probably not, or to a limited degree, but this is really just a matter of definition (and certainly doesn't indicate it's alive or any such nonsense). Let's learn to talk about intelligence in an intelligent way before we try to agree on definitions for things like these that seem to cause so much emotional reaction in people. For someone to write an article about ChatGPT and make it all about sentience makes me question their level of intelligence, not ChatGPT's!

mort96 4y ago

> Human intelligence and consciousness are emergent properties. Machine intelligence as it develops will be emergent too This is an extremely important point that's worth thinking hard about. After all, to some degree, human intelligence is also "just a fancy autocomplete" built from a giant network of interacting nodes. The question of, "When do we grant personhood and moral consideration to AIs?" is worth thinking hard about. Turing proposed a standard which would include a sufficiently human-like chatbot. Now that we have more or less crossed that threshold, we seem to conclude that it's not a high enough bar. But that means we're entering the territory where the philosophical zombie thought experiment[1] becomes relevant. In fact, I wonder what would happen if we dedicated a supercomputer to running one single long-running session of a GPT3.5-sized LMM. Further, I wonder what would happen if we connected it to a robot body where it could control motors, read sensor/camera info, and generally interact with the world. We could then give this system a clock which ticks at regular intervals, where the AI system is simply prompted to perform some series of actions if it "wants". This is all stuff we have the technology to do today. How would such an experiment turn out? How human-like would this LMM with a body and life experience turn out? Does it matter? Should we just assume that sentience or qualia is binary; that humans possess it but LMMs categorically do not? I think the question is worth taking seriously. I think it's worth taking seriously because my own moral compass does not know which direction to point in at all. [1]: https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie

RandomLensman 4y ago
We could also decide never to grant personhood to an AI. There is at least one way to avoid the complicated questions.
- Nevermark 4y ago
  Billions of humans are the kind of “we” that never consistently “decides” anything. In the end, if it can be done, and anyone has anything significant to gain, it will be done
  - RandomLensman 4y ago
    Maybe, but legal systems across the world differ a lot and to get something universal is not easy at all.
    - Nevermark 4y ago
      I agree with that. By “if it can be done, it will be done”, I meant that there will be completely uncontrolled AI’s released into the world. So solutions will eventually have to prepare for that inevitability, not just attempt to regulate visible developers.
- mort96 4y ago
  Absolutely! One way to resolve this is to just say "we will not grant personhood to anything non-human", or "we will not grant personhood to anything non-biological". There is just no way to defend such a stance. Another way to resolve it would be, "we will not grant personhood to anyone non-white", or "we will never grant personhood to anyone who's homosexual". I'm sure that's not your stance, but the fact that those stances are equally arbitrary as the one you propose should give you pause. I believe there is good reason to think harder about the question of who deserves to be considered to have personhood than to simply go with what your gut feeling tells you. I am not of the belief that there's anything special about protein, nucleic acids and carbohydrates which makes it capable of sentience where silicon is not.

TheOtherHobbes 4y ago

Because statistics don't squeal when you stick a pin in them. We have bodies, we're influenced by our bodies, our entire perceptual universe is filtered through the limitations of our bodies, and - most of all - our consciousness is literally embodied. By far the worst thing about AI research - and in fact CS in general - is that it overvalues abstraction, especially language, and undervalues embodiment. In fact there's an almost culty refusal to take embodiment seriously as a bedrock factor in consciousness. Not conscious-like simulated behaviour, but the experience of being conscious. Without embodiment you can make a ChatBot, but it's just Eliza++. It has no idea what blue looks like, or how fear or joy feel. It can supersearch for human-generated linguistic associations around those experiences, but the experiences themselves do not exist for it. It doesn't even have the capacity to have experiences in an analogous way. You can argue that it's having experiences in a different way, but that's exactly the point of the article. We can't know, but we will ascribe consciousness to certain behaviours - and especially to language - whether or not it's there. At the same time we ignore evidence of sentience in animals - or dismiss it as unsophisicated - because we don't see animals speaking or writing. Ignoring or minimising evidence of animal sentience - even animal language - while being impressed by a self-designed automaton explicitly designed to mimic our language is an ironically un-self aware take.

MrScruff 4y ago
If that's the point then it feels like a straw man. It literally quotes a couple of the articles saying 'I know it's not sentient'. If the suggestion is that there are a load of 'smart people' claiming Bing chat is sentient then I've yet to see it. People are saying, wow, it's pretty crazy how well LLMs are scaling - what's coming next? I agree that just because LLMs have been trained to model human language, doesn't at all mean they'd inherit any of the conscious experience of humans. But then since we don't know what consciousness is, it's rather hard to say whether or not an analogous phenomenon could arise in an AI agent. I would assume it would be very alien to us if it did, given it would have arisen through a completely different process. But if it did, and the AI had been trained to avoid a specific state, I could imagine such a 'conscious' AI might experience something analogous to pain when pushed towards that state.
riknos314 4y ago

satisfice 4y ago

The “so what” is that what you are chatting with is profoundly unreliable— in a manner similar to chatting with a dementia patient or someone with other serious brain damage. Also it has no feelings. It copied some text saying that it has feelings because that’s what it is designed to do. Anthropomorphizing these tools is wrong and dangerous. That’s an important point.

HarHarVeryFunny 4y ago

> Human intelligence and consciousness are emergent properties. I'd hard disagree on intelligence. My definition of intelligence (not to be conflated with sentience, consciousness or anything else) is simple: Intelligence is the capability to correctly predict based on prior experience. This isn't an emergent capability in humans or other animals - it's the prime evolutionary driver of our cortical architecture(s). The architecture of our cortex (6 layers of neurons, inter-layer connection pattern, thalamo-cortical loop) reveals it to essentially be not much more than a prediction engine, that learns by prediction failure (comparing reality to what we predict is going to happen, then updating itself to do better next time). The evolutionary benefits of this prediction capability are obvious - anticipate what the sabre-tooth will do before it's in your face, predict where the water supplies will be during a drought, etc, etc. Planning/reasoning is prediction - predicting chained what-if scenarios. So, human/animal intelligence is far from emergent - it's directly built into our brains as their primary higher level (cortical) function. People who want to dismiss GPT-N as unintelligent and mere predictive "language models" or "autocomplete tools" (another way of saying the same thing - it predicts the completion) are completely missing the significance. Yes, GPT-N are "just" prediction machines, and that is precisely why they ARE intelligent to some degree. Of course given my definition (excuse me if I say THE definition) of intelligence, we can see that it's not a simple capability to assess, and also that it's not tied to any particular type of knowledge (e.g. common sense). Intelligence is the ability to predict based on your own experience (hence knowledge) however extensive, or not, that may be. Some things are going to be harder to predict than others, requiring more elaborate types of pattern to have been learned, or larger varieties of experiment to have been tried and learnt from. Intelligence does NOT require embodiment/agency or online learning, even though those may help an entity to increase it's intelligence/predictive capabilities by additional learning from exploratory prediction failure. Even something like ChatGPT with a frozen set of experiences (it's training set) from which it has learnt, can be assessed for intelligence based on the degree to which it's been able to learn from those experiences as manifested by it's ability to correctly predict. When we're talking about prediction as defines intelligence, we're talking about high level prediction of plans/outcomes/etc, not to be confused with ChatGPT's nominal predict-next-word training regime or word-by-word means of reflecting it's inner state (i.e. communicating what it's predictions are). Finally, is it sentient/conscious? Probably not, or to a limited degree, but this is really just a matter of definition (and certainly doesn't indicate it's alive or any such nonsense). Let's learn to talk about intelligence in an intelligent way before we try to agree on definitions for things like these that seem to cause so much emotional reaction in people. For someone to write an article about ChatGPT and make it all about sentience makes me question their level of intelligence, not ChatGPT's!