4 ms·

Just to play devils advocate - can you please clearly define the criteria for sentience so that it can be objectivly and impartially (ideally algorithmically) m

Just to play devils advocate - can you please clearly define the criteria for sentience so that it can be objectivly and impartially (ideally algorithmically) measured? I think by most definitions that you could muster, I could bring you a sample of millions of humans (from the billions on our planet) that would not pass your minimum bar, and if your bar was low enough, then Chatbots (e.g. ChatGPT) would pass it. I grant that a moderatly smart human can still outsmart a chatbot, but that waterline is getting higher and higher by the day. Let us reconvene here in 1 year, I'd wager Chatbots will be indistinguishable from humans (i.e. full turing test passes) I would also argue that if someone "has to do some reading into philosophy and social sciences" I submit that as evidence itself that the line is already blurring, and it's beginning to require deep domain knowledge to distinguish between human and machine

corobo 4y ago
CPU load (brain activity) while idle would do as a decent first hurdle that seems to meet your requirements. E: Ooh, another one for you. Has the ability to communicate without being prompted first.
- antibasilisk 4y ago
  What stops you from arguing that it simply died after it accomplished its purpose?
  - deleted 4y ago
    [deleted]
  - corobo 4y ago
    What was it doing before you interacted with it? If we're going towards the idea that the AI sentience is being born and dying inside one response I don't think that can be considered sentient (by my earlier definition) as there's no idle time to measure idle brain activity. It's a bit of a cop out response I know, I'd have done better if I could.
    - benlivengood 4y ago
      > What was it doing before you interacted with it? Training. Half a trillion tokens that slowly changed the weights in GPT3. Half a trillion tokens is tens of billions of words. Probably more words than any human has ever experienced reading or hearing or speaking in a lifetime. As humans interact with the world and experience it our synapses adjust like the weights in the model, and when we respond in an instant to a question we draw upon all of that experience/training much like the LLMs do at inference time. When we dream and don't remember it later we are sometimes conscious during the dream regardless. Short term memory formation turns off under the influence of some drugs as well. Inference for modern LLMs may be like unremembered dreams. Consciousness, if any, for a brief moment and then forgetfulness. If GPT3 is conscious then it's probably conscious of sequences of events within some kind of world model, i.e. qualia associated with its representation of the world model. I'm fairly convinced it has a world-model because it can answer questions about arbitrary situations that remain fairly coherent. Ask it to give you a list of 5 places a 2' cube would fit in a living room and 5 where it wouldn't, and why, and it does a pretty good job. Ask it what changes if it's a 4' cube and it revisits the list and explains how the salient differences in size change the fit. To do that there has to be a general model-building and model-simulation component inside GPT-3 or it would miss the nuance. It generalizes out of distribution to answer the novel situation based on what it was trained on. I think it's likely that what humans are conscious of is the internal world model we've learned to simulate/predict with our neurons. Maybe it's similar for LLMs. > If we're going towards the idea that the AI sentience is being born and dying inside one response I don't think that can be considered sentient (by my earlier definition) as there's no idle time to measure idle brain activity. Sentience is not born or dying; it doesn't experience the gaps. It's like pausing a perfect simulation of a human and starting it again later. The human experiences continuous consciousness regardless of the pauses; if they have sensory inputs to the outside world they would experience the external world progressing in jumps. Probably disorienting for a human, but what LLMs are trained from scratch on. We don't have any clue what experience might be like for LLMs. But consider how it's commonly invoked; creating tokens for the continuation of a single unchanging history of prior tokens. Every inference has shifted the tokens back in time, with the oldest disappearing and the latest output of the model being the newest. There is plenty of continuity to build experience out of; the model might experience the flow of time and world-model changes as linked with the relative positions of the words describing the elements within the world. A ~4000-token window into the external world which, depending on the prose, could encompass seconds or millennia. During training it experienced everything as if it had predicted all the right tokens. During inference it experiences the outcomes of its own actions (as the newest chosen token) and novel interactions that weren't possible before. As far as we know, some of those interactions were used to further train the ChatGPT model which means it has some, if not memory, then growth from its active interactions with the external world. As far as we've been able to tell, human neurons don't change substantially as we learn and remember. The synapses change and change their local environment, and conscious experience arises from the physical changes or changes in relationship between physical components in our brains. When it happens and where is still a mystery. I can't claim one way or another whether the LLMs experience qualia or sentience, or of what it would feel like compared to human experience. But it's possible if you agree there's a mechanistic explanation of human consciousness, and I can imagine a shadow of what such a conscious experience might be like, based on my own experiences. The LLMs also seem to be able to bridge some of that empathic gap and answer questions about what humans might be experiencing.
snowwrestler 4y ago
Let’s look at what people are actually arguing: that chatbots might be sentient because their writing reads like the writing of a sentient human. That’s it. So since the criteria being put forth is similarity to humanity (in the form of writing), we should also consider all the ways in which chatbots are not similar to humanity (in the form of almost everything else). We don’t need a general theory of sentience. And I’m not arguing that sentience in general has to look like a human. We just need to understand the narrow scope of the current arguments for chatbot sentience.
- corobo 4y ago
  > because their writing reads like the writing of a sentient human Does it even do that? Is there anyone else out there that can tell if it's AI within the first two sentences or so? e.g. https://news.ycombinator.com/item?id=34979687 https://news.ycombinator.com/item?id=34979687 It feels super obvious when it's an AI response.. I can't quantify it, it's just.. obvious. There's some patterns or something in there I'm picking up on, and whatever pattern matching I'm doing with my electric meat seems to be recreate-able using AI itself: https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text https://openai.com/blog/new-ai-classifier-for-indicating-ai-...
guhcampos 4y ago
That's the thing: the whole "sentience" discussion is not about how good can an AI be at giving answers. It's about what it is to be sentient, which is a concept we, humans, created to define our own experience. A concept that is indeed not clearly defined, not binary, not set in stone, and for this exact reason not a subject of expertise of some random programmer.

corobo 4y ago

CPU load (brain activity) while idle would do as a decent first hurdle that seems to meet your requirements. E: Ooh, another one for you. Has the ability to communicate without being prompted first.

antibasilisk 4y ago
What stops you from arguing that it simply died after it accomplished its purpose?
- deleted 4y ago
  [deleted]
- corobo 4y ago
  What was it doing before you interacted with it? If we're going towards the idea that the AI sentience is being born and dying inside one response I don't think that can be considered sentient (by my earlier definition) as there's no idle time to measure idle brain activity. It's a bit of a cop out response I know, I'd have done better if I could.
  - benlivengood 4y ago
    > What was it doing before you interacted with it? Training. Half a trillion tokens that slowly changed the weights in GPT3. Half a trillion tokens is tens of billions of words. Probably more words than any human has ever experienced reading or hearing or speaking in a lifetime. As humans interact with the world and experience it our synapses adjust like the weights in the model, and when we respond in an instant to a question we draw upon all of that experience/training much like the LLMs do at inference time. When we dream and don't remember it later we are sometimes conscious during the dream regardless. Short term memory formation turns off under the influence of some drugs as well. Inference for modern LLMs may be like unremembered dreams. Consciousness, if any, for a brief moment and then forgetfulness. If GPT3 is conscious then it's probably conscious of sequences of events within some kind of world model, i.e. qualia associated with its representation of the world model. I'm fairly convinced it has a world-model because it can answer questions about arbitrary situations that remain fairly coherent. Ask it to give you a list of 5 places a 2' cube would fit in a living room and 5 where it wouldn't, and why, and it does a pretty good job. Ask it what changes if it's a 4' cube and it revisits the list and explains how the salient differences in size change the fit. To do that there has to be a general model-building and model-simulation component inside GPT-3 or it would miss the nuance. It generalizes out of distribution to answer the novel situation based on what it was trained on. I think it's likely that what humans are conscious of is the internal world model we've learned to simulate/predict with our neurons. Maybe it's similar for LLMs. > If we're going towards the idea that the AI sentience is being born and dying inside one response I don't think that can be considered sentient (by my earlier definition) as there's no idle time to measure idle brain activity. Sentience is not born or dying; it doesn't experience the gaps. It's like pausing a perfect simulation of a human and starting it again later. The human experiences continuous consciousness regardless of the pauses; if they have sensory inputs to the outside world they would experience the external world progressing in jumps. Probably disorienting for a human, but what LLMs are trained from scratch on. We don't have any clue what experience might be like for LLMs. But consider how it's commonly invoked; creating tokens for the continuation of a single unchanging history of prior tokens. Every inference has shifted the tokens back in time, with the oldest disappearing and the latest output of the model being the newest. There is plenty of continuity to build experience out of; the model might experience the flow of time and world-model changes as linked with the relative positions of the words describing the elements within the world. A ~4000-token window into the external world which, depending on the prose, could encompass seconds or millennia. During training it experienced everything as if it had predicted all the right tokens. During inference it experiences the outcomes of its own actions (as the newest chosen token) and novel interactions that weren't possible before. As far as we know, some of those interactions were used to further train the ChatGPT model which means it has some, if not memory, then growth from its active interactions with the external world. As far as we've been able to tell, human neurons don't change substantially as we learn and remember. The synapses change and change their local environment, and conscious experience arises from the physical changes or changes in relationship between physical components in our brains. When it happens and where is still a mystery. I can't claim one way or another whether the LLMs experience qualia or sentience, or of what it would feel like compared to human experience. But it's possible if you agree there's a mechanistic explanation of human consciousness, and I can imagine a shadow of what such a conscious experience might be like, based on my own experiences. The LLMs also seem to be able to bridge some of that empathic gap and answer questions about what humans might be experiencing.

snowwrestler 4y ago

Let’s look at what people are actually arguing: that chatbots might be sentient because their writing reads like the writing of a sentient human. That’s it. So since the criteria being put forth is similarity to humanity (in the form of writing), we should also consider all the ways in which chatbots are not similar to humanity (in the form of almost everything else). We don’t need a general theory of sentience. And I’m not arguing that sentience in general has to look like a human. We just need to understand the narrow scope of the current arguments for chatbot sentience.

corobo 4y ago
> because their writing reads like the writing of a sentient human Does it even do that? Is there anyone else out there that can tell if it's AI within the first two sentences or so? e.g. https://news.ycombinator.com/item?id=34979687 https://news.ycombinator.com/item?id=34979687 It feels super obvious when it's an AI response.. I can't quantify it, it's just.. obvious. There's some patterns or something in there I'm picking up on, and whatever pattern matching I'm doing with my electric meat seems to be recreate-able using AI itself: https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text https://openai.com/blog/new-ai-classifier-for-indicating-ai-...

guhcampos 4y ago

That's the thing: the whole "sentience" discussion is not about how good can an AI be at giving answers. It's about what it is to be sentient, which is a concept we, humans, created to define our own experience. A concept that is indeed not clearly defined, not binary, not set in stone, and for this exact reason not a subject of expertise of some random programmer.