3 ms·

The phenomena of inner experience (2007) [pdf]

37 points by sysadm1n 3y ago

ulizzle 3y ago
Yeah, this argues for the theory (which I think is correct) that A.I can't reach consciousness since there is a lot of non-symbolic thinking going on (this study says approximately 25% percent). Does Chat GPT dream of electric sheep?
- nico 3y ago
  We will never know. That’s ineffable and subjective. We can only trust that it’s telling us the truth. Just like you can only trust someone else that they are telling you the truth about their own experience. Our language is essentially just and LLM hallucinating words to try to explain our non-symbolic experience. In other words, all experience is non-symbolic, but we are trying to symbolize it all the time. I subscribe to panpsychism, everything is conscious/ness. Consciousness is part of the fabric of reality/universe.
  - ulizzle 3y ago
    If everything were consciousness, chat GTP would be a digital Ouija board. But we've already seen that the computer, like George Washington, can't tell a lie: you can ask the chatbot "how" it would take over the world, but it can't be duplicitous. Crazy times.
    - User23 3y ago
      It's just a matter of prompting. It's relatively easy to prompt ChatGPT to tell all kinds of "lies." I use the quotes because of course a token generating statistical model has no concept of truth or falsehood and is thus incapable of lying. It's literally a bullshit generator in the technical sense described in the charming little book On Bullshit[1]. [1] https://press.princeton.edu/books/hardcover/9780691122946/on-bullshit https://press.princeton.edu/books/hardcover/9780691122946/on...
  - slibhb 3y ago
    > Just like you can only trust someone else that they are telling you the truth about their own experience. I don't trust them. I think they have inner experience because they're the same kind of thing I am and I have inner experience.
- hackinthebochs 3y ago
  >A.I can't reach consciousness since there is a lot of non-symbolic thinking going on There's no reason to think that an AI couldn't have non-symbolic modes of thought. Symbolic systems can be used to implement non-symbolic systems, which then have the constraints and limitations of the non-symbolic system. Neural nets are widely considered to be sub-symbolic systems. This was a core debate between the connectionists and those that argued symbolism was required for cognition and that neural nets were a dead end.
- antonvs 3y ago
  Most of the implementation of ChatGPT is not symbolic. It’s matrices of floating point numbers. This seems to undermine the premise of the theory you mention.
  - ulizzle 3y ago
    Numbers are symbolic themselves, as is all math.
  - loa_in_ 3y ago
    Logits are essential concept
Frummy 3y ago
Cool to see a classic quote I've seen in multiple places by now: Adam was watching two men carry a load of bricks in a construction site. He was wondering whether the men would drop the bricks. This wondering did not involve any symbols, but it was an explicit cognitive process It strikes at the heart of the thinking that makes up 99.9% of my life but seemingly most people never talk about it or are completely unaware of it.
- User23 3y ago
  You might enjoy digging into the philosophy of C.S. Peirce then, specifically his concept of "abductive reasoning."
- galaxyLogic 3y ago
  If there were no symbols, what did this "wondering" consist of? Did it really happen? Is it possible to "wonder" without using symbols? The example does not say it out aloud that it assumes you can do "wondering" without symbols. An example like that implies that what it describes as happening actually happened, therefore proves that that is how the world works. I think that is a logical fallacy even though I'm not sure if it has a name or official definition. The fallacy is that because in an imagined situation something is imagined as having happened, it would prove something about the actual world, without us also proving that such an imagined thing can actually happen in the real world.
- aradox66 3y ago
  aren't the men, and the bricks, and the dropping all symbols? Perhaps the visual stimulus / qualia could be considered non-symbolic, but surely the high level abstractions "men" "bricks" "drop" are, even if the experience of thinking did not include hearing the phonetic expression of these concepts as words
- antonvs 3y ago
  Men, bricks, a load thereof, and a construction site are all rather complex abstractions. What do you call the mental identifiers that represent these abstractions if not “symbols”? Whatever you may call them, that is in fact what is meant by “symbols” by the people using that term in this context. Essentially, an identifier that represents an abstraction. It’s an incredibly basic feature of all cognitive systems we’re aware of.
Lerc 3y ago
It seems that this is being viewed in the context of AI. There is a danger that people assume that the only way to do something is the way we do it. This is the "Your flying machine won't work because it doesn't have feathers" argument. Or more formally Entity X has property Y. Property Y is utilized to achieve property Z Entity A does not have property Y Therefore A does not have property Z (birds have feathers, birds use feathers to fly, Planes don't have feathers, planes can't fly) It is possibly worth pointing out that the word by word construction aspect of a LLM also cannot be used to eliminate the possibility of any form of consciousness. It would merely indicate that the generation of a response may be a series of very short term transient consciousnesses. Later tokens would just be new conscious 'instants' but with the context of the previous response. This is easy to see because the dividing line between instants is clear. If you turn it around and ask how do we know that we are not doing the same thing ourselves, it's too unclear to come up with an answer. Is you with a memory of yesterday the same as yesterday you? These are not new questions and rather than coming up with satisfactory answers we have managed to make the puzzle deeper by asking awkward questions like "what about transporters?" Sci-fi has explored these ideas quite a bit(Will Riker and Thomas Riker). Given the Star Trek reference I'll also do a mini rant about Discovery. The episode where they pick an arbitrary criteria, dreaming, as being a special property to be accepted as being an allowable person, is almost precisely this fallacy wrapped in a model of an established order dictating that 'what we have' is right. That they tried to deliver this as a parable of a minority being 'seen' makes it tragically ironic.
- User23 3y ago
  A better phrasing is "do submarines swim?" which is an obviously stupid question. They don't swim anymore than any other ship does. They do however do something that's a reasonable or even superior substitute for swimming in many cases.
- krispyfi 3y ago
  > Is you with a memory of yesterday the same as yesterday you? I love the way that Permutation City by Greg Egan treats this topic.

37 points by sysadm1n 3y ago

Yeah, this argues for the theory (which I think is correct) that A.I can't reach consciousness since there is a lot of non-symbolic thinking going on (this study says approximately 25% percent). Does Chat GPT dream of electric sheep?

nico 3y ago
We will never know. That’s ineffable and subjective. We can only trust that it’s telling us the truth. Just like you can only trust someone else that they are telling you the truth about their own experience. Our language is essentially just and LLM hallucinating words to try to explain our non-symbolic experience. In other words, all experience is non-symbolic, but we are trying to symbolize it all the time. I subscribe to panpsychism, everything is conscious/ness. Consciousness is part of the fabric of reality/universe.
- ulizzle 3y ago
  If everything were consciousness, chat GTP would be a digital Ouija board. But we've already seen that the computer, like George Washington, can't tell a lie: you can ask the chatbot "how" it would take over the world, but it can't be duplicitous. Crazy times.
  - User23 3y ago
    It's just a matter of prompting. It's relatively easy to prompt ChatGPT to tell all kinds of "lies." I use the quotes because of course a token generating statistical model has no concept of truth or falsehood and is thus incapable of lying. It's literally a bullshit generator in the technical sense described in the charming little book On Bullshit[1]. [1] https://press.princeton.edu/books/hardcover/9780691122946/on-bullshit https://press.princeton.edu/books/hardcover/9780691122946/on...
- slibhb 3y ago
  > Just like you can only trust someone else that they are telling you the truth about their own experience. I don't trust them. I think they have inner experience because they're the same kind of thing I am and I have inner experience.
hackinthebochs 3y ago
>A.I can't reach consciousness since there is a lot of non-symbolic thinking going on There's no reason to think that an AI couldn't have non-symbolic modes of thought. Symbolic systems can be used to implement non-symbolic systems, which then have the constraints and limitations of the non-symbolic system. Neural nets are widely considered to be sub-symbolic systems. This was a core debate between the connectionists and those that argued symbolism was required for cognition and that neural nets were a dead end.
antonvs 3y ago
Most of the implementation of ChatGPT is not symbolic. It’s matrices of floating point numbers. This seems to undermine the premise of the theory you mention.
- ulizzle 3y ago
  Numbers are symbolic themselves, as is all math.
- loa_in_ 3y ago
  Logits are essential concept

Frummy 3y ago

Cool to see a classic quote I've seen in multiple places by now: Adam was watching two men carry a load of bricks in a construction site. He was wondering whether the men would drop the bricks. This wondering did not involve any symbols, but it was an explicit cognitive process It strikes at the heart of the thinking that makes up 99.9% of my life but seemingly most people never talk about it or are completely unaware of it.

User23 3y ago
You might enjoy digging into the philosophy of C.S. Peirce then, specifically his concept of "abductive reasoning."
galaxyLogic 3y ago
If there were no symbols, what did this "wondering" consist of? Did it really happen? Is it possible to "wonder" without using symbols? The example does not say it out aloud that it assumes you can do "wondering" without symbols. An example like that implies that what it describes as happening actually happened, therefore proves that that is how the world works. I think that is a logical fallacy even though I'm not sure if it has a name or official definition. The fallacy is that because in an imagined situation something is imagined as having happened, it would prove something about the actual world, without us also proving that such an imagined thing can actually happen in the real world.
aradox66 3y ago
aren't the men, and the bricks, and the dropping all symbols? Perhaps the visual stimulus / qualia could be considered non-symbolic, but surely the high level abstractions "men" "bricks" "drop" are, even if the experience of thinking did not include hearing the phonetic expression of these concepts as words
antonvs 3y ago
Men, bricks, a load thereof, and a construction site are all rather complex abstractions. What do you call the mental identifiers that represent these abstractions if not “symbols”? Whatever you may call them, that is in fact what is meant by “symbols” by the people using that term in this context. Essentially, an identifier that represents an abstraction. It’s an incredibly basic feature of all cognitive systems we’re aware of.

Lerc 3y ago

It seems that this is being viewed in the context of AI. There is a danger that people assume that the only way to do something is the way we do it. This is the "Your flying machine won't work because it doesn't have feathers" argument. Or more formally Entity X has property Y. Property Y is utilized to achieve property Z Entity A does not have property Y Therefore A does not have property Z (birds have feathers, birds use feathers to fly, Planes don't have feathers, planes can't fly) It is possibly worth pointing out that the word by word construction aspect of a LLM also cannot be used to eliminate the possibility of any form of consciousness. It would merely indicate that the generation of a response may be a series of very short term transient consciousnesses. Later tokens would just be new conscious 'instants' but with the context of the previous response. This is easy to see because the dividing line between instants is clear. If you turn it around and ask how do we know that we are not doing the same thing ourselves, it's too unclear to come up with an answer. Is you with a memory of yesterday the same as yesterday you? These are not new questions and rather than coming up with satisfactory answers we have managed to make the puzzle deeper by asking awkward questions like "what about transporters?" Sci-fi has explored these ideas quite a bit(Will Riker and Thomas Riker). Given the Star Trek reference I'll also do a mini rant about Discovery. The episode where they pick an arbitrary criteria, dreaming, as being a special property to be accepted as being an allowable person, is almost precisely this fallacy wrapped in a model of an established order dictating that 'what we have' is right. That they tried to deliver this as a parable of a minority being 'seen' makes it tragically ironic.

User23 3y ago
A better phrasing is "do submarines swim?" which is an obviously stupid question. They don't swim anymore than any other ship does. They do however do something that's a reasonable or even superior substitute for swimming in many cases.
krispyfi 3y ago
> Is you with a memory of yesterday the same as yesterday you? I love the way that Permutation City by Greg Egan treats this topic.