8 ms·

Many philosophical traditions which incorporate a meditation practice emphasize that your consciousness is distinct from the contents of your thoughts. Meditati

Many philosophical traditions which incorporate a meditation practice emphasize that your consciousness is distinct from the contents of your thoughts. Meditation (even practiced casually) can provide a direct experience of this. When it comes to the various kinds of thought-processes that humans engage in (linguistic thinking, logic, math, etc) I agree that you can describe things in terms of functions that have definite inputs and outputs. So human thinking is probably computable, and I think that LLMs can be said to be ”think” in ways that are analogous to what we do. But human consciousness produces an experience (the experience of being conscious) as opposed to some definite output. I do not think it is computable in the same way. I don’t necessarily think that you need to subscribe to dualism or religious beliefs to explain consciousness - it seems entirely possible (maybe even likely) that what we experience as consciousness is some kind of illusory side-effect of biological processes as opposed to something autonomous and “real”. But I do think it’s still important to maintain a distinction between “thinking” (computable, we do it, AIs do it as well) and “consciousness” (we experience it, probably many animals experience it also, but it’s orthogonal to the linguistic or logical reasoning processes that AIs are currently capable of). At some point this vague experience of awareness may be all that differentiates us from the machines, so we shouldn’t dismiss it.

mont_tag 2y ago
> I think that LLMs can be said to be ”think” in ways that are analogous to what we do. ... But human consciousness produces an experience (the experience of being conscious) as opposed to some definite output. I do not think it is computable in the same way. "We've all been dancing around the basic issue: does Data have a soul?" -- Captain Louvois. https://memory-alpha.fandom.com/wiki/The_Measure_Of_A_Man_(episode) https://memory-alpha.fandom.com/wiki/The_Measure_Of_A_Man_(e...
- xpe 2y ago
  I for one (along with many thinkers) define intelligence as the extent to which an agent can solve a particular task. I choose the definition to separate it from issues involving consciousness. Both matter of course.
  - eru 2y ago
    And that's a useful and pragmatic definition, because it's very hard to measure the other definition even just for other humans.
- KoolKat23 2y ago
  That may be an illusion. And easily outputtable in the same way. Function calling in the output to release certain hormones etc.
osaariki 2y ago
We don’t know that LLMs generating tokens for scenarios involving simulations of conscious don’t already involve such experience. Certainly such threads of consciousness would currently be much less coherent and fleeting than the human experience, but I see no reason to simply ignore the possibility. To whatever degree it is even coherent to talk about the conscious experience of others than yourself (p-zombies and such), I expect that as AIs’ long term coherency improves and AI minds become more tangible to us, people will settle into the same implicit assumption afforded to fellow humans that there is consciousness behind the cognition.
- szvsw 2y ago
  The very tricky part then is to ask if the consciousness/phenomenological experience that you postulate still happens if, say, we were to compute the outputs of an LLM by hand… while difficult, if every single person on earth did one operation per second, plus some very complicated coordination and results gathering, we could probably predict a couple of tokens for an LLM at some moderate frequency… say, a couple of tokens a month? a week? A year? A decade? Regardless… would that consciousness still have an experience? Or is there some threshold of speed and coherence, or coloration that would be missing and result in failure for it to emerge? Impossible to answer. Btw I mostly think it’s reasonable to think that there might be consciousness, phenomenology etc are possible in silicon, but it’s tricky and unverifiable ofc.
  - Timwi 2y ago
    > would that consciousness still have an experience? If the original one did, then yes, of course. You're performing the exact same processing. Imagine if instead of an LLM the billions of people instead simulated a human brain. Would that human brain experience consciousness? Of course it would, otherwise they're not simulating the whole brain. The individual humans performing the simulation are now comparable to the individual neurons in a real brain. Similarly, in your scenario, the humans are just the computer hardware running the LLM. Apart from that it's the same LLM. Anything that the original LLM experiences, the simulated one does too, otherwise they're not simulating it fully.
sobellian 2y ago
> It's very difficult to find some way of defining rather precisely something we can do that we can say a computer will never be able to do. There are some things that people make up that say that, "While it's doing it, will it feel good?" or, "While it's doing it, will it understand what it's doing?" or some other abstraction. I rather feel that these are things like, "While it's doing it, will it be able to scratch the lice out of it's hair?" No, it hasn't got any hair nor lice to scratch from it, okay? > You've got to be careful when you say what the human does, if you add to the actual result of his effort some other things that you like, the appreciation of the aesthetic... then it gets harder and harder for the computer to do it because the human beings have a tendency to try to make sure that they can do something that no machine can do. Somehow it doesn't bother them anymore, it must have bothered them in earlier times, that machines are stronger physically than they are... - Feynman https://www.youtube.com/watch?v=ipRvjS7q1DI https://www.youtube.com/watch?v=ipRvjS7q1DI
- Seattle3503 2y ago
  "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra Maybe we can swap out "think" with "experience consciousness"
  - int_19h 2y ago
    You need to define "consciousness" first for the question to have any meaning, but all our definitions of consciousness seem to ultimately boil down to, "this thing that I'm experiencing".
- Xunjin 2y ago
  What about the famous solution provided by Descartes, “Cogito ergo sum”? Let's assume the fact that “we think”, so we can put it in a function to be computable, how is that going to prove that “I exist” for a machine? How is the machine going to perceive itself as a conscious being?
almarst 2y ago
What we have and AI don't are emotions. After all, that what animates us to survive and reproduce. Without emotions we can't classify and therefore store our experiences because there no reason to remember something which we are indifferent about. This includes everything not accessible by our senses. Our abilities are limited to what is needed for survival and reproduction because all the rest would consume our precious resources.
- nazgul17 2y ago
  The larger picture is that our brains are very much influenced by all the chemistry that happens around our units of computation (neurones); especially hormones. But (maybe) unlike consciousness, this is all "reproducible", meaning it can be part of the algorithm.
Nevermark 2y ago
> When it comes to the various kinds of thought-processes that humans engage in (linguistic thinking, logic, math, etc) I agree that you can describe things in terms of functions that have definite inputs and outputs. Function can mean inputs-outputs. But it can also mean system behaviors. For instance, recurrence is a functional behavior, not a functional mapping. Similarly, self-awareness is some kind of internal loop of information, not an input-output mapping. Specifically, an information loop regarding our own internal state. Today's LLMs are mostly not very recurrent. So might be said to be becoming more intelligent (better responses to complex demands), but not necessarily more conscious. An input-output process has no ability to monitor itself, no matter how capable of generating outputs. Not even when its outputs involve symbols and reasoning about concepts like consciousness. So I think it is fair to say intelligence and consciousness are different things. But I expect that both can enhance the other. Meditation reveals a lot about consciousness. We choose to eliminate most thought, focusing instead on some simple experience like breathing, or a concept of "nothing". Yet even with this radical reduction in general awareness, and our higher level thinking, we remain aware of our awareness of experience. We are not unconscious. To me that basic self-awareness is what consciousness is. We have it, even when we are not being analytical about it. In meditation our mind is still looping information about its current state, from the state to our sensory experience of our state, even when the state has been reduced so much. There is not nothing. We are not actually doing nothing. Our mental resting state is still a dynamic state we continue to actively process, that our neurons continue to give us feedback on, even when that processing has been simplified to simply letting that feedback of our state go by with no need to act on it in any way. So consciousness is inherently at least self-awareness in terms of internal access to our own internal activity. And that we retain a memory of doing this minimal active or passive self-monitoring, even after we resume more complex activity. My own view is that is all it is, with the addition of enough memory of the minimal loop, and a rich enough model of ourselves, to be able to consider that strange self-awareness looping state afterwards. Ask questions about its nature, etc.
- telegtron 2y ago
  This is what I wrote while I was thinking about the same topic before I can across your excellent comment; as if it’s a summary of what you just said: Consciousness is nothing but the ability to have internal and external senses, being able to enumerate them, recursively sense them, and remember the previous steps. If any of those ingredients are missing, you cannot create or maintain consciousness.
JauntTrooper 2y ago
When I was a kid, I used to imagine if that society ever developed AI, there would be widespread pushback to the idea that computers could ever develop consciousness. I imagined the Catholic Church, for example, would be publishing missives reminding everyone that only humans can have souls, and biologists would be fighting an quixotic battle to claim that consciousness can arise from physical structures and forces. I'm still surprised at how credulous and accepting societies have been of AI developments over the last few years.
- cmsj 2y ago
  Probably because we've been conditioned to accept that machines, no matter how friendly, are not really conscious in the way we are, so there is no risk of them needing to be treated differently than a hammer. AI developments over the last few years have not needed that view to change.
ghransa 2y ago
To state it's a turing machine might be a bit much but there might be a map between substrates to some degree, and computers can have a form of consciousness, an inner experience, basically the hidden layers and clearly the input of senses, but it wouldn't be the same qualia as a mind, I suspect it has more to due with chemputation and is dependent on the substrate doing the computing as opposed to a facility thereof, up to some accuracy limit, we can only detect light we have receptors for after all. To have qualia distinct to another being you need to compute on a substrate that can accurately fool the computation, fake sugar instead of sugar for example.
gmoot 2y ago
>it seems entirely possible (maybe even likely) that what we experience as consciousness is some kind of illusory side-effect of biological processes as opposed to something autonomous and “real”. I've heard this idea before but I have never been able to make head or tail of it. Consciousness can't be an illusion, because to have an illusion you must already be conscious. Can a rock have illusions?
- LPisGood 2y ago
  I think it’s more apt to say that free will is an illusion.
  - eru 2y ago
    Well, it entirely depends on how you even define free will. Btw, Turing machines provide some inspiration for an interesting definition: Turing (and Gödel) essentially say that you can't predict what a computer program does: you have to run it to even figure out whether it'll halt. (I think in general, even if you fix some large fixed step size n, you can't even predict whether an arbitrary program will halt after n steps or not, without essentially running it anyway.) Humans could have free will in the same sense, that you can't predict what they are doing, without actually simulating them. And by an argument implied by Turing in his paper on the Turing test, that simulation would have the same experience as the human would have had. (To go even further: if quantum fluctuations have an impact on human behaviour, you can't even do that simulation 100% accurately, because of the no cloning theorem. To be more precise: I'm not saying, like Penrose, that human brains use quantum computing. My much weaker claim is that human brains are likely a chaotic system, so even a very small deviation in starting conditions can quickly lead to differences in outcome. If you are only interested in approximate predictions, identical twins show that just getting the same DNA and approximation of the environment gets you pretty far in making good predictions. So cell level scans could be even better. But: not perfect.)
    - breuleux 2y ago
      > Humans could have free will in the same sense, that you can't predict what they are doing, without actually simulating them. I think it's a good point, but I would argue it's even more direct than that. Humans themselves can't reliably predict what they are going to do before they do it. That's because any knowledge we have is part of our deliberative decision-making process, so whenever we think we will do X, there is always a possibility that we will use that knowledge to change our mind. In general, you can't feed a machine's output into its input except for a very limited class of fixed point functions, which we aren't. So the bottom line is that seen from the inside, our self-model is a necessarily nondeterministic machine. We are epistemically uncertain about our own actions, for good reason, and yet we know that we cause them. This forms the basis of our intuition of free will, but we can't tell this epistemic uncertainty apart from metaphysical uncertainty, hence all the debate about whether free will is "real" or an "illusion". I'd say it's a bit of both: a real thing that we misinterpret.

mont_tag 2y ago

> I think that LLMs can be said to be ”think” in ways that are analogous to what we do. ... But human consciousness produces an experience (the experience of being conscious) as opposed to some definite output. I do not think it is computable in the same way. "We've all been dancing around the basic issue: does Data have a soul?" -- Captain Louvois. https://memory-alpha.fandom.com/wiki/The_Measure_Of_A_Man_(episode) https://memory-alpha.fandom.com/wiki/The_Measure_Of_A_Man_(e...

xpe 2y ago
I for one (along with many thinkers) define intelligence as the extent to which an agent can solve a particular task. I choose the definition to separate it from issues involving consciousness. Both matter of course.
- eru 2y ago
  And that's a useful and pragmatic definition, because it's very hard to measure the other definition even just for other humans.
KoolKat23 2y ago
That may be an illusion. And easily outputtable in the same way. Function calling in the output to release certain hormones etc.

osaariki 2y ago

We don’t know that LLMs generating tokens for scenarios involving simulations of conscious don’t already involve such experience. Certainly such threads of consciousness would currently be much less coherent and fleeting than the human experience, but I see no reason to simply ignore the possibility. To whatever degree it is even coherent to talk about the conscious experience of others than yourself (p-zombies and such), I expect that as AIs’ long term coherency improves and AI minds become more tangible to us, people will settle into the same implicit assumption afforded to fellow humans that there is consciousness behind the cognition.

szvsw 2y ago
The very tricky part then is to ask if the consciousness/phenomenological experience that you postulate still happens if, say, we were to compute the outputs of an LLM by hand… while difficult, if every single person on earth did one operation per second, plus some very complicated coordination and results gathering, we could probably predict a couple of tokens for an LLM at some moderate frequency… say, a couple of tokens a month? a week? A year? A decade? Regardless… would that consciousness still have an experience? Or is there some threshold of speed and coherence, or coloration that would be missing and result in failure for it to emerge? Impossible to answer. Btw I mostly think it’s reasonable to think that there might be consciousness, phenomenology etc are possible in silicon, but it’s tricky and unverifiable ofc.
- Timwi 2y ago
  > would that consciousness still have an experience? If the original one did, then yes, of course. You're performing the exact same processing. Imagine if instead of an LLM the billions of people instead simulated a human brain. Would that human brain experience consciousness? Of course it would, otherwise they're not simulating the whole brain. The individual humans performing the simulation are now comparable to the individual neurons in a real brain. Similarly, in your scenario, the humans are just the computer hardware running the LLM. Apart from that it's the same LLM. Anything that the original LLM experiences, the simulated one does too, otherwise they're not simulating it fully.

sobellian 2y ago

> It's very difficult to find some way of defining rather precisely something we can do that we can say a computer will never be able to do. There are some things that people make up that say that, "While it's doing it, will it feel good?" or, "While it's doing it, will it understand what it's doing?" or some other abstraction. I rather feel that these are things like, "While it's doing it, will it be able to scratch the lice out of it's hair?" No, it hasn't got any hair nor lice to scratch from it, okay? > You've got to be careful when you say what the human does, if you add to the actual result of his effort some other things that you like, the appreciation of the aesthetic... then it gets harder and harder for the computer to do it because the human beings have a tendency to try to make sure that they can do something that no machine can do. Somehow it doesn't bother them anymore, it must have bothered them in earlier times, that machines are stronger physically than they are... - Feynman https://www.youtube.com/watch?v=ipRvjS7q1DI https://www.youtube.com/watch?v=ipRvjS7q1DI

Seattle3503 2y ago
"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra Maybe we can swap out "think" with "experience consciousness"
- int_19h 2y ago
  You need to define "consciousness" first for the question to have any meaning, but all our definitions of consciousness seem to ultimately boil down to, "this thing that I'm experiencing".
Xunjin 2y ago
What about the famous solution provided by Descartes, “Cogito ergo sum”? Let's assume the fact that “we think”, so we can put it in a function to be computable, how is that going to prove that “I exist” for a machine? How is the machine going to perceive itself as a conscious being?

almarst 2y ago

What we have and AI don't are emotions. After all, that what animates us to survive and reproduce. Without emotions we can't classify and therefore store our experiences because there no reason to remember something which we are indifferent about. This includes everything not accessible by our senses. Our abilities are limited to what is needed for survival and reproduction because all the rest would consume our precious resources.

nazgul17 2y ago
The larger picture is that our brains are very much influenced by all the chemistry that happens around our units of computation (neurones); especially hormones. But (maybe) unlike consciousness, this is all "reproducible", meaning it can be part of the algorithm.

Nevermark 2y ago

> When it comes to the various kinds of thought-processes that humans engage in (linguistic thinking, logic, math, etc) I agree that you can describe things in terms of functions that have definite inputs and outputs. Function can mean inputs-outputs. But it can also mean system behaviors. For instance, recurrence is a functional behavior, not a functional mapping. Similarly, self-awareness is some kind of internal loop of information, not an input-output mapping. Specifically, an information loop regarding our own internal state. Today's LLMs are mostly not very recurrent. So might be said to be becoming more intelligent (better responses to complex demands), but not necessarily more conscious. An input-output process has no ability to monitor itself, no matter how capable of generating outputs. Not even when its outputs involve symbols and reasoning about concepts like consciousness. So I think it is fair to say intelligence and consciousness are different things. But I expect that both can enhance the other. Meditation reveals a lot about consciousness. We choose to eliminate most thought, focusing instead on some simple experience like breathing, or a concept of "nothing". Yet even with this radical reduction in general awareness, and our higher level thinking, we remain aware of our awareness of experience. We are not unconscious. To me that basic self-awareness is what consciousness is. We have it, even when we are not being analytical about it. In meditation our mind is still looping information about its current state, from the state to our sensory experience of our state, even when the state has been reduced so much. There is not nothing. We are not actually doing nothing. Our mental resting state is still a dynamic state we continue to actively process, that our neurons continue to give us feedback on, even when that processing has been simplified to simply letting that feedback of our state go by with no need to act on it in any way. So consciousness is inherently at least self-awareness in terms of internal access to our own internal activity. And that we retain a memory of doing this minimal active or passive self-monitoring, even after we resume more complex activity. My own view is that is all it is, with the addition of enough memory of the minimal loop, and a rich enough model of ourselves, to be able to consider that strange self-awareness looping state afterwards. Ask questions about its nature, etc.

telegtron 2y ago
This is what I wrote while I was thinking about the same topic before I can across your excellent comment; as if it’s a summary of what you just said: Consciousness is nothing but the ability to have internal and external senses, being able to enumerate them, recursively sense them, and remember the previous steps. If any of those ingredients are missing, you cannot create or maintain consciousness.

JauntTrooper 2y ago

When I was a kid, I used to imagine if that society ever developed AI, there would be widespread pushback to the idea that computers could ever develop consciousness. I imagined the Catholic Church, for example, would be publishing missives reminding everyone that only humans can have souls, and biologists would be fighting an quixotic battle to claim that consciousness can arise from physical structures and forces. I'm still surprised at how credulous and accepting societies have been of AI developments over the last few years.

cmsj 2y ago
Probably because we've been conditioned to accept that machines, no matter how friendly, are not really conscious in the way we are, so there is no risk of them needing to be treated differently than a hammer. AI developments over the last few years have not needed that view to change.

ghransa 2y ago

To state it's a turing machine might be a bit much but there might be a map between substrates to some degree, and computers can have a form of consciousness, an inner experience, basically the hidden layers and clearly the input of senses, but it wouldn't be the same qualia as a mind, I suspect it has more to due with chemputation and is dependent on the substrate doing the computing as opposed to a facility thereof, up to some accuracy limit, we can only detect light we have receptors for after all. To have qualia distinct to another being you need to compute on a substrate that can accurately fool the computation, fake sugar instead of sugar for example.

gmoot 2y ago

>it seems entirely possible (maybe even likely) that what we experience as consciousness is some kind of illusory side-effect of biological processes as opposed to something autonomous and “real”. I've heard this idea before but I have never been able to make head or tail of it. Consciousness can't be an illusion, because to have an illusion you must already be conscious. Can a rock have illusions?

LPisGood 2y ago
I think it’s more apt to say that free will is an illusion.
- eru 2y ago
  Well, it entirely depends on how you even define free will. Btw, Turing machines provide some inspiration for an interesting definition: Turing (and Gödel) essentially say that you can't predict what a computer program does: you have to run it to even figure out whether it'll halt. (I think in general, even if you fix some large fixed step size n, you can't even predict whether an arbitrary program will halt after n steps or not, without essentially running it anyway.) Humans could have free will in the same sense, that you can't predict what they are doing, without actually simulating them. And by an argument implied by Turing in his paper on the Turing test, that simulation would have the same experience as the human would have had. (To go even further: if quantum fluctuations have an impact on human behaviour, you can't even do that simulation 100% accurately, because of the no cloning theorem. To be more precise: I'm not saying, like Penrose, that human brains use quantum computing. My much weaker claim is that human brains are likely a chaotic system, so even a very small deviation in starting conditions can quickly lead to differences in outcome. If you are only interested in approximate predictions, identical twins show that just getting the same DNA and approximation of the environment gets you pretty far in making good predictions. So cell level scans could be even better. But: not perfect.)
  - breuleux 2y ago
    > Humans could have free will in the same sense, that you can't predict what they are doing, without actually simulating them. I think it's a good point, but I would argue it's even more direct than that. Humans themselves can't reliably predict what they are going to do before they do it. That's because any knowledge we have is part of our deliberative decision-making process, so whenever we think we will do X, there is always a possibility that we will use that knowledge to change our mind. In general, you can't feed a machine's output into its input except for a very limited class of fixed point functions, which we aren't. So the bottom line is that seen from the inside, our self-model is a necessarily nondeterministic machine. We are epistemically uncertain about our own actions, for good reason, and yet we know that we cause them. This forms the basis of our intuition of free will, but we can't tell this epistemic uncertainty apart from metaphysical uncertainty, hence all the debate about whether free will is "real" or an "illusion". I'd say it's a bit of both: a real thing that we misinterpret.