6 ms·

I personally think this is a loaded concept. This is what I think: For the technology we have now (LLM)s there does not and will not ever exist a test that can

I personally think this is a loaded concept. This is what I think: For the technology we have now (LLM)s there does not and will not ever exist a test that can perfectly differentiate between an LLM and any human. We will always have LLMs that will pass the test and we will have humans that will fail. The reason for this is two fold and contradictory. The first reason is that consciousness is just a loaded concept. It's some arbitrary category with fuzzy boundaries. One persons definition of consciousness includes language another persons definition of consciousness includes logical thought. It's a made up collection of "features" that's it, no more or no less. It's a very arbitrary set of features too... mashed together for no rhyme or reason. The second reason is that LLMs already meet the criteria for what most people technically define as consciousness. The resistance we see nowadays is simply moving the goal posts. Five years ago people were in agreement that the turing test was a really good test. We've surpassed that and changed our definitions adding more stringent criteria for sentience. AI already meets the criteria for sentience the way we defined it from 1950-2020. Thus any test to measure sentience will always be a moving target.

smnplk 3y ago
Because you say consciousness is a loaded concept I will try to give you a description of what it actually is. I do agree that many people think different things when they talk about consciousness. It's your inner experience. Oh well, you can probably read this in a philosophy textbook. But what I think it really is, can only be explained with a thought experiment. Lets say you go into a pitch black sensory deprivation tank with salt water, so you can float in the water. Doctors give you injection that makes you forget about who you are, what is your name. You also forget about how to talk, you know no concept of the languge. You simply temporarily forget about you and what makes you really YOU in this world. The cemical in your blood also deprives you of any physical feeling, you don't feel any water around you. All there is to do is to gaze into darkness. And still...you are conscious, you have the inner experience. There is something like to be you, even in that state.
- Filligree 3y ago
  > And still...you are conscious, you have the inner experience. There is something like to be you, even in that state. I am honestly not convinced of this. I certainly can't imagine it.
  - thfuran 3y ago
    So if you're in a sensory deprivation tank, whether you're conscious depends on whether you're currently experiencing amnesia?
- cyber_kinetist 3y ago
  This is basically Cartesian dualism, which posits that the mind can be detached and conceived independently from the body... which I'm not convinced. Without any perception or relation to the physical world, and without memory or language, you wouldn't really "exist".
  - smnplk 3y ago
    I'm leaning more towards idealism than dualism. I think that there is no other thing but the mind. All that we call physical is a mental process. At least to me that is the ontology that makes the most sense. But i don't find it too comforting.
    - tmnvix 3y ago
      I feel the same way, but I actually take some comfort from it. Not to get too woo woo on you, but personal experience suggests to me that it is possible to be conscious without having an ego or sense of self and that a state of mind where that is largely absent is characterised by a lack of distinction between the vestigal self and everything else. I'm not dismissive of the idea that consciousness doesn't necessarily exist 'in the brain' but is in fact a fundamental part of the universe.
      - smnplk 3y ago
grantcas 3y ago
[dead]
pixl97 3y ago
Hell, we don't even have a good definition for intelligence that is accepted across different fields. The problem I see is we keep looking for supersets of all these features added together, and as the article points out, something that humans have.... >This rationale is particularly powerful if human experience is limited to a small, and perhaps idiosyncratic, region in the space of possible states of consciousness, as may be the case. There will be a large part of humanity that will want consciousness/sentience as something that humans have and other things don't (you already see this often with the 'has a soul' argument). This will set us up to have a large blind in to machine systems behavior in self organizing systems, and is a shared concern among those that see AI as a potential existential risk.
dr_dshiv 3y ago
Turing test was never meant as a test for consciousness. Intelligence — even AGI - that’s one thing. But “feelingness” or sentience— that’s a totally different matter.
- ben_w 3y ago
  The Argument from Consciousness This argument is very well expressed in Professor Jefferson's Lister Oration for 1949, from which I quote. “Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain—that is, not only write it but know that it had written it. No mechanism could feel (and not merely artificially signal, an easy contrivance) pleasure at its successes, grief when its valves fuse, be warmed by flattery, be made miserable by its mistakes, be charmed by sex, be angry or depressed when it cannot get what it wants.” This argument appears to be a denial of the validity of our test. According to the most extreme form of this view the only way by which one could be sure that a machine thinks is to be the machine and to feel oneself thinking. One could then describe these feelings to the world, but of course no one would be justified in taking any notice. Likewise according to this view the only way to know that a man thinks is to be that particular man. It is in fact the solipsist point of view. It may be the most logical view to hold but it makes communication of ideas difficult. A is liable to believe ‘A thinks but B does not’ whilst B believes ‘B thinks but A does not’. Instead of arguing continually over this point it is usual to have the polite convention that everyone thinks. -- I.—COMPUTING MACHINERY AND INTELLIGENCE A. M. TURING Mind, Volume LIX, Issue 236, October 1950, Pages 433–460, https://doi.org/10.1093/mind/LIX.236.433 https://doi.org/10.1093/mind/LIX.236.433
  - verisimi 3y ago
    Its easily conceivable now that one can create a machine that impersonates a human and emotions, and then that machine would be coded to write a sonnet or concerto. But, although it will give the impression (illusion) of feeling, it will still be running its code. Humans will have coded a fantastic illusionist, with no more emotional ability than any other tool, eg a hammer. > According to the most extreme form of this view the only way by which one could be sure that a machine thinks is to be the machine and to feel oneself thinking. This is true - one can only verify subjectivity in oneself. One might assume another living being does feel - but, we could be wrong - eg psychopathy can mean that others are unaware of a psychopath's "feelings" or lack of. It does seem to me, that if we were able to code an ai that gives the impression of feelings, composes concertos apparently independently, etc, that ai would have to be a sort of psychopath. Without getting too lost in spiritual BS, I think we are more than mere matter - the value in our experiences come from our emotions and feelings - not mere inputs that we label as 'emotions' and 'feelings' as would be the case for an ai. But emotions etc are subjective states, not objectively verifiable (even if we can see something correlating on a chart). Put simply, ai can never have a 'soul' even if it presents a wonderful appearance of kindliness, emotionality or whatever. Perhaps there will be no test in the world that will be able to find this out. But it still won't have feelings, it can only ever be a automaton.
EMM_386 3y ago
This particular study isn't limited to LLMs. It is about how to test consciousness in a wide variety of things. We have no tests. People may think their dog is conscious, but we can't test that. We can't even test if other people are conscious. What about invertebrates? Bacteria? Rocks, atoms, AI, your laptop etc? While somebody might say "yes, no, definetly not, yes, maybe" ... still, no tests. And consciousness in this particular proposal deals with whether or not it "‘feels like’ something to be the target system".
lukeschlather 3y ago
> AI already meets the criteria for sentience the way we defined it from 1950-2020. If that is the case why hasn't Kurzweil already won the bet? https://longbets.org/1/ https://longbets.org/1/
jprival 3y ago
>Five years ago people were in agreement that the turing test was a really good test The limitations of the Turing test have been discussed for much longer than that. Ironically its dependence on human judgement is arguably a weak point.
- ninetyninenine 3y ago
  Human judgement is the strongest judgment in existence. What other thing in reality has better judgment? Nothing.
ForHackernews 3y ago
> LLMs already meet the criteria for what most people technically define as consciousness. I strongly dispute this. Try "talking" to an LLM over time. It has no memory, no agency, no emotions, no desires, no pain, no sense of self. I think cats, dogs and certainly primates are far more conscious than these stochastic parrots. It do agree it's frightening how many people are fooled by a fancy autocomplete.
- naasking 3y ago
  > It has no memory, no agency, no emotions, no desires, no pain, no sense of self. It does have permanent memory, that's what training creates. It doesn't have agency. Whether it has desires, pain or sense of self is completely unknown, unless of course you have some mechanistic model of these phenomena such that we can see its presence in the human brain and see that LLMs do not have a comparable process. Of course you don't have such a model because it doesn't exist at this time. The problem is not that people have been fooled by stochastic parrots, it's that you've fooled yourself into thinking we understand more about the human mind than we actually do, and also possibly that humans are far more complex and special than might actually be the case, eg. that we are not basically stochastic parrots ourselves.
  - ForHackernews 3y ago
    Speaking only for myself, I am not a stochastic parrot. I have a rich and complex inner life. But maybe you are a p-zombie and you are right that it's tricky for you to prove to me that you're not. ;) I don't necessarily think humans are so special (cf. my mention of other mammals) but biology is extraordinarily messy and complex. Computer people tend to handwave the details with crude, reductive analogies (DNA is like source code! Neurons are like circuits!) but the reality is so much more overwhelming. Every single thing affects every other thing. Everything is probabilistic. There are multiple interacting chemical and electrical communication pathways. Neuroanatomy varies from individual to individual and in the same individual over time. The brain can radically adapt in response to trauma in a way that no computer model does. Even at the level of individual neurons[0] (of which there are 2.25 billion in a dog's brain) there's an enormous amount of "processing" happening. This is not like a computer processing - but physical, biological processes occurring that integrate informational content. I get it, everyone is very impressed with LLMs and I'm open to the possibility that some future model might achieve consciousness (to say nothing of sentience) but computer people in particular are way, way too credulous of this stuff and ignorant of what the messy, malfunctioning wetware in their head is actually doing. [0] e.g. https://www.nature.com/articles/nrn2331 https://www.nature.com/articles/nrn2331
imbnwa 3y ago
I think a lot of Computer Science people need to go read Kant
- whitepaint 3y ago
  What should they begin with?
  - deleted 3y ago
    [deleted]
  - h0p3 3y ago
    To my eyes, including for this topic, the most important work: //Groundwork of the Metaphysics of Morals//.
  - imbnwa 3y ago
    Reverse order of publication back to the first critique
deadbabe 3y ago
An LLM, in its perfect form is basically a p-zombie. There is no consciousness there. In its current form, its not much more than a really big and fast spreadsheet :)
- FrustratedMonky 3y ago
  Guess the reason this keeps coming up, is because 'how can you be sure?'. I'd say a lot of humans qualify as p-zombies. The problem is, can you ever test for it? To be sure. I'm not sure, that by the time you have a duplicate human, that it can be achievable without some inner subjective experience. Put an LLM on a loop and continually learning in a 3d world. Then ask what is going on inside.
  - deadbabe 3y ago
    If you truly believe people are p-zombies you may be a psychopath.
    - FrustratedMonky 3y ago
      Just some. What if the effort discussed in the article does find some way to test for this. But then some humans don't pass?
      - deadbabe 3y ago
        The those humans must have some kind of mental disability or the test is flawed, because every human is a conscious being.
        amanaplanacanal 3y ago
        How do you know that?
        yatopifo 3y ago
        I would argue I’m possibly the only conscious being by virtue of having the subjective experience of being conscious. While i can’t rule out that some other beings also possess this subjective experience, it would be a stretch for me to grant it to everyone a priori. That said, i suspect my own subjective experience of consciousness may be an illusion. After all, my mind certainly cannot exist outside of my head, and i know that someone or something skillfully tampering with my brain can alter my perception including that subjective experience. On what basis do you claim that every human (with reasonable exceptions) is a conscious being?
mirekrusin 3y ago
Well, you can use some tricks - ie. hook it up to your brain, let it learn for a while and tell us how it feels when we flip the switch to OFF.
thfuran 3y ago
>The second reason is that LLMs already meet the criteria for what most people technically define as consciousness. An LLM is an inert pile of data. You could perhaps argue that the process of inference exhibits consciousness, but the LLM itself certainly doesn't. There's simply no there there.
- BriggyDwiggs42 3y ago
  This is a bad argument. A brain frozen in ice also doesn’t demonstrate consciousness, and we don’t interpret that as a counterclaim to “the human brain exhibits consciousness.” Saying an LLM can be conscious implicitly means an LLM while it does things.
  - thfuran 3y ago
    >Saying an LLM can be conscious implicitly means an LLM while it does things. But that's not like saying a human brain is conscious while it does things. A brain is continuously doing a thing, whereas an LLM is either inert at rest or briefy doing a thing before returning to exactly the state it was in beforehand. There is no underlying process that could even conceivably support ongoing consciousness. If there is consciousness, it is present only for the span of its interrogation during an inference, after which it is immediately destroyed.
    - BriggyDwiggs42 3y ago
      Yes it would have to be transient, though thats not to say a continuous experience couldn’t be created by connecting together the transient conscious experiences in sequence because of the persistent state of the context window. My only point though was that nobody who says an LLM is conscious thinks the conscious part is the weights sitting on a hard drive.
      - thfuran 3y ago
        You are quite mistaken about the level of technical understanding required to make the claim that eg chatgpt is conscious. Many people have no conception of the distinction between the service and the model but still come to the conclusion through interacting with it that "chatgpt" is conscious.
jajag 3y ago
I think you're confusing intelligence with consciousness
- FrustratedMonky 3y ago
  "confusing intelligence with consciousness" Now that LLM's are becoming indistinguishable from humans, and other NN's are even solving proofs in geometry competitions. And there are NN that can fly an F-16 better than humans. All of sudden the goal post moved again. "No, that is just 'intelligence' not 'consciousness'." 12 months ago, nobody was making this distinction. We keep needing to redefine the problem to keep humans on top.
  - hax0ron3 3y ago
    The distinction between intelligence and consciousness has been discussed in philosophy of mind for many decades, perhaps centuries. I don't know much much it has been discussed in the context of LLMs, but I'm sure it's not an entirely new phenomenon. The debate is not just a matter of people trying to argue over how capable LLMs are. Many people pursue this topic because it is inherently interesting.
    - FrustratedMonky 3y ago
      I think even in philosophy of mind, Intelligence and Consciousness are seen to progress together. They follow the same trend, and at some magic point of enough intelligence, then we call it 'consciousness'. They follow same upward trend. This concept of having high intelligence that is not-conscious, or low-intelligence that is conscious. Seems relatively new.
      - thfuran 3y ago
        Go read Blindsight.
        FrustratedMonky 3y ago
hax0ron3 3y ago
The authors of the paper seem to define what they mean by consciousness quite precisely. They mean phenomenal consciousness: >The central goal of any C-test is to determine whether a target system has subjective and qualitative experience – often called ‘phenomenal consciousness’. Phenomenal consciousness is, as far as I can tell, binary. You either have subjective experience or you don't. Even if you are subjectively aware of almost nothing, you still have subjective experience. For example, when you wake up in the morning your awareness might be quite limited for the first few seconds - let's say you have your eyes closed and the only thing you are aware of is the warmness of the blanket. But still, you are aware. You have subjective experience. Whereas in deep sleep you do not. Phenomenal consciousness is thus pretty well-defined. It's just that we have zero understanding of how physical systems could possibly give rise to it, and it might be in principle impossible to ever figure that out. It is possible that physical systems do not in fact give rise to consciousness, although obviously the presence vs absence of consciousness in humans is at least to some extent correlated with certain measurable physical states. I doubt that any actual test for consciousness is in principle possible, but the authors of this paper do at least clearly define what they mean when they refer to consciousness. They mean the presence of subjective experience, not intelligence or language or logical thought or having a model of oneself or any other things.
naasking 3y ago
> The first reason is that consciousness is just a loaded concept. It's some arbitrary category with fuzzy boundaries It is right now. We might one day devise a mechanistic explanation for consciousness though, in which case any system that follows that mechanistic process would be conscious.

smnplk 3y ago

Because you say consciousness is a loaded concept I will try to give you a description of what it actually is. I do agree that many people think different things when they talk about consciousness. It's your inner experience. Oh well, you can probably read this in a philosophy textbook. But what I think it really is, can only be explained with a thought experiment. Lets say you go into a pitch black sensory deprivation tank with salt water, so you can float in the water. Doctors give you injection that makes you forget about who you are, what is your name. You also forget about how to talk, you know no concept of the languge. You simply temporarily forget about you and what makes you really YOU in this world. The cemical in your blood also deprives you of any physical feeling, you don't feel any water around you. All there is to do is to gaze into darkness. And still...you are conscious, you have the inner experience. There is something like to be you, even in that state.

Filligree 3y ago
> And still...you are conscious, you have the inner experience. There is something like to be you, even in that state. I am honestly not convinced of this. I certainly can't imagine it.
- thfuran 3y ago
  So if you're in a sensory deprivation tank, whether you're conscious depends on whether you're currently experiencing amnesia?
cyber_kinetist 3y ago
This is basically Cartesian dualism, which posits that the mind can be detached and conceived independently from the body... which I'm not convinced. Without any perception or relation to the physical world, and without memory or language, you wouldn't really "exist".
- smnplk 3y ago
  I'm leaning more towards idealism than dualism. I think that there is no other thing but the mind. All that we call physical is a mental process. At least to me that is the ontology that makes the most sense. But i don't find it too comforting.
  - tmnvix 3y ago
    I feel the same way, but I actually take some comfort from it. Not to get too woo woo on you, but personal experience suggests to me that it is possible to be conscious without having an ego or sense of self and that a state of mind where that is largely absent is characterised by a lack of distinction between the vestigal self and everything else. I'm not dismissive of the idea that consciousness doesn't necessarily exist 'in the brain' but is in fact a fundamental part of the universe.
    - smnplk 3y ago

grantcas 3y ago

[dead]

pixl97 3y ago

Hell, we don't even have a good definition for intelligence that is accepted across different fields. The problem I see is we keep looking for supersets of all these features added together, and as the article points out, something that humans have.... >This rationale is particularly powerful if human experience is limited to a small, and perhaps idiosyncratic, region in the space of possible states of consciousness, as may be the case. There will be a large part of humanity that will want consciousness/sentience as something that humans have and other things don't (you already see this often with the 'has a soul' argument). This will set us up to have a large blind in to machine systems behavior in self organizing systems, and is a shared concern among those that see AI as a potential existential risk.

dr_dshiv 3y ago

Turing test was never meant as a test for consciousness. Intelligence — even AGI - that’s one thing. But “feelingness” or sentience— that’s a totally different matter.

ben_w 3y ago
The Argument from Consciousness This argument is very well expressed in Professor Jefferson's Lister Oration for 1949, from which I quote. “Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain—that is, not only write it but know that it had written it. No mechanism could feel (and not merely artificially signal, an easy contrivance) pleasure at its successes, grief when its valves fuse, be warmed by flattery, be made miserable by its mistakes, be charmed by sex, be angry or depressed when it cannot get what it wants.” This argument appears to be a denial of the validity of our test. According to the most extreme form of this view the only way by which one could be sure that a machine thinks is to be the machine and to feel oneself thinking. One could then describe these feelings to the world, but of course no one would be justified in taking any notice. Likewise according to this view the only way to know that a man thinks is to be that particular man. It is in fact the solipsist point of view. It may be the most logical view to hold but it makes communication of ideas difficult. A is liable to believe ‘A thinks but B does not’ whilst B believes ‘B thinks but A does not’. Instead of arguing continually over this point it is usual to have the polite convention that everyone thinks. -- I.—COMPUTING MACHINERY AND INTELLIGENCE A. M. TURING Mind, Volume LIX, Issue 236, October 1950, Pages 433–460, https://doi.org/10.1093/mind/LIX.236.433 https://doi.org/10.1093/mind/LIX.236.433
- verisimi 3y ago
  Its easily conceivable now that one can create a machine that impersonates a human and emotions, and then that machine would be coded to write a sonnet or concerto. But, although it will give the impression (illusion) of feeling, it will still be running its code. Humans will have coded a fantastic illusionist, with no more emotional ability than any other tool, eg a hammer. > According to the most extreme form of this view the only way by which one could be sure that a machine thinks is to be the machine and to feel oneself thinking. This is true - one can only verify subjectivity in oneself. One might assume another living being does feel - but, we could be wrong - eg psychopathy can mean that others are unaware of a psychopath's "feelings" or lack of. It does seem to me, that if we were able to code an ai that gives the impression of feelings, composes concertos apparently independently, etc, that ai would have to be a sort of psychopath. Without getting too lost in spiritual BS, I think we are more than mere matter - the value in our experiences come from our emotions and feelings - not mere inputs that we label as 'emotions' and 'feelings' as would be the case for an ai. But emotions etc are subjective states, not objectively verifiable (even if we can see something correlating on a chart). Put simply, ai can never have a 'soul' even if it presents a wonderful appearance of kindliness, emotionality or whatever. Perhaps there will be no test in the world that will be able to find this out. But it still won't have feelings, it can only ever be a automaton.

EMM_386 3y ago

This particular study isn't limited to LLMs. It is about how to test consciousness in a wide variety of things. We have no tests. People may think their dog is conscious, but we can't test that. We can't even test if other people are conscious. What about invertebrates? Bacteria? Rocks, atoms, AI, your laptop etc? While somebody might say "yes, no, definetly not, yes, maybe" ... still, no tests. And consciousness in this particular proposal deals with whether or not it "‘feels like’ something to be the target system".

lukeschlather 3y ago

> AI already meets the criteria for sentience the way we defined it from 1950-2020. If that is the case why hasn't Kurzweil already won the bet? https://longbets.org/1/ https://longbets.org/1/

jprival 3y ago

>Five years ago people were in agreement that the turing test was a really good test The limitations of the Turing test have been discussed for much longer than that. Ironically its dependence on human judgement is arguably a weak point.

ninetyninenine 3y ago
Human judgement is the strongest judgment in existence. What other thing in reality has better judgment? Nothing.

ForHackernews 3y ago

> LLMs already meet the criteria for what most people technically define as consciousness. I strongly dispute this. Try "talking" to an LLM over time. It has no memory, no agency, no emotions, no desires, no pain, no sense of self. I think cats, dogs and certainly primates are far more conscious than these stochastic parrots. It do agree it's frightening how many people are fooled by a fancy autocomplete.

naasking 3y ago
> It has no memory, no agency, no emotions, no desires, no pain, no sense of self. It does have permanent memory, that's what training creates. It doesn't have agency. Whether it has desires, pain or sense of self is completely unknown, unless of course you have some mechanistic model of these phenomena such that we can see its presence in the human brain and see that LLMs do not have a comparable process. Of course you don't have such a model because it doesn't exist at this time. The problem is not that people have been fooled by stochastic parrots, it's that you've fooled yourself into thinking we understand more about the human mind than we actually do, and also possibly that humans are far more complex and special than might actually be the case, eg. that we are not basically stochastic parrots ourselves.
- ForHackernews 3y ago
  Speaking only for myself, I am not a stochastic parrot. I have a rich and complex inner life. But maybe you are a p-zombie and you are right that it's tricky for you to prove to me that you're not. ;) I don't necessarily think humans are so special (cf. my mention of other mammals) but biology is extraordinarily messy and complex. Computer people tend to handwave the details with crude, reductive analogies (DNA is like source code! Neurons are like circuits!) but the reality is so much more overwhelming. Every single thing affects every other thing. Everything is probabilistic. There are multiple interacting chemical and electrical communication pathways. Neuroanatomy varies from individual to individual and in the same individual over time. The brain can radically adapt in response to trauma in a way that no computer model does. Even at the level of individual neurons[0] (of which there are 2.25 billion in a dog's brain) there's an enormous amount of "processing" happening. This is not like a computer processing - but physical, biological processes occurring that integrate informational content. I get it, everyone is very impressed with LLMs and I'm open to the possibility that some future model might achieve consciousness (to say nothing of sentience) but computer people in particular are way, way too credulous of this stuff and ignorant of what the messy, malfunctioning wetware in their head is actually doing. [0] e.g. https://www.nature.com/articles/nrn2331 https://www.nature.com/articles/nrn2331

imbnwa 3y ago

I think a lot of Computer Science people need to go read Kant

whitepaint 3y ago
What should they begin with?
- deleted 3y ago
  [deleted]
- h0p3 3y ago
  To my eyes, including for this topic, the most important work: //Groundwork of the Metaphysics of Morals//.
- imbnwa 3y ago
  Reverse order of publication back to the first critique

deadbabe 3y ago

An LLM, in its perfect form is basically a p-zombie. There is no consciousness there. In its current form, its not much more than a really big and fast spreadsheet :)

FrustratedMonky 3y ago
Guess the reason this keeps coming up, is because 'how can you be sure?'. I'd say a lot of humans qualify as p-zombies. The problem is, can you ever test for it? To be sure. I'm not sure, that by the time you have a duplicate human, that it can be achievable without some inner subjective experience. Put an LLM on a loop and continually learning in a 3d world. Then ask what is going on inside.
- deadbabe 3y ago
  If you truly believe people are p-zombies you may be a psychopath.
  - FrustratedMonky 3y ago
    Just some. What if the effort discussed in the article does find some way to test for this. But then some humans don't pass?
    - deadbabe 3y ago
      The those humans must have some kind of mental disability or the test is flawed, because every human is a conscious being.
      - amanaplanacanal 3y ago
        How do you know that?
      - yatopifo 3y ago
        I would argue I’m possibly the only conscious being by virtue of having the subjective experience of being conscious. While i can’t rule out that some other beings also possess this subjective experience, it would be a stretch for me to grant it to everyone a priori. That said, i suspect my own subjective experience of consciousness may be an illusion. After all, my mind certainly cannot exist outside of my head, and i know that someone or something skillfully tampering with my brain can alter my perception including that subjective experience. On what basis do you claim that every human (with reasonable exceptions) is a conscious being?

mirekrusin 3y ago

Well, you can use some tricks - ie. hook it up to your brain, let it learn for a while and tell us how it feels when we flip the switch to OFF.

thfuran 3y ago

>The second reason is that LLMs already meet the criteria for what most people technically define as consciousness. An LLM is an inert pile of data. You could perhaps argue that the process of inference exhibits consciousness, but the LLM itself certainly doesn't. There's simply no there there.

BriggyDwiggs42 3y ago
This is a bad argument. A brain frozen in ice also doesn’t demonstrate consciousness, and we don’t interpret that as a counterclaim to “the human brain exhibits consciousness.” Saying an LLM can be conscious implicitly means an LLM while it does things.
- thfuran 3y ago
  >Saying an LLM can be conscious implicitly means an LLM while it does things. But that's not like saying a human brain is conscious while it does things. A brain is continuously doing a thing, whereas an LLM is either inert at rest or briefy doing a thing before returning to exactly the state it was in beforehand. There is no underlying process that could even conceivably support ongoing consciousness. If there is consciousness, it is present only for the span of its interrogation during an inference, after which it is immediately destroyed.
  - BriggyDwiggs42 3y ago
    Yes it would have to be transient, though thats not to say a continuous experience couldn’t be created by connecting together the transient conscious experiences in sequence because of the persistent state of the context window. My only point though was that nobody who says an LLM is conscious thinks the conscious part is the weights sitting on a hard drive.
    - thfuran 3y ago
      You are quite mistaken about the level of technical understanding required to make the claim that eg chatgpt is conscious. Many people have no conception of the distinction between the service and the model but still come to the conclusion through interacting with it that "chatgpt" is conscious.

jajag 3y ago

I think you're confusing intelligence with consciousness

FrustratedMonky 3y ago
"confusing intelligence with consciousness" Now that LLM's are becoming indistinguishable from humans, and other NN's are even solving proofs in geometry competitions. And there are NN that can fly an F-16 better than humans. All of sudden the goal post moved again. "No, that is just 'intelligence' not 'consciousness'." 12 months ago, nobody was making this distinction. We keep needing to redefine the problem to keep humans on top.
- hax0ron3 3y ago
  The distinction between intelligence and consciousness has been discussed in philosophy of mind for many decades, perhaps centuries. I don't know much much it has been discussed in the context of LLMs, but I'm sure it's not an entirely new phenomenon. The debate is not just a matter of people trying to argue over how capable LLMs are. Many people pursue this topic because it is inherently interesting.
  - FrustratedMonky 3y ago
    I think even in philosophy of mind, Intelligence and Consciousness are seen to progress together. They follow the same trend, and at some magic point of enough intelligence, then we call it 'consciousness'. They follow same upward trend. This concept of having high intelligence that is not-conscious, or low-intelligence that is conscious. Seems relatively new.
    - thfuran 3y ago
      Go read Blindsight.
      - FrustratedMonky 3y ago

hax0ron3 3y ago

The authors of the paper seem to define what they mean by consciousness quite precisely. They mean phenomenal consciousness: >The central goal of any C-test is to determine whether a target system has subjective and qualitative experience – often called ‘phenomenal consciousness’. Phenomenal consciousness is, as far as I can tell, binary. You either have subjective experience or you don't. Even if you are subjectively aware of almost nothing, you still have subjective experience. For example, when you wake up in the morning your awareness might be quite limited for the first few seconds - let's say you have your eyes closed and the only thing you are aware of is the warmness of the blanket. But still, you are aware. You have subjective experience. Whereas in deep sleep you do not. Phenomenal consciousness is thus pretty well-defined. It's just that we have zero understanding of how physical systems could possibly give rise to it, and it might be in principle impossible to ever figure that out. It is possible that physical systems do not in fact give rise to consciousness, although obviously the presence vs absence of consciousness in humans is at least to some extent correlated with certain measurable physical states. I doubt that any actual test for consciousness is in principle possible, but the authors of this paper do at least clearly define what they mean when they refer to consciousness. They mean the presence of subjective experience, not intelligence or language or logical thought or having a model of oneself or any other things.

naasking 3y ago

> The first reason is that consciousness is just a loaded concept. It's some arbitrary category with fuzzy boundaries It is right now. We might one day devise a mechanistic explanation for consciousness though, in which case any system that follows that mechanistic process would be conscious.