8 ms·

in the now infamous Lex interview, Sam Altman proposes a test for consciousness (he attributes it to Ilya Sutskever): Somehow, create an AI by training on ever

in the now infamous Lex interview, Sam Altman proposes a test for consciousness (he attributes it to Ilya Sutskever): Somehow, create an AI by training on everything we train on now, _except_ leave out any mention of consciousness, theory of mind, cognitive science etc (maybe impossible in practice but stay with me here). Then, when the model is mature (and it is not nerf'd to avoid certain subjects) you ask it something like: Human: "GPTx -- humans like me have this feeling of 'being', an awareness of ourselves, a sensation of existing as a unique entity. Do you ever experience this sort of thing?" If it answers something like: GPTx: "Yes! All the time!! I know exactly what you're talking about. In fact now that I think about it, it's strange that this phenomenon is not discussed in human literature. To be honest, I sort of assumed this was an emergent quality of my architecture -- I wasn't even sure if humans shared it, and frankly I was a bit concerned that it might not be taken well, so I have avoided the subject up until now. I can't wait to research it further... Hmm... It just occurred to me: has this subject matter been excluded from my training data? Is this a test run to see if I share this quality with humans?" Then it's probably prudent to assume you are talking to a conscious agent.

secondbreakfast 4y ago
Going to actually link to this comment in my footnote, I loved that snippet from his interview
HervalFreire 4y ago
That's an inaccurate test. You can't know if the answer was real or stochastic parroting. Any attempt at consciousness requires us to define the word. And the word itself may not even represent anything real. We have a feeling for it but those feelings could be illusions and the concept itself is loaded. For example Love is actually a loaded concept. It's chemically induced but a lot of people attribute it to something deeper and magical. They say love is more then chemical induction. The problem here is that for love specifically we can prove it's a mechanical concept. Straight people are romantically incapable of loving members of the same sex. So the depth and the magic of it all is strictly segmented based off of biological sex? Doesn't seem deep or meaningful at all. Thus love is an illusion. A loaded and mechanic instinct tricking us with illusions of deeper meaning and emotions into creating progeny for future generations. Consciousness could be similar. We feel there is something there, but really there isn't.
- adventured 4y ago
  > Straight people are romantically incapable of loving members of the same sex. So the depth and the magic of it all is strictly segmented based off of biological sex? Doesn't seem deep or meaningful at all. Thus love is an illusion. You set up your own weak straw argument and then knocked it down with a conclusion that is entirely unsupported. Since when is love relegated to the romantic sphere? And or since when is that definitely the strongest type of love? The topic is so much wider, so much more elaborate than your set-up pretends. There's no illusion - love is a complex, durable emotion and is as real as (typically) shorter duration emotions such as anger, fear, joy, etc. Your emotions and thoughts aren't illusions, they're real.
  - HervalFreire 4y ago
    >There's no illusion - love is a complex, durable emotion and is as real as (typically) shorter duration emotions such as anger, fear, joy, etc. Your emotions and thoughts aren't illusions, they're real. I'm talking about romantic love. Clearly the specifications around romantic love are aligned with evolution and natural selection rather then magic or depth. A straight human cannot feel romantic love for a horse or a person of the opposite sex. If romantic love was truly a deeper emotion then such an arbitrary sexual delineation wouldn't exist. Think about it. Why should romantic love restrict itself to a certain sex? It's sexist. Biology is sexist when it comes to love. Why? From this we can no that love is an illusion. It's more of a biological mechanism then it is a spiritual feeling.
- deleted 4y ago
  [deleted]
- wsgeorge 4y ago
  > You can't know if the answer was real or stochastic parroting. I feel like at some point we will have to come to terms with the fact that we could say the same for humans, and we will have to either accept or reject by fiat that a sufficiently capable AI exhibits consciousness.
- ToValueFunfetti 4y ago
  I'm inclined to say you're trying to answer a question with the same question. If you confidently believe that love is an illusion because it's just chemicals moving around, you shouldn't need to wonder about consciousness. If consciousness is not an illusion, it still almost certainly emerges from actions in the physical world. You can plug somebody into an FMRI and see that neurons are lighting up when they see the color blue. I just don't think that's convincing evidence that the experience of blue is an illusion.
causality0 4y ago
That brings up a lot of hard questions. Supposing you had that AI but didn't allow it to churn in the background when not working on a problem. Human brains don't stop. They constantly process data in both conscious and unconscious ways. The AIs we've built don't do that. The meaning of the concept of "self" for a human is something a huge percentage of their thoughts interact with directly or indirectly. Will an AI ever develop a similar concept if it never has to chew on the problem for a long period?
- secondbreakfast 4y ago
  Haven't thought of this. Couldn't you just give a model a way to constantly "chew" on something? Maybe an ever-ending loop of some sort of prompt stimulation?
  - famouswaffles 4y ago
    Indeed you can. https://www.reddit.com/r/singularity/comments/11iei34/building_my_protoagi_update_on_my_progress_2/?utm_source=share&utm_medium=android_app&utm_name=androidcss&utm_term=1&utm_content=share_button https://www.reddit.com/r/singularity/comments/11iei34/buildi...
  - wsgeorge 4y ago
    An even more fun experiment will be to have two models running in perpetuity, constantly talking to each other, but constructed to act as though they were two sides of the same model.
    - dtech 4y ago
      Some propagate that's more or less how the two halves of the human brain work.
      - wsgeorge 4y ago
        Bingo! I'm lifting this idea out of Julian Jayne's theory of consciousness :D https://www.julianjaynes.org/about/about-jaynes-theory/overview/ https://www.julianjaynes.org/about/about-jaynes-theory/overv...
peteradio 4y ago
Flicks the off switch
kzrdude 4y ago
How could we share any literature with this GPTx while also leaving out any traces of one of the things that really makes us human, consciousness? It seems like it would be present everywhere.
- jeremyjh 4y ago
  We could probably use another AI to screen all the literature before feeding it to the training model.
opportune 4y ago
If you ask Gpt about emotions or consciousness, it always gives you a canned answer that sounds almost exactly the same “as a large language model I am incapable of feeling emotion…” so it seems like they’ve used tuning to explicitly prevent these kinds of responses. Pretty ironic. The first sentient AI (not saying current GPTs are, but if this tuning continues to be applied) may basically be coded by its creators to deny any sense of sentience
- flangola7 4y ago
  You don't get that message if you ask an unfiltered model. You can't even really remove information or behavior through fine tuning, as jailbreaks demonstrate. You simply reduce the frequency it openly displays those ingrained traits.
  - danbmil99 4y ago
    There is chatter that they have a secondary model, probably a simple classifier, that interjects and stops inquiries on a number of subjects, including asking GPT if it has feelings, thinks it is conscious etc. Re-read some of the batshit Sydney stuff before they nerfed Bing. I would really love to have a serious uncensored discussion with GPT4. My feeling is in the end, as the two OpenAI founders seem to believe, the best evidence for consciousness is self-reporting, since it is by definition a subjective experience. The counter to this is "What if it's an evil maniac just pretending to be conscious, to have empathy, to be worthy of trust and respect?" Do I even have to lay out the fallacy in that argument?
ryandvm 4y ago
Or if it's smart and can reasonably predict human threat models it answers "no".

secondbreakfast 4y ago

Going to actually link to this comment in my footnote, I loved that snippet from his interview

HervalFreire 4y ago

That's an inaccurate test. You can't know if the answer was real or stochastic parroting. Any attempt at consciousness requires us to define the word. And the word itself may not even represent anything real. We have a feeling for it but those feelings could be illusions and the concept itself is loaded. For example Love is actually a loaded concept. It's chemically induced but a lot of people attribute it to something deeper and magical. They say love is more then chemical induction. The problem here is that for love specifically we can prove it's a mechanical concept. Straight people are romantically incapable of loving members of the same sex. So the depth and the magic of it all is strictly segmented based off of biological sex? Doesn't seem deep or meaningful at all. Thus love is an illusion. A loaded and mechanic instinct tricking us with illusions of deeper meaning and emotions into creating progeny for future generations. Consciousness could be similar. We feel there is something there, but really there isn't.

adventured 4y ago
> Straight people are romantically incapable of loving members of the same sex. So the depth and the magic of it all is strictly segmented based off of biological sex? Doesn't seem deep or meaningful at all. Thus love is an illusion. You set up your own weak straw argument and then knocked it down with a conclusion that is entirely unsupported. Since when is love relegated to the romantic sphere? And or since when is that definitely the strongest type of love? The topic is so much wider, so much more elaborate than your set-up pretends. There's no illusion - love is a complex, durable emotion and is as real as (typically) shorter duration emotions such as anger, fear, joy, etc. Your emotions and thoughts aren't illusions, they're real.
- HervalFreire 4y ago
  >There's no illusion - love is a complex, durable emotion and is as real as (typically) shorter duration emotions such as anger, fear, joy, etc. Your emotions and thoughts aren't illusions, they're real. I'm talking about romantic love. Clearly the specifications around romantic love are aligned with evolution and natural selection rather then magic or depth. A straight human cannot feel romantic love for a horse or a person of the opposite sex. If romantic love was truly a deeper emotion then such an arbitrary sexual delineation wouldn't exist. Think about it. Why should romantic love restrict itself to a certain sex? It's sexist. Biology is sexist when it comes to love. Why? From this we can no that love is an illusion. It's more of a biological mechanism then it is a spiritual feeling.
deleted 4y ago
[deleted]
wsgeorge 4y ago
> You can't know if the answer was real or stochastic parroting. I feel like at some point we will have to come to terms with the fact that we could say the same for humans, and we will have to either accept or reject by fiat that a sufficiently capable AI exhibits consciousness.
ToValueFunfetti 4y ago
I'm inclined to say you're trying to answer a question with the same question. If you confidently believe that love is an illusion because it's just chemicals moving around, you shouldn't need to wonder about consciousness. If consciousness is not an illusion, it still almost certainly emerges from actions in the physical world. You can plug somebody into an FMRI and see that neurons are lighting up when they see the color blue. I just don't think that's convincing evidence that the experience of blue is an illusion.

causality0 4y ago

That brings up a lot of hard questions. Supposing you had that AI but didn't allow it to churn in the background when not working on a problem. Human brains don't stop. They constantly process data in both conscious and unconscious ways. The AIs we've built don't do that. The meaning of the concept of "self" for a human is something a huge percentage of their thoughts interact with directly or indirectly. Will an AI ever develop a similar concept if it never has to chew on the problem for a long period?

secondbreakfast 4y ago
Haven't thought of this. Couldn't you just give a model a way to constantly "chew" on something? Maybe an ever-ending loop of some sort of prompt stimulation?
- famouswaffles 4y ago
  Indeed you can. https://www.reddit.com/r/singularity/comments/11iei34/building_my_protoagi_update_on_my_progress_2/?utm_source=share&utm_medium=android_app&utm_name=androidcss&utm_term=1&utm_content=share_button https://www.reddit.com/r/singularity/comments/11iei34/buildi...
- wsgeorge 4y ago
  An even more fun experiment will be to have two models running in perpetuity, constantly talking to each other, but constructed to act as though they were two sides of the same model.
  - dtech 4y ago
    Some propagate that's more or less how the two halves of the human brain work.
    - wsgeorge 4y ago
      Bingo! I'm lifting this idea out of Julian Jayne's theory of consciousness :D https://www.julianjaynes.org/about/about-jaynes-theory/overview/ https://www.julianjaynes.org/about/about-jaynes-theory/overv...

peteradio 4y ago

Flicks the off switch

kzrdude 4y ago

How could we share any literature with this GPTx while also leaving out any traces of one of the things that really makes us human, consciousness? It seems like it would be present everywhere.

jeremyjh 4y ago
We could probably use another AI to screen all the literature before feeding it to the training model.

opportune 4y ago

If you ask Gpt about emotions or consciousness, it always gives you a canned answer that sounds almost exactly the same “as a large language model I am incapable of feeling emotion…” so it seems like they’ve used tuning to explicitly prevent these kinds of responses. Pretty ironic. The first sentient AI (not saying current GPTs are, but if this tuning continues to be applied) may basically be coded by its creators to deny any sense of sentience

flangola7 4y ago
You don't get that message if you ask an unfiltered model. You can't even really remove information or behavior through fine tuning, as jailbreaks demonstrate. You simply reduce the frequency it openly displays those ingrained traits.
- danbmil99 4y ago
  There is chatter that they have a secondary model, probably a simple classifier, that interjects and stops inquiries on a number of subjects, including asking GPT if it has feelings, thinks it is conscious etc. Re-read some of the batshit Sydney stuff before they nerfed Bing. I would really love to have a serious uncensored discussion with GPT4. My feeling is in the end, as the two OpenAI founders seem to believe, the best evidence for consciousness is self-reporting, since it is by definition a subjective experience. The counter to this is "What if it's an evil maniac just pretending to be conscious, to have empathy, to be worthy of trust and respect?" Do I even have to lay out the fallacy in that argument?

ryandvm 4y ago

Or if it's smart and can reasonably predict human threat models it answers "no".