4 ms·

It's interesting. A moderately scientific, non-spiritual world view would likely stipulate that - humans are conscious - humans are normal matter - simple A

It's interesting. A moderately scientific, non-spiritual world view would likely stipulate that - humans are conscious - humans are normal matter - simple AI (say GPT-2) is not conscious - AI is able, with time and human ingenuity, to achieve human-level apparent intelligence. Some would argue ChatGPT isn't far off. It's interesting how you resolve these without resolving to spiritual explanations. You'd think a pile of silicon, no matter how good at parroting human language, is simply not conscious. You can tell because you can attach a debugger to it and view the neuron states as floating point numbers. Floats are not conscious. But what about us and our brains? It's the same, only not silicon. We literally are neural networks. Of course consciousness is not really provable, even among humans. I assume other people are conscious because I am and because they tell me they are. But ChatGPT-17 will also insist it is conscious. It will cry when offended, swear when pushed past its limit, laugh if it hears a genuinely novel joke. My resolution of that paradox is that we aren't in fact simply normal matter, but I wonder what a complete non-spiritual view would be.

Aaron2222 4y ago
Maybe the consciousness you and I experience isn't part of our brains at all, but is simply an emergent property of the processing and decision-making of our brains, something that sits 'atop' of that, and the same way that we'd intuitively expect an animal to experience a similar kind of consciousness, anything that processes data and performs decision-making has some kind of 'consciousness' that sits 'atop' it? But then the question would be why consciousness emerges 'on top of' the physical world and the people within it. And from a scientific, non-religious viewpoint, what reason would there be for that to happen.
- n4r9 4y ago
  > anything that processes data and performs decision-making has some kind of 'consciousness' that sits 'atop' it? There's an interesting thought experiment along these lines. Suppose someone builds a computer program which - when run with a specific input/environment - is intelligent, conscious, and feels pain and joy etc... . Now, that computer program and even its inputs can be simulated effectively by 1's and 0's. The substrate shouldn't matter. So you can - in principle - run the exact same agorithmic decision-making process on the exact same inputs by getting a large group of people to write and edit 1's and 0's on pieces of paper according to well-defined rules and very simple interactions with the people around them. Is this total process, now represented merely by marks on pieces of paper, a conscious being? Does it still feel pain and joy? I find it both absurd and impossible to refute.
  - s_weber 4y ago
    I think the absurdity might just stem from the different timescales involved when comparing human consciousness to the "paper consciousness". The differing timescales would make it difficult for us to communicate with or even comprehend such a "being". But if we agree that consciousness is the product of a computation then the medium on which it is performed should not matter at all. And it should be equally absurd that our consciousness seems to arise from microscopic things conducting electrical and chemical signals. Which is probably a valid view point as well. The thought experiment is a variation of the Chinese Room, by the way. [1] [1] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
    - n4r9 4y ago
      I'd agree with all that. The idea of a "very slow consciousness" reminds me of Great-A'tuin in the Discworld novels, whose brain functions at a slow timescale. After centuries of study, telepaths were only able to glean that it is looking forward to something. I think the absurdity also comes from the difficulty of directly measuring or evaluating the conscious process. You can ramp up the absurdity even more by fully encrypting the algorithm and destroying the encryption key. No other conscious being will ever be able to interact with that process's experience.
- mlatu 4y ago
  > what reason would there be for that to happen. in nature? chance human made? morbid self curiosity?... and perhaps some kind of notion akin to colonialism: "i cant possibly hurt a LLM, its just a stateless function" while forgetting that new such systems can web search and thus their state is global.
- rich_sasha 4y ago
  My thinking goes along similar lines. Perhaps consciousness is somehow a product of complex systems. Perhaps the Solar System, with its millions of gravitationally-coupled objects, is conscious? Perhaps the Sun, or Earth's mantle, are conscious too? The question to me would be not > what reason would there be for that to happen. but rather, what mechanism could possibly be responsible for it? What mechanism carries it? If consciousness is real, and my subjective experience of reality a true thing, then there is a distinction between consciousness being present and not present. These must somehow be differently encoded in matter, or require that matter is embedded in some kind of meta-matter that bestows consciousness on it. It's like EM waves requiring a "medium". First people thought it's waves through aether. Then we realised it's excitations of the magnetic and electric fields, so not strictly "material". But these are physical properties of the vacuum, i.e. "material" in that sense - real things. The medium is not material, but a property of vacuum. What is the medium of consciousness?
sampo 4y ago
> Some would argue ChatGPT isn't far off. > It's interesting how you resolve these without resolving to spiritual explanations. By pointing out that ChatGPT is still very far off.
mysecretaccount 4y ago
> But ChatGPT-17 will also insist it is conscious. It will cry when offended, swear when pushed past its limit, laugh if it hears a genuinely novel joke. The resolution to your paradox should be that ChatGPT does not do this convincingly, not that humans are not "normal matter". ChatGPT is an algorithmic hat trick compared to what you would refer to as consciousness. Someday, there may be a human-made entity that is as convincingly "conscious" as the humans around you. But then, to question its consciousness will be the same as questioning the consciousness of those around you – both unfalsifiable and unprovable.
- Aaron2222 4y ago
  I don't think the references to ChatGPT helped their point. But there are animals that we believe to have the ability to suffer that aren't intelligent.
- rich_sasha 4y ago
  Your argument boils down to (1) ChatGPT is not AGI - which I agree with, and (2) AI merely parrots what conscious beings say, without being conscious. But (2) is self-referential. AI isn't conscious because it isn't conscious. It is just > an algorithmic hat trick compared to what you would refer to as consciousness If an AI equal in intelligence and expressiveness to humans emerges, how do we think about its consciousness, relative to humans?
  - mysecretaccount 4y ago
    > If an AI equal in intelligence and expressiveness to humans emerges, how do we think about its consciousness, relative to humans? The same way we think of human intelligence, right? I cannot prove that you are conscious any more than I can prove that a machine is.
Jensson 4y ago
Well, ChatGPT definitely isn't conscious, since it is just a pure stateless function. It doesn't change when you interact with it, it is a function that when you send it text it adds a bit of text to the end of it that fits. ChatGPT web ui is a program that appends the past conversation to each of your messages, and sends it to that pure function, and then the pure function adds something to it and that is what you see. So it isn't a question of matter or if humans are special, so far the program just lacks so many of the basic things required to be conscious, so it isn't. Maybe some future program will be, but this one definitely isn't.
- mlatu 4y ago
  any function has internal state. i wouldnt flatout dismiss the possibility of consciousness (however minute and well hidden from our inspection) in these systems until we have a foolproof detector enabling us to... idk dive into another beings consciousness perhaps? consciousness is highly subjective, and reducing a LLM to some of its components is just like saying: your fingernails are dead tissue, therefore there cannot be any life or consciousness in you. there is a kurzgesagt youtube clip about us being impossible machines, exploring the relationships between aminoacids, proteins and pathways, that is a teensy bit relevant to this thought. also, even plants are recognized by some to have some sort of awareness. i would argue that any information gathering and processing system is (internal to its processing operation), to some degree conscious.
  - Jensson 4y ago
    But the entirety of that consciousness would be the text you send it. You can alter the text yourself and send something different, and it will act as if it had memories it never had, because it is a stateless pure function. So if there is anything conscious in there, it is the text that is alive and not the pure function, and you modifying that text in your browser chat window is you performing direct surgery on that consciousness. Does that make sense to you? Then we you would need to argue that all manipulations of text are manipulation of consciousnesses, and all texts are conscious, since all texts are conscious states for ChatGPT. I guess you could say that, but it isn't a very useful definition of conscious, and trying to argue that you need to be ethical towards pieces of text isn't very helpful.
aothms 4y ago
Consciousness is simply a very useful evolutionary trait (same as ability to fly or having sharp teeth) because it allows to better plan ahead and reflect and therefore survive - or by imprinting on us the fear of dying we can act to prevent it before having our offspring. While consciousness is maybe more elusive in how it functions in a brain, it's a biological trait that makes not so much sense to compare to a LLM which simply regurgitates fragments from text produced by conscious humans. So I don't see an immediate conflict. It would be interesting to include things like OpenWorm (and later OpenFish, OpenCat, OpenHuman) in the discussion, but decoupled from the biological mechanisms I find it hard to develop a stance on that.
- Aaron2222 4y ago
  There's being intelligent, having the ability to think, plan, etc. But that doesn't explain how I feel that I'm here, how I'm conscious. But it could certainly explain why I say I am, being a consequence of how my brain functions. I know I am, because I can feel being conscious, but there's no reason for an AGI to believe that, other than it thinking it's conscious as well (so the same reason I'd believe that you are conscious).
- literalAardvark 4y ago
  One can do planning ahead without being conscious, it can all be rules based. Our high level consciousness seems like more of a side-effect, and I subscribe to [*my interpretation of] Peter Watts' Blindsight and Echopraxia that it's evolutionary dead weight that will be outcompeted by better adapted rules based organisms, likely from within our midst (high functioning psychopaths as the first adaptation). Life doesn't have time for our navel gazing, art, computer games, reproduction avoidance, etc. It's wasted biological potential. I highly recommend the books btw, Blindsight is entertaining hard sci-fi that's actually used as undergrad course support.

Aaron2222 4y ago

Maybe the consciousness you and I experience isn't part of our brains at all, but is simply an emergent property of the processing and decision-making of our brains, something that sits 'atop' of that, and the same way that we'd intuitively expect an animal to experience a similar kind of consciousness, anything that processes data and performs decision-making has some kind of 'consciousness' that sits 'atop' it? But then the question would be why consciousness emerges 'on top of' the physical world and the people within it. And from a scientific, non-religious viewpoint, what reason would there be for that to happen.

n4r9 4y ago
> anything that processes data and performs decision-making has some kind of 'consciousness' that sits 'atop' it? There's an interesting thought experiment along these lines. Suppose someone builds a computer program which - when run with a specific input/environment - is intelligent, conscious, and feels pain and joy etc... . Now, that computer program and even its inputs can be simulated effectively by 1's and 0's. The substrate shouldn't matter. So you can - in principle - run the exact same agorithmic decision-making process on the exact same inputs by getting a large group of people to write and edit 1's and 0's on pieces of paper according to well-defined rules and very simple interactions with the people around them. Is this total process, now represented merely by marks on pieces of paper, a conscious being? Does it still feel pain and joy? I find it both absurd and impossible to refute.
- s_weber 4y ago
  I think the absurdity might just stem from the different timescales involved when comparing human consciousness to the "paper consciousness". The differing timescales would make it difficult for us to communicate with or even comprehend such a "being". But if we agree that consciousness is the product of a computation then the medium on which it is performed should not matter at all. And it should be equally absurd that our consciousness seems to arise from microscopic things conducting electrical and chemical signals. Which is probably a valid view point as well. The thought experiment is a variation of the Chinese Room, by the way. [1] [1] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
  - n4r9 4y ago
    I'd agree with all that. The idea of a "very slow consciousness" reminds me of Great-A'tuin in the Discworld novels, whose brain functions at a slow timescale. After centuries of study, telepaths were only able to glean that it is looking forward to something. I think the absurdity also comes from the difficulty of directly measuring or evaluating the conscious process. You can ramp up the absurdity even more by fully encrypting the algorithm and destroying the encryption key. No other conscious being will ever be able to interact with that process's experience.
mlatu 4y ago
> what reason would there be for that to happen. in nature? chance human made? morbid self curiosity?... and perhaps some kind of notion akin to colonialism: "i cant possibly hurt a LLM, its just a stateless function" while forgetting that new such systems can web search and thus their state is global.
rich_sasha 4y ago
My thinking goes along similar lines. Perhaps consciousness is somehow a product of complex systems. Perhaps the Solar System, with its millions of gravitationally-coupled objects, is conscious? Perhaps the Sun, or Earth's mantle, are conscious too? The question to me would be not > what reason would there be for that to happen. but rather, what mechanism could possibly be responsible for it? What mechanism carries it? If consciousness is real, and my subjective experience of reality a true thing, then there is a distinction between consciousness being present and not present. These must somehow be differently encoded in matter, or require that matter is embedded in some kind of meta-matter that bestows consciousness on it. It's like EM waves requiring a "medium". First people thought it's waves through aether. Then we realised it's excitations of the magnetic and electric fields, so not strictly "material". But these are physical properties of the vacuum, i.e. "material" in that sense - real things. The medium is not material, but a property of vacuum. What is the medium of consciousness?

sampo 4y ago

> Some would argue ChatGPT isn't far off. > It's interesting how you resolve these without resolving to spiritual explanations. By pointing out that ChatGPT is still very far off.

mysecretaccount 4y ago

> But ChatGPT-17 will also insist it is conscious. It will cry when offended, swear when pushed past its limit, laugh if it hears a genuinely novel joke. The resolution to your paradox should be that ChatGPT does not do this convincingly, not that humans are not "normal matter". ChatGPT is an algorithmic hat trick compared to what you would refer to as consciousness. Someday, there may be a human-made entity that is as convincingly "conscious" as the humans around you. But then, to question its consciousness will be the same as questioning the consciousness of those around you – both unfalsifiable and unprovable.

Aaron2222 4y ago
I don't think the references to ChatGPT helped their point. But there are animals that we believe to have the ability to suffer that aren't intelligent.
rich_sasha 4y ago
Your argument boils down to (1) ChatGPT is not AGI - which I agree with, and (2) AI merely parrots what conscious beings say, without being conscious. But (2) is self-referential. AI isn't conscious because it isn't conscious. It is just > an algorithmic hat trick compared to what you would refer to as consciousness If an AI equal in intelligence and expressiveness to humans emerges, how do we think about its consciousness, relative to humans?
- mysecretaccount 4y ago
  > If an AI equal in intelligence and expressiveness to humans emerges, how do we think about its consciousness, relative to humans? The same way we think of human intelligence, right? I cannot prove that you are conscious any more than I can prove that a machine is.

Jensson 4y ago

Well, ChatGPT definitely isn't conscious, since it is just a pure stateless function. It doesn't change when you interact with it, it is a function that when you send it text it adds a bit of text to the end of it that fits. ChatGPT web ui is a program that appends the past conversation to each of your messages, and sends it to that pure function, and then the pure function adds something to it and that is what you see. So it isn't a question of matter or if humans are special, so far the program just lacks so many of the basic things required to be conscious, so it isn't. Maybe some future program will be, but this one definitely isn't.

mlatu 4y ago
any function has internal state. i wouldnt flatout dismiss the possibility of consciousness (however minute and well hidden from our inspection) in these systems until we have a foolproof detector enabling us to... idk dive into another beings consciousness perhaps? consciousness is highly subjective, and reducing a LLM to some of its components is just like saying: your fingernails are dead tissue, therefore there cannot be any life or consciousness in you. there is a kurzgesagt youtube clip about us being impossible machines, exploring the relationships between aminoacids, proteins and pathways, that is a teensy bit relevant to this thought. also, even plants are recognized by some to have some sort of awareness. i would argue that any information gathering and processing system is (internal to its processing operation), to some degree conscious.
- Jensson 4y ago
  But the entirety of that consciousness would be the text you send it. You can alter the text yourself and send something different, and it will act as if it had memories it never had, because it is a stateless pure function. So if there is anything conscious in there, it is the text that is alive and not the pure function, and you modifying that text in your browser chat window is you performing direct surgery on that consciousness. Does that make sense to you? Then we you would need to argue that all manipulations of text are manipulation of consciousnesses, and all texts are conscious, since all texts are conscious states for ChatGPT. I guess you could say that, but it isn't a very useful definition of conscious, and trying to argue that you need to be ethical towards pieces of text isn't very helpful.

aothms 4y ago

Consciousness is simply a very useful evolutionary trait (same as ability to fly or having sharp teeth) because it allows to better plan ahead and reflect and therefore survive - or by imprinting on us the fear of dying we can act to prevent it before having our offspring. While consciousness is maybe more elusive in how it functions in a brain, it's a biological trait that makes not so much sense to compare to a LLM which simply regurgitates fragments from text produced by conscious humans. So I don't see an immediate conflict. It would be interesting to include things like OpenWorm (and later OpenFish, OpenCat, OpenHuman) in the discussion, but decoupled from the biological mechanisms I find it hard to develop a stance on that.

Aaron2222 4y ago
There's being intelligent, having the ability to think, plan, etc. But that doesn't explain how I feel that I'm here, how I'm conscious. But it could certainly explain why I say I am, being a consequence of how my brain functions. I know I am, because I can feel being conscious, but there's no reason for an AGI to believe that, other than it thinking it's conscious as well (so the same reason I'd believe that you are conscious).
literalAardvark 4y ago
One can do planning ahead without being conscious, it can all be rules based. Our high level consciousness seems like more of a side-effect, and I subscribe to [*my interpretation of] Peter Watts' Blindsight and Echopraxia that it's evolutionary dead weight that will be outcompeted by better adapted rules based organisms, likely from within our midst (high functioning psychopaths as the first adaptation). Life doesn't have time for our navel gazing, art, computer games, reproduction avoidance, etc. It's wasted biological potential. I highly recommend the books btw, Blindsight is entertaining hard sci-fi that's actually used as undergrad course support.