5 ms·

"These guys have no idea what consciousness is (nobody does)" Where do people get off saying no one has any idea what consciousness is? I agree that there is a

"These guys have no idea what consciousness is (nobody does)" Where do people get off saying no one has any idea what consciousness is? I agree that there is a significant sliver of a philosophical problem which remains stubborn (how precisely does physical activity produce qualia), but neuroscience knows quite a bit about what physical processes underlie our behavior from the behavior of individual neurons to the activity of the entire brain. I object to the wholesale dismissal of neuroscience because thinking about the brain relative to LLMs is genuinely informative about what sorts of things you could expect to be going on in an LLM. And, to my mind, a real appraisal of the differences between brains and LLMs makes the case pretty strongly that LLMs experience nothing and are, furthermore, fairly well characterized as stochastic parrots. "They can't prove I'm not a stochastic parrot anymore than they can prove whatever cutting edge LLM isn't." Prove is a very strong word, but I think its actually quite possible to demonstrate via scientific observation that you differ in many, significant, and relevant to the question of "being a stochastic parrot", ways, from LLMs. It astounds me that people routinely suggest that human brains and LLMs are somehow indistinguishable.

ARandomerDude 3y ago
Likewise, the nobody knows what consciousness is mindset is smuggling in the idea that in order to know something at all, you must know it comprehensively. I know exactly what consciousness is based on my experience with it, even though I could not possibly give a comprehensive account of everything consciousness entails. By analogy, I've been married for just shy of 20 years. I know my wife very well. I certainly do not know everything there is to know about her, but I do know her.
- eimrine 3y ago
  > I know exactly what consciousness is based on my experience with it, even though I could not possibly give a comprehensive account of everything consciousness entails. What is the price of that kind of knowledge? You don't even know where is the border between your knowledge and your absence of knowledge. How much can you tell about consciousness without stepping on "absence of knowledge" field? Pretty nothing, isn't it? > By analogy, I've been married for just shy of 20 years. I know my wife very well. I certainly do not know everything there is to know about her, but I do know her. I have a better example. I speak English for 20 years if to start counting from my first English lesson when I learned my first English word. You can find plenty of silly mistakes in my comments. But at least I know what I can express or understand and what I can not.
- mo_42 3y ago
  > I know exactly what consciousness is based on my experience with it, even though I could not possibly give a comprehensive account of everything consciousness entails. Not sure if personal experiences count. Generally, we laugh at people who talk about esoteric experiences. So a simple explanation could be that consciousness is an illusion? Or put differently, is there any phenomenon that needs the assumption of consciousness? The way I experience myself could be just the history of experiences. So there is something that the brain can refer to.
  - ARandomerDude 3y ago
    > So a simple explanation could be that consciousness is an illusion? Doesn't having an illusion presuppose consciousness?
  - nathan_compton 3y ago
    What perceives the illusion?
    - visarga 3y ago
      So "the illusion" is like a GPT or StableDiffusion model making stuff up based on conditioning. Nothing mysterious, we have AI that can do that. The same simulator predicts not just how the world will evolve, but also actions and their estimated rewards. It's an imagination based planning system. Bringing all perceptions together into the simulation, integrating them into the same reference system, and using them to imagine, plan, act and learn - that could be consciousness.
      - nathan_compton 3y ago
        Yes, but why would such a system experience anything? I'm a died in the wool monist materialist but I can acknowledge that there is something tricky (at the very least) going on here.
        visarga 3y ago
- Symmetry 3y ago
  Scientific study of consciousness comes from ignoring our subjective impressions of our own consciousnesses, which might be illusory, and going only by what can be seen by other people. So you have experiments doing things like showing subjects subliminal images trying to probe the boundaries between conscious experience and unconscious experience. You start with results like "If we show this image to subjects for 50 ms it only has a slight effect on their behavior which fades out after a second, but if we show it to them for 60 ms it has a large effect for the rest of the experiment including being able to talk about it" and then you keep going from there.
  - dinvlad 3y ago
    This is kind of a stretch of an argument though. We could say the same about physics and any other sciences - everything is an abstraction at some level, but if this abstraction is reproduced by multiple independent types of measurement and is falsifiable, that is what we call scientific. I don't think LLMs pass this test.
    - Symmetry 3y ago
      I certainly wouldn't say that a traditional LLM is conscious. Once an input falls off of a LLM's input buffer it ceases to have any effect on its output, the exact same way that a subliminal stimuli's effects are limited in a human brain. The size, in bytes, of a LLM's input buffer isn't all that far off from a human brain's input activations either. So strictly feed-forward neural networks aren't conscious in this sense but its easy to image that architectural changes might provide an analogue to what a humans' consciousness provides.
- JohnFen 3y ago
  > I know exactly what consciousness is based on my experience with it OK, then what is it?
ben_w 3y ago
> Where do people get off saying no one has any idea what consciousness is? I'm not "getting off" saying that, but I do say it often. For me, it's important to know: If we think an artificial neutral network can have consciousness and we're wrong, then there is a risk of all the people who want to have their minds uploaded having a continued existence no better than the one of TV stars reproduced on VHS tape. There is also a risk of this being done as a standard treatment for lesser injuries, especially if it's cheaper. If we think an artificial neutral network can't have consciousness and we're wrong, then there is a risk of creating a new slave class that makes real the fears of the Haitian slaves in the form of the Vodou concept of a zombie — not even death will free them from eternal slavey.
- nathan_compton 3y ago
  Well, that is an entirely different question. My personal view is that nothing prevents an artificial neural network from having a consciousness (I don't think it makes sense to believe there is anything magical about human brains). What I am saying is that we emphatically know things about the physical processes that (almost certainly) generate consciousness and that we should take that knowledge seriously when examining artificial neural networks. People eager to attribute more to these networks than they plausibly constitute love to dismiss all this knowledge so as to muddy the waters of comparison.
  - ben_w 3y ago
    > What I am saying is that we emphatically know things about the physical processes that (almost certainly) generate consciousness I'm prepared to believe that people who aren't me know such things, but last time I asked a PhD in brain research about this (a while ago now), they seemed to disagree. At least, assuming we're talking about the same usage of the word "consciousness" here — when it's defined as "opposite of unconscious" then sure we have drugs to turn that off, and also separately with the non-overlapping definition of "opposite of autonomous or reflexive"… …but the weird thing where I have an experience rather than just producing responses to stimuli? If anyone knows about that, my search engine bubble hides it from me.
vernon99 3y ago
This is a very interesting point that requires some examples and further elaboration to have value for the readers. It refutes but doesn't provide arguments. Can you please elaborate?
avg_dev 3y ago
thank you for this. i am not well up on consciousness (or machine learning) and i have seen chatbots/llms hallucinate and such, and i have also seen them do amazing things. i have wondered to myself a few times lately: how do i know that im dissimilar in nature from these things? so i ask you a followup question: what are some easy to understand ways in which a human's thought process would differ from an llm's behavior? also for anyone else wondering: https://www.merriam-webster.com/dictionary/qualia https://www.merriam-webster.com/dictionary/qualia
- nathan_compton 3y ago
  There are a lot. First of all, these LLMs do not learn in-situ. They are entirely static (apart from the prompt). To teach an LLM something new is an ex-situ process, more or less totally unrelated to the way it predicts. Contrast that with a brain: brains are constantly learning (in fact, it is difficult to imagine how a brain as we understand it could work without constantly learning). In a related way, because we learn on-line and constantly, our brains have to also maintain goals, rewards and punishments, etc etc. We have neurons for all of the trivia of keeping us moving, seeking new input, generalizing it, throwing away bad information, etc. For an LLM all of that is external. The LLM doesn't have any reason to even distinguish between generation and training. All the weight updates are calculated by a (relatively simple) external process. Furthermore, LLMs are entirely _feed forward_. The input comes in, a lot of numbers are crunched, and then output comes out. There is no rumination (again, the analogy for rumination in an LLM is in the training process, which is not embodied in the LLM). Much of the content of our consciousness is perceptions relating to all of these things. I think its possible that artificial neural networks may one day do enough of these things that I would admit they are conscious, but architecturally and fundamentally, I don't see any reason that an LLM would have them. I also don't think even GPT4 is that intelligent (fantastic recall, though). It does an impression of a cognitive process (literally by printing out steps) but that doesn't seem compelling enough for me to imagine a theory of mind underneath. A model of text, sure, but not a mind.
  - avg_dev 3y ago
    Wow, thanks again! I really enjoyed this response and I have learned from it. (Which I guess an LLM could not do while generating something!) Some references I glanced at (I mostly read the top paras): - https://en.wikipedia.org/wiki/In_situ https://en.wikipedia.org/wiki/In_situ - https://en.wikipedia.org/wiki/Theory_of_mind https://en.wikipedia.org/wiki/Theory_of_mind Really enjoyed this response, and feel like I've developed a better understanding of some of the concepts relating to generative ML as it is used in LLMs. An aside: I took a course on ML in a university a few years back, and it was interesting (it was an intro and survey course offered by the CompSci faculty), but difficult for me. I excelled at implementing using Keras/TF code in Python, and I had fun manually implementing some gradient descent algorithm but a lot of the math including all of the multi-var calc, stats, probability was quite difficult for me to wrap my head around, and I really didn't feel like I got a solid grounding on a meta-level of what we were doing or why. I have been reading a bit about LLMs and I think your post has filled in some of the gaps in what at this point I was really looking to understand.
deleted 3y ago
[deleted]
emptysongglass 3y ago
Unfortunately, this is the thread that all these arguments begin: demonstrative ignorance of either the science, the research, or both. They originate in ignorance and because of ignorance, the response is terrible fear and prophecies of the End Times. It's no different than any other brand of fear. There is no evidence that processing power = mind. None. There is no evidence that the human condition is any way related to some kind of terra firma of logic. In fact, there's considerable evidence that feelings are so entangled in the experience of humanness that the idea of divorce or separation is a false one. "Being human" is primarily a feeling experience that drives narratives, motivations: it underlies every single activity we engage in. This is why people like Eliezer Yudkowsky and his ilk are so totally off the mark: it's no coincidence that the Less Wrong community and AI doomsayers can often be found on the same side of the aisle. Both camps believe in and idealize a distinct logic mind that can be attained. Funnily enough, it's still fear, a very human feeling, that is the basis for all these proclamations. My worry is this camp garners enough influence to convince someone an AI doomsday is right around the corner unless immediate action is taken.
- fnordpiglet 3y ago
  Buddhism says that this isn’t at all what being a human is. It says that the feelings and narratives are an aspect of the human mind, but not core to it. My conception is they are like feedback sources/optimizers and constraint engines. The worries, narratives, voices, chattering are self important feeling and trick us into believing they are who we are. However, a key aspect of vipassana and related mindfulness is building the ability to be aware of those processes from a part of us that has no voice, has no experienced feelings, but is able to be aware of them independently and control them. This source is where our agency derives from, and it does not have to be driven by feelings or the narratives, it is in fact able to suspend them and simply exist as it is. This is what is known as nirvana, which has attained weird mystical meanings in our culture, but essentially means attaining and maintaining a mind that has fully subordinated the “self” driving narrative and emotional soup. The loudness of feelings and our chattering monkey mind self support themselves internally as being “who we are,” but, again my own conception, their importance is an illusion they create internally. All this said, they are certainly a part of what makes us human, in so far as a foot makes us human. But you can function and live a full and complete life in the state of nirvana without losing anything. In fact in my 30 year practice at one point I was scared to bring the practice into my daily lived life fearing being uncompelled by these processes and having a clear mind would make a robot or something - but the opposite was true. At some core level I knew my experiences and connections deeper than a feeling, and the people around me felt I was finally with them for the first time. My point here is that the western conception of what it means to be a human is not particularly simple and it’s not the case, assuming thousands of years of Buddhist practice isn’t a crock, that our feelings and thoughts are the core of what it is to be human. Further - if they are illusions and feedback systems, they can be simulated as constraining feedback systems in an artificial mind just as easily. I think the nature of what is human is much deeper in our minds, but because it’s not easy to examine like feelings and thoughts, I think we really do not understand it very well. This leads me to my long labored point - I agree with the original poster that we don’t understand consciousness. I believe we over estimate our understanding of what it means to be human. I do not however think our machines will achieve it either. But I don’t know why we need to make an artificial human. AI means intelligence, not human. A natural human takes 9 months and we have too many of them, let’s try for something different.
SamBam 3y ago
> I agree that there is a significant sliver of a philosophical problem which remains stubborn (how precisely does physical activity produce qualia) But that IS the definition of consciousness! This is like saying "We understand practically everything about airplanes, except how they stay in the air." Nothing discussed in neuroscience is relevant to understanding what consciousness IS (which is the question posed above). Finding out that stimulating such and such a region makes us sad, or that this bundle of nerves activates before we're consciously aware of a decision doesn't tell us anything about consciousness itself. We've known for hundreds of years that there is a relationship between the brain and consciousness, finding out more details doesn't answer the question. (Now, whether consciousness is necessary for AGI is a separate question.)
- nathan_compton 3y ago
  I believe it is eminently credible draw a strong association between consciousness and the physical activity of the brain since it is relatively well backed up by scientific observation that there is a one to one correspondence between conscious experience and brain activity. Although we still don't understand precisely how the physical activity creates qualia, I think its perfectly reasonable to say that studying and understanding brain activity constitutes studying and understanding consciousness.
  - JohnFen 3y ago
    Sure, but none of the means we know what "consciousness" is.
    - c22 3y ago
      I know what consciousness "is." I'm experiencing it right now.
      - Bridgoo 3y ago
        Prove it
panda-giddiness 3y ago
> I agree that there is a significant sliver of a philosophical problem which remains stubborn (how precisely does physical activity produce qualia) But that "sliver" of a problem is known as the hard problem of consciousness for a reason [1], which is exactly the sort of problem neuroscience can only address in a limited capacity. Understanding how nerves propagate a signal to produce a sensory input (an "easy" problem of consciousness) doesn't inform us as to why certain physical mechanisms result in conscious experience (or more fundamentally what it even means to have a conscious experience). To return to the topic at hand, a stochastic parrot generates grammatical, sensible language without understanding its underlying meaning. Of course, you can debate what it means to understand something; but for a person to vocalize an idea they understand, they must first somehow consciously process that idea. This is firmly a hard problem to which neuroscience offers limited guidance. Of course, I'd agree that human beings aren't stochastic parrots -- if human beings were stochastic parrots, then what would it even mean to understand something? But I doubt you could use neuroscience to ascertain whether large language models are or aren't stochastic parrots. Indeed, depending on your definition of "understanding", consciousness might not even be a prerequisite, making the comparison to neuroscience moot. --- [1] https://en.wikipedia.org/wiki/Hard_problem_of_consciousness https://en.wikipedia.org/wiki/Hard_problem_of_consciousness
fnovd 3y ago
Here's a thought experiment: what are the qualities of a parrot that would make its consciousness different than that of a stochastic parrot? For that matter, what are the qualities that separate a parrot's consciousness from a human's? Or a human's from a pig's? Given the way we treat pigs, we clearly don't think consciousness in and of itself is worthy of any formal consideration, as long as the benefit we derive from exploiting it is high enough. So, with AI, is it fair to say that anyone really cares whether it will develop qualities that make it seem as though it is an emergent consciousness? Why would we treat digital consciousness any better than we treat organic consciousness? What is the point of pontificating whether or not the type of thinking an AI does crosses an arbitrary threshold when that threshold only exists as a tool for creating useful outgroups? However sophisticated the thing that our thinking is, it exists on a scale and we sit at an arbitrary spot. We treat thinking that occurs further down the scale as functionally irrelevant not because of any real distinction but because doing so has a high utility for our species. So, the question of how we will treat a "truly conscious and sentient" AI has already been answered. Look at how we treat pigs. Good luck out there, HAL.
- nathan_compton 3y ago
  The way we treat pigs is truly blood curdling. If there were a god, the whole lot of us would be damned, I'm sure.
  - deleted 3y ago
    [deleted]
loandbehold 3y ago
Is there ANY evidence that "qualia" is a real thing? That sounds like vitalism that was debunked a long time ago.
- nathan_compton 3y ago
  I have some basic observational evidence that it does exist and presumably you do as well. I do suspect that we are thinking about it in some fundamentally wrong way, but I reserve judgment.
- jakelazaroff 3y ago
  I mean… you experience stuff, right? "Qualia" is the word for that. We can tweak the definition, but I think it's pretty obvious that "subjective, conscious experience" [1] does in fact exist. [1] https://en.wikipedia.org/wiki/Qualia https://en.wikipedia.org/wiki/Qualia

ARandomerDude 3y ago

Likewise, the nobody knows what consciousness is mindset is smuggling in the idea that in order to know something at all, you must know it comprehensively. I know exactly what consciousness is based on my experience with it, even though I could not possibly give a comprehensive account of everything consciousness entails. By analogy, I've been married for just shy of 20 years. I know my wife very well. I certainly do not know everything there is to know about her, but I do know her.

eimrine 3y ago
> I know exactly what consciousness is based on my experience with it, even though I could not possibly give a comprehensive account of everything consciousness entails. What is the price of that kind of knowledge? You don't even know where is the border between your knowledge and your absence of knowledge. How much can you tell about consciousness without stepping on "absence of knowledge" field? Pretty nothing, isn't it? > By analogy, I've been married for just shy of 20 years. I know my wife very well. I certainly do not know everything there is to know about her, but I do know her. I have a better example. I speak English for 20 years if to start counting from my first English lesson when I learned my first English word. You can find plenty of silly mistakes in my comments. But at least I know what I can express or understand and what I can not.
mo_42 3y ago
> I know exactly what consciousness is based on my experience with it, even though I could not possibly give a comprehensive account of everything consciousness entails. Not sure if personal experiences count. Generally, we laugh at people who talk about esoteric experiences. So a simple explanation could be that consciousness is an illusion? Or put differently, is there any phenomenon that needs the assumption of consciousness? The way I experience myself could be just the history of experiences. So there is something that the brain can refer to.
- ARandomerDude 3y ago
  > So a simple explanation could be that consciousness is an illusion? Doesn't having an illusion presuppose consciousness?
- nathan_compton 3y ago
  What perceives the illusion?
  - visarga 3y ago
    So "the illusion" is like a GPT or StableDiffusion model making stuff up based on conditioning. Nothing mysterious, we have AI that can do that. The same simulator predicts not just how the world will evolve, but also actions and their estimated rewards. It's an imagination based planning system. Bringing all perceptions together into the simulation, integrating them into the same reference system, and using them to imagine, plan, act and learn - that could be consciousness.
    - nathan_compton 3y ago
      Yes, but why would such a system experience anything? I'm a died in the wool monist materialist but I can acknowledge that there is something tricky (at the very least) going on here.
      - visarga 3y ago
Symmetry 3y ago
Scientific study of consciousness comes from ignoring our subjective impressions of our own consciousnesses, which might be illusory, and going only by what can be seen by other people. So you have experiments doing things like showing subjects subliminal images trying to probe the boundaries between conscious experience and unconscious experience. You start with results like "If we show this image to subjects for 50 ms it only has a slight effect on their behavior which fades out after a second, but if we show it to them for 60 ms it has a large effect for the rest of the experiment including being able to talk about it" and then you keep going from there.
- dinvlad 3y ago
  This is kind of a stretch of an argument though. We could say the same about physics and any other sciences - everything is an abstraction at some level, but if this abstraction is reproduced by multiple independent types of measurement and is falsifiable, that is what we call scientific. I don't think LLMs pass this test.
  - Symmetry 3y ago
    I certainly wouldn't say that a traditional LLM is conscious. Once an input falls off of a LLM's input buffer it ceases to have any effect on its output, the exact same way that a subliminal stimuli's effects are limited in a human brain. The size, in bytes, of a LLM's input buffer isn't all that far off from a human brain's input activations either. So strictly feed-forward neural networks aren't conscious in this sense but its easy to image that architectural changes might provide an analogue to what a humans' consciousness provides.
JohnFen 3y ago
> I know exactly what consciousness is based on my experience with it OK, then what is it?

ben_w 3y ago

> Where do people get off saying no one has any idea what consciousness is? I'm not "getting off" saying that, but I do say it often. For me, it's important to know: If we think an artificial neutral network can have consciousness and we're wrong, then there is a risk of all the people who want to have their minds uploaded having a continued existence no better than the one of TV stars reproduced on VHS tape. There is also a risk of this being done as a standard treatment for lesser injuries, especially if it's cheaper. If we think an artificial neutral network can't have consciousness and we're wrong, then there is a risk of creating a new slave class that makes real the fears of the Haitian slaves in the form of the Vodou concept of a zombie — not even death will free them from eternal slavey.

nathan_compton 3y ago
Well, that is an entirely different question. My personal view is that nothing prevents an artificial neural network from having a consciousness (I don't think it makes sense to believe there is anything magical about human brains). What I am saying is that we emphatically know things about the physical processes that (almost certainly) generate consciousness and that we should take that knowledge seriously when examining artificial neural networks. People eager to attribute more to these networks than they plausibly constitute love to dismiss all this knowledge so as to muddy the waters of comparison.
- ben_w 3y ago
  > What I am saying is that we emphatically know things about the physical processes that (almost certainly) generate consciousness I'm prepared to believe that people who aren't me know such things, but last time I asked a PhD in brain research about this (a while ago now), they seemed to disagree. At least, assuming we're talking about the same usage of the word "consciousness" here — when it's defined as "opposite of unconscious" then sure we have drugs to turn that off, and also separately with the non-overlapping definition of "opposite of autonomous or reflexive"… …but the weird thing where I have an experience rather than just producing responses to stimuli? If anyone knows about that, my search engine bubble hides it from me.

vernon99 3y ago

This is a very interesting point that requires some examples and further elaboration to have value for the readers. It refutes but doesn't provide arguments. Can you please elaborate?

avg_dev 3y ago

thank you for this. i am not well up on consciousness (or machine learning) and i have seen chatbots/llms hallucinate and such, and i have also seen them do amazing things. i have wondered to myself a few times lately: how do i know that im dissimilar in nature from these things? so i ask you a followup question: what are some easy to understand ways in which a human's thought process would differ from an llm's behavior? also for anyone else wondering: https://www.merriam-webster.com/dictionary/qualia https://www.merriam-webster.com/dictionary/qualia

nathan_compton 3y ago
There are a lot. First of all, these LLMs do not learn in-situ. They are entirely static (apart from the prompt). To teach an LLM something new is an ex-situ process, more or less totally unrelated to the way it predicts. Contrast that with a brain: brains are constantly learning (in fact, it is difficult to imagine how a brain as we understand it could work without constantly learning). In a related way, because we learn on-line and constantly, our brains have to also maintain goals, rewards and punishments, etc etc. We have neurons for all of the trivia of keeping us moving, seeking new input, generalizing it, throwing away bad information, etc. For an LLM all of that is external. The LLM doesn't have any reason to even distinguish between generation and training. All the weight updates are calculated by a (relatively simple) external process. Furthermore, LLMs are entirely _feed forward_. The input comes in, a lot of numbers are crunched, and then output comes out. There is no rumination (again, the analogy for rumination in an LLM is in the training process, which is not embodied in the LLM). Much of the content of our consciousness is perceptions relating to all of these things. I think its possible that artificial neural networks may one day do enough of these things that I would admit they are conscious, but architecturally and fundamentally, I don't see any reason that an LLM would have them. I also don't think even GPT4 is that intelligent (fantastic recall, though). It does an impression of a cognitive process (literally by printing out steps) but that doesn't seem compelling enough for me to imagine a theory of mind underneath. A model of text, sure, but not a mind.
- avg_dev 3y ago
  Wow, thanks again! I really enjoyed this response and I have learned from it. (Which I guess an LLM could not do while generating something!) Some references I glanced at (I mostly read the top paras): - https://en.wikipedia.org/wiki/In_situ https://en.wikipedia.org/wiki/In_situ - https://en.wikipedia.org/wiki/Theory_of_mind https://en.wikipedia.org/wiki/Theory_of_mind Really enjoyed this response, and feel like I've developed a better understanding of some of the concepts relating to generative ML as it is used in LLMs. An aside: I took a course on ML in a university a few years back, and it was interesting (it was an intro and survey course offered by the CompSci faculty), but difficult for me. I excelled at implementing using Keras/TF code in Python, and I had fun manually implementing some gradient descent algorithm but a lot of the math including all of the multi-var calc, stats, probability was quite difficult for me to wrap my head around, and I really didn't feel like I got a solid grounding on a meta-level of what we were doing or why. I have been reading a bit about LLMs and I think your post has filled in some of the gaps in what at this point I was really looking to understand.

deleted 3y ago

[deleted]

emptysongglass 3y ago

Unfortunately, this is the thread that all these arguments begin: demonstrative ignorance of either the science, the research, or both. They originate in ignorance and because of ignorance, the response is terrible fear and prophecies of the End Times. It's no different than any other brand of fear. There is no evidence that processing power = mind. None. There is no evidence that the human condition is any way related to some kind of terra firma of logic. In fact, there's considerable evidence that feelings are so entangled in the experience of humanness that the idea of divorce or separation is a false one. "Being human" is primarily a feeling experience that drives narratives, motivations: it underlies every single activity we engage in. This is why people like Eliezer Yudkowsky and his ilk are so totally off the mark: it's no coincidence that the Less Wrong community and AI doomsayers can often be found on the same side of the aisle. Both camps believe in and idealize a distinct logic mind that can be attained. Funnily enough, it's still fear, a very human feeling, that is the basis for all these proclamations. My worry is this camp garners enough influence to convince someone an AI doomsday is right around the corner unless immediate action is taken.

fnordpiglet 3y ago
Buddhism says that this isn’t at all what being a human is. It says that the feelings and narratives are an aspect of the human mind, but not core to it. My conception is they are like feedback sources/optimizers and constraint engines. The worries, narratives, voices, chattering are self important feeling and trick us into believing they are who we are. However, a key aspect of vipassana and related mindfulness is building the ability to be aware of those processes from a part of us that has no voice, has no experienced feelings, but is able to be aware of them independently and control them. This source is where our agency derives from, and it does not have to be driven by feelings or the narratives, it is in fact able to suspend them and simply exist as it is. This is what is known as nirvana, which has attained weird mystical meanings in our culture, but essentially means attaining and maintaining a mind that has fully subordinated the “self” driving narrative and emotional soup. The loudness of feelings and our chattering monkey mind self support themselves internally as being “who we are,” but, again my own conception, their importance is an illusion they create internally. All this said, they are certainly a part of what makes us human, in so far as a foot makes us human. But you can function and live a full and complete life in the state of nirvana without losing anything. In fact in my 30 year practice at one point I was scared to bring the practice into my daily lived life fearing being uncompelled by these processes and having a clear mind would make a robot or something - but the opposite was true. At some core level I knew my experiences and connections deeper than a feeling, and the people around me felt I was finally with them for the first time. My point here is that the western conception of what it means to be a human is not particularly simple and it’s not the case, assuming thousands of years of Buddhist practice isn’t a crock, that our feelings and thoughts are the core of what it is to be human. Further - if they are illusions and feedback systems, they can be simulated as constraining feedback systems in an artificial mind just as easily. I think the nature of what is human is much deeper in our minds, but because it’s not easy to examine like feelings and thoughts, I think we really do not understand it very well. This leads me to my long labored point - I agree with the original poster that we don’t understand consciousness. I believe we over estimate our understanding of what it means to be human. I do not however think our machines will achieve it either. But I don’t know why we need to make an artificial human. AI means intelligence, not human. A natural human takes 9 months and we have too many of them, let’s try for something different.

SamBam 3y ago

> I agree that there is a significant sliver of a philosophical problem which remains stubborn (how precisely does physical activity produce qualia) But that IS the definition of consciousness! This is like saying "We understand practically everything about airplanes, except how they stay in the air." Nothing discussed in neuroscience is relevant to understanding what consciousness IS (which is the question posed above). Finding out that stimulating such and such a region makes us sad, or that this bundle of nerves activates before we're consciously aware of a decision doesn't tell us anything about consciousness itself. We've known for hundreds of years that there is a relationship between the brain and consciousness, finding out more details doesn't answer the question. (Now, whether consciousness is necessary for AGI is a separate question.)

nathan_compton 3y ago
I believe it is eminently credible draw a strong association between consciousness and the physical activity of the brain since it is relatively well backed up by scientific observation that there is a one to one correspondence between conscious experience and brain activity. Although we still don't understand precisely how the physical activity creates qualia, I think its perfectly reasonable to say that studying and understanding brain activity constitutes studying and understanding consciousness.
- JohnFen 3y ago
  Sure, but none of the means we know what "consciousness" is.
  - c22 3y ago
    I know what consciousness "is." I'm experiencing it right now.
    - Bridgoo 3y ago
      Prove it

panda-giddiness 3y ago

> I agree that there is a significant sliver of a philosophical problem which remains stubborn (how precisely does physical activity produce qualia) But that "sliver" of a problem is known as the hard problem of consciousness for a reason [1], which is exactly the sort of problem neuroscience can only address in a limited capacity. Understanding how nerves propagate a signal to produce a sensory input (an "easy" problem of consciousness) doesn't inform us as to why certain physical mechanisms result in conscious experience (or more fundamentally what it even means to have a conscious experience). To return to the topic at hand, a stochastic parrot generates grammatical, sensible language without understanding its underlying meaning. Of course, you can debate what it means to understand something; but for a person to vocalize an idea they understand, they must first somehow consciously process that idea. This is firmly a hard problem to which neuroscience offers limited guidance. Of course, I'd agree that human beings aren't stochastic parrots -- if human beings were stochastic parrots, then what would it even mean to understand something? But I doubt you could use neuroscience to ascertain whether large language models are or aren't stochastic parrots. Indeed, depending on your definition of "understanding", consciousness might not even be a prerequisite, making the comparison to neuroscience moot. --- [1] https://en.wikipedia.org/wiki/Hard_problem_of_consciousness https://en.wikipedia.org/wiki/Hard_problem_of_consciousness

fnovd 3y ago

Here's a thought experiment: what are the qualities of a parrot that would make its consciousness different than that of a stochastic parrot? For that matter, what are the qualities that separate a parrot's consciousness from a human's? Or a human's from a pig's? Given the way we treat pigs, we clearly don't think consciousness in and of itself is worthy of any formal consideration, as long as the benefit we derive from exploiting it is high enough. So, with AI, is it fair to say that anyone really cares whether it will develop qualities that make it seem as though it is an emergent consciousness? Why would we treat digital consciousness any better than we treat organic consciousness? What is the point of pontificating whether or not the type of thinking an AI does crosses an arbitrary threshold when that threshold only exists as a tool for creating useful outgroups? However sophisticated the thing that our thinking is, it exists on a scale and we sit at an arbitrary spot. We treat thinking that occurs further down the scale as functionally irrelevant not because of any real distinction but because doing so has a high utility for our species. So, the question of how we will treat a "truly conscious and sentient" AI has already been answered. Look at how we treat pigs. Good luck out there, HAL.

nathan_compton 3y ago
The way we treat pigs is truly blood curdling. If there were a god, the whole lot of us would be damned, I'm sure.
- deleted 3y ago
  [deleted]

loandbehold 3y ago

Is there ANY evidence that "qualia" is a real thing? That sounds like vitalism that was debunked a long time ago.

nathan_compton 3y ago
I have some basic observational evidence that it does exist and presumably you do as well. I do suspect that we are thinking about it in some fundamentally wrong way, but I reserve judgment.
jakelazaroff 3y ago
I mean… you experience stuff, right? "Qualia" is the word for that. We can tweak the definition, but I think it's pretty obvious that "subjective, conscious experience" [1] does in fact exist. [1] https://en.wikipedia.org/wiki/Qualia https://en.wikipedia.org/wiki/Qualia