5 ms·

In Buddhism there’s the idea that our core self is awareness, which is silent - it doesn’t think in a perceptible way, it doesn’t feel in a visceral way, but it

In Buddhism there’s the idea that our core self is awareness, which is silent - it doesn’t think in a perceptible way, it doesn’t feel in a visceral way, but it underpins thought and feeling, and is greatly impacted by it. A large part of meditation and “release of suffering” is learning to let your awareness lead your thinking rather than your thinking lead your awareness. To be clear, I think this is in fact a correct assessment of the architecture of intelligence. You can suspend thought and still function throughout your day in all ways. Discursive thought is entirely unnecessary, but it is often helpful for planning. My observation of LLMs in such a construction of intelligence is they are entirely the thinking mind - verbal, articulate, but unmoored. There is no, for lack of a better word, “soul,” or that internal awareness that underpins that discursive thinking mind. And because that underlying awareness is non articulate and not directly observable by our thinking and feeling mind, we really don’t understand it or have a science about it. To that end, it’s really hard to pin specifically what is missing in LLMs because we don’t really understand ourselves beyond our observable thinking and emotive minds. I look at what we are doing with LLMs and adjacent technologies and I wonder if this is sufficient, and building an AGI is perhaps not nearly as useful as we might think, if what we mean is build an awareness. Power tools of the thinking mind are amazingly powerful. Agency and awareness - to what end? And once we do build an awareness, can we continue to consider it a tool?

danenania 3y ago
Another idea from Buddhism is that this core of awareness you're talking about is nothingness. So when you stop all thought (if such a thing is really possible), you temporarily cease to exist as an individual consciousness. "Awareness" is when the thoughts come back online and you think "whoa, I was just gone for a bit". If that's how it works, then the "soul" is more like an emergent phenomenon created by the interplay between the various layers of conscious thought and the base layer of nothingness when it's all turned off. That architecture wouldn't necessarily be so difficult to replicate in AI systems.
- NoMoreNicksLeft 3y ago
  > So when you stop all thought (if such a thing is really possible), It's not. They don't realize it, they're merely referring to stopping your internal monologue. There are dozens of other mental processes going on in any given waking moment. Even actual top shelf cognition is going on, it just occurs in a "language of thought".
  - astrange 3y ago
    > It's not. They don't realize it, they're merely referring to stopping your internal monologue. They certainly have realized that. It's one of the first things you notice doing awareness meditation; thoughts appear from nowhere even if you didn't try to think them.
    - NoMoreNicksLeft 3y ago
      [flagged]
      - astrange 3y ago
        That seems like an unusually rude thing to say about Buddha. You might be experiencing dukkha.
      - fnordpiglet 3y ago
        Are you saying every thought you’ve ever had is a logical consequence of all prior thoughts with a definite traceable lineage? (Assuming your mind was too dumb as some point and a thought emerged that originated all future thoughts)
        NoMoreNicksLeft 3y ago
        I'm saying that I'm extremely confident that... 1. Most people (including Buddhists) think that their internal monologue comprises their thoughts, in majority or even in total. 2. Can't conceive of the possibility of a thought existing other than expressed in their spoken language 3. Find it difficult or impossible to suppress their inner monologue. 4. When successful at suppressing it believe that their thoughts have ceased. 5. Are apparently unaware of the paradox that belief entails... if they are no longer thinking, how is the decision made to initiate thinking once more? That's a conscious decision, or else successful Buddhists would turn into vegetables and die of starvation in their little meditation pose. This isn't a matter of speculation anymore. We can see inside the brain, non-invasively, while these things occur. Any mystical element is an artifact of people poorly defining words, or being completely ignorant of how a brain must operate in principle. You're all very confused.
jacobsimon 3y ago
This is a profound question but I also wonder if this non-thinking “awareness” you’re referring to is largely defined by quieting the thinking mind and listening to the senses more directly. A lot of meditation is about tuning out thoughts and focusing on proprioception like breathing, the feelings of the body, etc.
- vjerancrnjak 3y ago
  Fundamentally, this "awareness" isn't defined by quieting the thinking. It is a description of fundamental reality. No individual should be able to experience it, and the "glimpses" are just forms of brain dysfunction. Meditation techniques that focus on breath or the body are an attempt to make you do the breathing/sensing consciously. If you film yourself and later look at what you did, you'll notice you aren't breathing well when you're breathing consciously, so you're probably depriving yourself of oxygen, lowering blood concentration in certain brain regions and you hope it will be the brain region associated with conceptualizing, language etc. You can do the same with sleep. You can try to consciously fall asleep, and just like breathing, you will have a hard time because there's a reason why falling asleep is not conscious (or in other words it does not go through the regions of the brain that conceptualize). You can experience the balance center shutting down (feels like falling or turning) and you can go even deeper and feel the fear of the "ego" dying (temporarily). What remains is definitely much different than waking or dreaming state. But it is still not that "awareness/nothingness".
  - fnordpiglet 3y ago
    I think this is entirely incorrect. Vipassana meditation, the type focused on breathing, require intense awareness of your breathing and physical body. It’s a similar state to when you intensely focus on what’s around you and everything gets brighter and more vibrant and you pick out a lot of details you normally don’t notice because you’re distracted by your thoughts. If you’re doing it the way intended you would 100% be aware of your irregular breath or pausing. In fact beginners vipassana often advises counting the breaths individually in a cycle 1..10, and resetting the count when you lose track of your breathing. You intensely focus on the sensation of the air moving through your nostrils, the muscles contracting, your clothing shifting. However it’s not about controlling your breathing, so it’s not the same as breathing consciously. It’s observing passively. Often you’ll notice that you are breathing irregularly, not because of the meditation, but because you typically are stressed and tight in your musculature due to the way you’re thinking. You can then loosen and reset your patterns of breath to be more natural, deep, and complete. A goal isn’t to stop with observing the breath though, and you work towards having a total awareness of the entire body at once, shifting your center of existence from your head to the rest of your body. You then incorporate sounds and events in your environment. This requires an intense amount of mental power, and is entirely different from your description of oxygen deprivation. Thought ceases because it interferes with being aware, not because you are experiencing brain death.
sdwr 3y ago
Maybe the soul is social, and oriented towards others? I believe it can be constructed. If you assume that "the eyes are the window to the soul", you notice some interesting properties. 1. It is far more observable from the outside (eyes open/lidded/closed, emotion read in eyes) 2. It affects behavior in a diffuse way 3. It pays attention but does not dictate
- munificent 3y ago
  > Maybe the soul is social My pet theory about human consciousness is that is that consciousness is simply recursive theory of mind. Theory of mind [1] is our ability to simulate and reason about the mental states of others. It's how we predict what people are thinking and how they will react to our actions, which is critical for choosing how to act in a social environment. But when you're thinking about what's in someone's head, one of the things might be them thinking about you. So now you're imagining your own mind from the perspective of another mind. I believe that's entirely what our sense of consciousness is. It's our social reasoning applied to ourselves. If my pet theory is correct, it implies that the level of consciousness of any species would directly correlate to how social the species is. Solitary animals with little need for theory of mind would have no self awareness in the way that we experience it. They'd live in a zen-like perpetual auto-pilot where they do but couldn't explain why they do what they do... because they will never explain it to anyone anyway. [1]: https://en.wikipedia.org/wiki/Theory_of_mind https://en.wikipedia.org/wiki/Theory_of_mind
  - crdrost 3y ago
    Theory of mind is interesting but one wouldn't want to hinge consciousness upon it. That direction would likely contain weird outcomes if the science progressed, something like "Dogs are barely-conscious due to their pack structure, they have a couple levels of recursive theory of mind but they can't sustain it as deep as we can. But cats didn't have that pack structure, they're not conscious at all." Or, "this person has such severe autism that he cannot fundamentally understand others' minds or what others interpret his mind to be, so we've downgraded his classification to unconscious. He'll talk your ear off about the various cars produced in a golden age between 1972 and 1984, but because he doesn't really know what it means for you to be listening we regard it as sleep-talking." It also just kind of doesn't sound right. "What happens when we go to sleep? Well, we stop thinking about what others think we think, and we simply accept what they think about us." That doesn't sound like any sleep I experience -- it might describe some of my dreams, but of course dreams are anomalous conscious experiences that happen during sleep so that also misses the mark.
deleted 3y ago
[deleted]
dimal 3y ago
I’ve been thinking along similar lines. It’s like with LLMs, they’ve created the part of the mind that is endlessly chattering, generating stories, sometimes true, sometimes false, but there’s no awareness or consciousness that ever steps back and can see thoughts as thoughts. And I don’t see how awareness or consciousness would arise from just more of the same (bigger models). It seems to be a fundamentally different part of the mind. I wonder if AGI is possible without this. AGI under some definition (good enough to replace most humans) may be possible. But it wouldn’t be aware. And without awareness, I don’t see how it could be aligned. It may appear to be aligned but then eventually it would probably get caught in a delusional feedback loop that it has no capacity to escape, because it can’t be aware of its own delusion.
- hotpotamus 3y ago
  > It may appear to be aligned but then eventually it would probably get caught in a delusional feedback loop that it has no capacity to escape, because it can’t be aware of its own delusion. I believe this is more or less the definition of human mental illness. I have to say that while I know it's really not possible, I wish people would stop pulling on these threads. I got into this line of work because I thought video games were cool, not because I wanted to philosophize about theories of mind and what intelligence is. I really don't like thinking about whether I'm just some sort of automaton made out of meat rather than metal and silicon.
  - pixl97 3y ago
    Ah, the first releases of the Bing AI were fun here as they plunged into feedback loops of madness that were scarily human sounding. Thank you humanity for making artificial insanity.
pixl97 3y ago
https://en.wikipedia.org/wiki/Moravec%27s_paradox https://en.wikipedia.org/wiki/Moravec%27s_paradox While you're adding a bunch of eastern philosophy to it, we need to take a step back from 'human' intelligence and go to animal and plant intelligence to get a better idea of the massive variation in what covers thought. In animal/insects we can see that thinking is not some binary function of on or off. It is an immense range of different electrical and chemical processes that involve everything from the brain and the nerves along with chemical signaling from cells. In things like plants and molds 'thinking' doesn't even involve nerves, it's a chemical process. A good example of this at the human level is a reflex. Your hand didn't go back to your brain to ask for instructions on how to get away from the fire. That's encoded in the meat and nerves of your arm by systems that are much older than higher intelligence. All the systems for breath, drink, eat, procreate were in place long before high level intelligence existed. Intelligence just happens to be a new floor stacked hastily on top of these legacy systems that happened to be beneficial enough it didn't go extinct. Awareness is another one of those very deep rabbit hole questions. There are 'intelligent' animals without self awareness, but with awareness of the world around them. And they obviously have agency. Of course this is where the AI existentialists come in and say wrapping up agency, awareness, and superintelligence may not work out for humans as well as we expect.
- thegiogi 3y ago
  > A good example of this at the human level is a reflex. Your hand didn't go back to your brain to ask for instructions on how to get away from the fire. Is this actually true? I thought it just involved a different part of the brain. Is there actually no brain involvement? Sure it does not need your awareness or decision making, but no brain? I find that hard to believe.
  - pixl97 3y ago
    https://en.wikipedia.org/wiki/Reflex_arc https://en.wikipedia.org/wiki/Reflex_arc Simple answer: No, it does not go to the brain Detailed answer: We are complex as all hell. >A reflex arc is a neural pathway that controls a reflex. In vertebrates, most sensory neurons do not pass directly into the brain, but synapse in the spinal cord. This allows for faster reflex actions to occur by activating spinal motor neurons without the delay of routing signals through the brain. The brain will receive the input while the reflex is being carried out and the analysis of the signal takes place after the reflex action.
Merrill 3y ago
Decision making seems fundamental to intelligence, is done by animals and humans, and can be done without the use of language or logic. This is the case when someone "decided without thinking". Decision making requires imagination or the ability to envision alternative future states that may result from various choices. Imagination is the start of abstract thinking. Consciousness results from the individual thinking abstractly about itself and how it interacts with the world.

danenania 3y ago

Another idea from Buddhism is that this core of awareness you're talking about is nothingness. So when you stop all thought (if such a thing is really possible), you temporarily cease to exist as an individual consciousness. "Awareness" is when the thoughts come back online and you think "whoa, I was just gone for a bit". If that's how it works, then the "soul" is more like an emergent phenomenon created by the interplay between the various layers of conscious thought and the base layer of nothingness when it's all turned off. That architecture wouldn't necessarily be so difficult to replicate in AI systems.

NoMoreNicksLeft 3y ago
> So when you stop all thought (if such a thing is really possible), It's not. They don't realize it, they're merely referring to stopping your internal monologue. There are dozens of other mental processes going on in any given waking moment. Even actual top shelf cognition is going on, it just occurs in a "language of thought".
- astrange 3y ago
  > It's not. They don't realize it, they're merely referring to stopping your internal monologue. They certainly have realized that. It's one of the first things you notice doing awareness meditation; thoughts appear from nowhere even if you didn't try to think them.
  - NoMoreNicksLeft 3y ago
    [flagged]
    - astrange 3y ago
      That seems like an unusually rude thing to say about Buddha. You might be experiencing dukkha.
    - fnordpiglet 3y ago
      Are you saying every thought you’ve ever had is a logical consequence of all prior thoughts with a definite traceable lineage? (Assuming your mind was too dumb as some point and a thought emerged that originated all future thoughts)
      - NoMoreNicksLeft 3y ago
        I'm saying that I'm extremely confident that... 1. Most people (including Buddhists) think that their internal monologue comprises their thoughts, in majority or even in total. 2. Can't conceive of the possibility of a thought existing other than expressed in their spoken language 3. Find it difficult or impossible to suppress their inner monologue. 4. When successful at suppressing it believe that their thoughts have ceased. 5. Are apparently unaware of the paradox that belief entails... if they are no longer thinking, how is the decision made to initiate thinking once more? That's a conscious decision, or else successful Buddhists would turn into vegetables and die of starvation in their little meditation pose. This isn't a matter of speculation anymore. We can see inside the brain, non-invasively, while these things occur. Any mystical element is an artifact of people poorly defining words, or being completely ignorant of how a brain must operate in principle. You're all very confused.

jacobsimon 3y ago

This is a profound question but I also wonder if this non-thinking “awareness” you’re referring to is largely defined by quieting the thinking mind and listening to the senses more directly. A lot of meditation is about tuning out thoughts and focusing on proprioception like breathing, the feelings of the body, etc.

vjerancrnjak 3y ago
Fundamentally, this "awareness" isn't defined by quieting the thinking. It is a description of fundamental reality. No individual should be able to experience it, and the "glimpses" are just forms of brain dysfunction. Meditation techniques that focus on breath or the body are an attempt to make you do the breathing/sensing consciously. If you film yourself and later look at what you did, you'll notice you aren't breathing well when you're breathing consciously, so you're probably depriving yourself of oxygen, lowering blood concentration in certain brain regions and you hope it will be the brain region associated with conceptualizing, language etc. You can do the same with sleep. You can try to consciously fall asleep, and just like breathing, you will have a hard time because there's a reason why falling asleep is not conscious (or in other words it does not go through the regions of the brain that conceptualize). You can experience the balance center shutting down (feels like falling or turning) and you can go even deeper and feel the fear of the "ego" dying (temporarily). What remains is definitely much different than waking or dreaming state. But it is still not that "awareness/nothingness".
- fnordpiglet 3y ago
  I think this is entirely incorrect. Vipassana meditation, the type focused on breathing, require intense awareness of your breathing and physical body. It’s a similar state to when you intensely focus on what’s around you and everything gets brighter and more vibrant and you pick out a lot of details you normally don’t notice because you’re distracted by your thoughts. If you’re doing it the way intended you would 100% be aware of your irregular breath or pausing. In fact beginners vipassana often advises counting the breaths individually in a cycle 1..10, and resetting the count when you lose track of your breathing. You intensely focus on the sensation of the air moving through your nostrils, the muscles contracting, your clothing shifting. However it’s not about controlling your breathing, so it’s not the same as breathing consciously. It’s observing passively. Often you’ll notice that you are breathing irregularly, not because of the meditation, but because you typically are stressed and tight in your musculature due to the way you’re thinking. You can then loosen and reset your patterns of breath to be more natural, deep, and complete. A goal isn’t to stop with observing the breath though, and you work towards having a total awareness of the entire body at once, shifting your center of existence from your head to the rest of your body. You then incorporate sounds and events in your environment. This requires an intense amount of mental power, and is entirely different from your description of oxygen deprivation. Thought ceases because it interferes with being aware, not because you are experiencing brain death.

sdwr 3y ago

Maybe the soul is social, and oriented towards others? I believe it can be constructed. If you assume that "the eyes are the window to the soul", you notice some interesting properties. 1. It is far more observable from the outside (eyes open/lidded/closed, emotion read in eyes) 2. It affects behavior in a diffuse way 3. It pays attention but does not dictate

munificent 3y ago
> Maybe the soul is social My pet theory about human consciousness is that is that consciousness is simply recursive theory of mind. Theory of mind [1] is our ability to simulate and reason about the mental states of others. It's how we predict what people are thinking and how they will react to our actions, which is critical for choosing how to act in a social environment. But when you're thinking about what's in someone's head, one of the things might be them thinking about you. So now you're imagining your own mind from the perspective of another mind. I believe that's entirely what our sense of consciousness is. It's our social reasoning applied to ourselves. If my pet theory is correct, it implies that the level of consciousness of any species would directly correlate to how social the species is. Solitary animals with little need for theory of mind would have no self awareness in the way that we experience it. They'd live in a zen-like perpetual auto-pilot where they do but couldn't explain why they do what they do... because they will never explain it to anyone anyway. [1]: https://en.wikipedia.org/wiki/Theory_of_mind https://en.wikipedia.org/wiki/Theory_of_mind
- crdrost 3y ago
  Theory of mind is interesting but one wouldn't want to hinge consciousness upon it. That direction would likely contain weird outcomes if the science progressed, something like "Dogs are barely-conscious due to their pack structure, they have a couple levels of recursive theory of mind but they can't sustain it as deep as we can. But cats didn't have that pack structure, they're not conscious at all." Or, "this person has such severe autism that he cannot fundamentally understand others' minds or what others interpret his mind to be, so we've downgraded his classification to unconscious. He'll talk your ear off about the various cars produced in a golden age between 1972 and 1984, but because he doesn't really know what it means for you to be listening we regard it as sleep-talking." It also just kind of doesn't sound right. "What happens when we go to sleep? Well, we stop thinking about what others think we think, and we simply accept what they think about us." That doesn't sound like any sleep I experience -- it might describe some of my dreams, but of course dreams are anomalous conscious experiences that happen during sleep so that also misses the mark.

deleted 3y ago

[deleted]

dimal 3y ago

I’ve been thinking along similar lines. It’s like with LLMs, they’ve created the part of the mind that is endlessly chattering, generating stories, sometimes true, sometimes false, but there’s no awareness or consciousness that ever steps back and can see thoughts as thoughts. And I don’t see how awareness or consciousness would arise from just more of the same (bigger models). It seems to be a fundamentally different part of the mind. I wonder if AGI is possible without this. AGI under some definition (good enough to replace most humans) may be possible. But it wouldn’t be aware. And without awareness, I don’t see how it could be aligned. It may appear to be aligned but then eventually it would probably get caught in a delusional feedback loop that it has no capacity to escape, because it can’t be aware of its own delusion.

hotpotamus 3y ago
> It may appear to be aligned but then eventually it would probably get caught in a delusional feedback loop that it has no capacity to escape, because it can’t be aware of its own delusion. I believe this is more or less the definition of human mental illness. I have to say that while I know it's really not possible, I wish people would stop pulling on these threads. I got into this line of work because I thought video games were cool, not because I wanted to philosophize about theories of mind and what intelligence is. I really don't like thinking about whether I'm just some sort of automaton made out of meat rather than metal and silicon.
- pixl97 3y ago
  Ah, the first releases of the Bing AI were fun here as they plunged into feedback loops of madness that were scarily human sounding. Thank you humanity for making artificial insanity.

pixl97 3y ago

https://en.wikipedia.org/wiki/Moravec%27s_paradox https://en.wikipedia.org/wiki/Moravec%27s_paradox While you're adding a bunch of eastern philosophy to it, we need to take a step back from 'human' intelligence and go to animal and plant intelligence to get a better idea of the massive variation in what covers thought. In animal/insects we can see that thinking is not some binary function of on or off. It is an immense range of different electrical and chemical processes that involve everything from the brain and the nerves along with chemical signaling from cells. In things like plants and molds 'thinking' doesn't even involve nerves, it's a chemical process. A good example of this at the human level is a reflex. Your hand didn't go back to your brain to ask for instructions on how to get away from the fire. That's encoded in the meat and nerves of your arm by systems that are much older than higher intelligence. All the systems for breath, drink, eat, procreate were in place long before high level intelligence existed. Intelligence just happens to be a new floor stacked hastily on top of these legacy systems that happened to be beneficial enough it didn't go extinct. Awareness is another one of those very deep rabbit hole questions. There are 'intelligent' animals without self awareness, but with awareness of the world around them. And they obviously have agency. Of course this is where the AI existentialists come in and say wrapping up agency, awareness, and superintelligence may not work out for humans as well as we expect.

thegiogi 3y ago
> A good example of this at the human level is a reflex. Your hand didn't go back to your brain to ask for instructions on how to get away from the fire. Is this actually true? I thought it just involved a different part of the brain. Is there actually no brain involvement? Sure it does not need your awareness or decision making, but no brain? I find that hard to believe.
- pixl97 3y ago
  https://en.wikipedia.org/wiki/Reflex_arc https://en.wikipedia.org/wiki/Reflex_arc Simple answer: No, it does not go to the brain Detailed answer: We are complex as all hell. >A reflex arc is a neural pathway that controls a reflex. In vertebrates, most sensory neurons do not pass directly into the brain, but synapse in the spinal cord. This allows for faster reflex actions to occur by activating spinal motor neurons without the delay of routing signals through the brain. The brain will receive the input while the reflex is being carried out and the analysis of the signal takes place after the reflex action.

Merrill 3y ago

Decision making seems fundamental to intelligence, is done by animals and humans, and can be done without the use of language or logic. This is the case when someone "decided without thinking". Decision making requires imagination or the ability to envision alternative future states that may result from various choices. Imagination is the start of abstract thinking. Consciousness results from the individual thinking abstractly about itself and how it interacts with the world.