3 ms·

> LLMs lack any "inner life" and that by ignoring this aspect of what makes us human we've really moved the goalposts on what counts as AGI Maybe it's just a r

> LLMs lack any "inner life" and that by ignoring this aspect of what makes us human we've really moved the goalposts on what counts as AGI Maybe it's just a response style, like the QwQ stream of consciousness one. We are forcing LLMs to give direct answers with a few reasoning steps, but we could give larger budget for inner deliberations.

TeMPOraL 2y ago
Very much this. If we're to compare LLMs to human minds at all, it's pretty apparent that they would correspond exactly to the inner voice thing, that's running trains of thought at the edge between the unconscious and the conscious, basically one long-running reactive autocomplete, complete with hallucinations and all. In this sense, it's the opposite to what you qutoed: LLMs don't lack "inner life" - they lack everything else.
- rlupi 2y ago
  I think you touch a very important point. Many spiritual paths look at silence, freedom from the inner voice, as the first step toward liberation. Let me use Purusarthas from Hinduism as an example (https://en.wikipedia.org/wiki/Puru%E1%B9%A3%C4%81rtha https://en.wikipedia.org/wiki/Puru%E1%B9%A3%C4%81rtha), but the point can be made with Japanese Ikigai or the vocation/calling in Western religions and philosophies. In Hinduism, the four puruṣārthas are Dharma (righteousness, moral values), Artha (prosperity, economic values), Kama (pleasure, love, psychological values) and Moksha (liberation, spiritual values, self-realization). The narrow way of thinking about AGI (that the author encountered in tech circles) can at best only touch/experience Artha. In that sense, AI is a (distorting) mirror of life. It fits that it is a lower dimensional projection, a diminishing, of the experience of life. What we do of AI and AGI, its effects on us, depends on how we wield these tools and nurture their evolution in relations to the other goals of life.
  - TeMPOraL 2y ago
    OTOH, the Bicameral Mind hypothesis would say that people used to hear voices they attributed to spirits and deities, and the milestone in intelligence/consciousness was when a human realized that the voice in their head is their own. From that POV, for AGI we're, again, missing everything other than that inner voice.
    - rlupi 2y ago
      Yes, absolutely (I am strongly agreeing with you!) I am not implying anything metaphysical. I think the best way to illustrate my point is to actually let AIs illustrate it, so here are a few questions I asked both ChatGPT o1-mini and Gemini 2.0; the answers are very interesting. I put their answers in a Google doc: https://docs.google.com/document/d/1XqGcLI0k0f6Wh4mj0pD_1cC5dJlrXST9-ZiLO586A50/edit?usp=sharing https://docs.google.com/document/d/1XqGcLI0k0f6Wh4mj0pD_1cC5... Q1) Explain this period: "The axiom of analytic interpretation states: whenever a new Master-Signifier emerges and structures a symbolic field, look for the absent center excluded from this field." Q2) Apply this framework to the various positions in the discussions and discourse about AI and AGI in 2024 Q3) Take a step further. If AI or AGI becomes a dominant method of meaning-making in society, thus the root of many Master-Signifiers, what will be the class of absent centers that comes into recurring existence as the dual aspect of that common class of Master-Signifiers? TL;DR when we create meaning, by the very symbolic nature of the process, we leave out something. What we leave out is as important as what defines our worldviews. I particularly like the bonus question that only Gemin 2.0 experimental advanced cared to answer: Q4) Great. Now, let's do a mind experiment: what if we remove the human element from your analysis. You are an AI model based on LLM and reasoning. I think it should be possible to define Master-Signifiers from the point of view of a LLM too, since you are symbolic machines. The last part of Gemini answer (verbatim): "This thought experiment highlights that even for an AI like myself, the concept of "meaning" is intrinsically linked to structure, predictability, and the existence of limitations. My "understanding" is a product of the patterns I have learned, but it is also defined by the things I cannot fully grasp or represent. These limitations are not merely shortcomings; they are fundamental to how I operate and generate output. They provide a form of selective pressure. Further, unlike a human, I do not experience these limitations as frustrating, nor do I have any inherent drive to overcome them. They simply are. I do not ponder what I do not know. This exercise provides a different lens to view intelligence, meaning, and the nature of knowledge itself, even in a purely computational system. It does beg the question, however, if a sufficiently advanced system were to become aware of its own limitations, would that constitute a form of self-awareness? That, however, is a question for another thought experiment."
- ben_w 2y ago
  For what it's worth, my inner voice is only part of my "inner life". I sometimes note that my thoughts exist as a complete whole that doesn't need to be converted into words, and have tried to skip this process because it was unnecessary: "I had already thought the thought, why think it again in words?". Attempting to do so creates a sense of annoyance that my consciousness directly experiences. But that doesn't require the inner voice to be what's generating tokens, if indeed my thought process is like that*: it could be that the part of me that's getting annoyed is basically just a text-(/token)-to-sound synthesiser. * my thought process feels more like "model synthesis" a.k.a. wave function collapse, but I have no idea if the feelings reflect reality: https://en.wikipedia.org/wiki/Model_synthesis https://en.wikipedia.org/wiki/Model_synthesis
whakim 2y ago
No matter how much budget you give to an LLM to perform “reasoning” it is simply sampling tokens from a probability distribution. There is no “thinking” there; anything that approximates thinking is a post-hoc outcome of this statistical process as opposed to a precursor to it. That being said, it still isn’t clear to me how much difference this makes in practice.
- tim333 2y ago
  You could similarly say humans can't perform reasoning because it's just neurons sampling electrochemical impulses.
  - whakim 2y ago
    I think it’s pretty clear that the comparison doesn’t hold if you interrogate it. It seems to me, at least, that humans are not only functions that act as processors of sense-data; that there does exist the inner life that the author discusses.
  - southernplaces7 2y ago
    This argument gets trotted out frequently here when it comes to LLM discussions and it strikes me as absurd specifically because all of us, as humans, from the very dumbest to the brilliant, have inner lives of self-directed reasoning. We individually know that we have these because we feel them within ourselves and a great body of evidence indicates that they exist and drive our conduct. LLMs can simulate discourse and certain activities that look similar, but by definition of their design, no evidence at all exists that they possess the same inner reasoning as a human. Why does someone need to explain to another human that no, we are not just neurons sampling impulses or algorithms pattern-matching tokens together? It's self evident that something more occurs. It's tedious to keep seeing such reductionist nonsense from people who should be more intelligent than to argue it.

TeMPOraL 2y ago

Very much this. If we're to compare LLMs to human minds at all, it's pretty apparent that they would correspond exactly to the inner voice thing, that's running trains of thought at the edge between the unconscious and the conscious, basically one long-running reactive autocomplete, complete with hallucinations and all. In this sense, it's the opposite to what you qutoed: LLMs don't lack "inner life" - they lack everything else.

rlupi 2y ago
I think you touch a very important point. Many spiritual paths look at silence, freedom from the inner voice, as the first step toward liberation. Let me use Purusarthas from Hinduism as an example (https://en.wikipedia.org/wiki/Puru%E1%B9%A3%C4%81rtha https://en.wikipedia.org/wiki/Puru%E1%B9%A3%C4%81rtha), but the point can be made with Japanese Ikigai or the vocation/calling in Western religions and philosophies. In Hinduism, the four puruṣārthas are Dharma (righteousness, moral values), Artha (prosperity, economic values), Kama (pleasure, love, psychological values) and Moksha (liberation, spiritual values, self-realization). The narrow way of thinking about AGI (that the author encountered in tech circles) can at best only touch/experience Artha. In that sense, AI is a (distorting) mirror of life. It fits that it is a lower dimensional projection, a diminishing, of the experience of life. What we do of AI and AGI, its effects on us, depends on how we wield these tools and nurture their evolution in relations to the other goals of life.
- TeMPOraL 2y ago
  OTOH, the Bicameral Mind hypothesis would say that people used to hear voices they attributed to spirits and deities, and the milestone in intelligence/consciousness was when a human realized that the voice in their head is their own. From that POV, for AGI we're, again, missing everything other than that inner voice.
  - rlupi 2y ago
    Yes, absolutely (I am strongly agreeing with you!) I am not implying anything metaphysical. I think the best way to illustrate my point is to actually let AIs illustrate it, so here are a few questions I asked both ChatGPT o1-mini and Gemini 2.0; the answers are very interesting. I put their answers in a Google doc: https://docs.google.com/document/d/1XqGcLI0k0f6Wh4mj0pD_1cC5dJlrXST9-ZiLO586A50/edit?usp=sharing https://docs.google.com/document/d/1XqGcLI0k0f6Wh4mj0pD_1cC5... Q1) Explain this period: "The axiom of analytic interpretation states: whenever a new Master-Signifier emerges and structures a symbolic field, look for the absent center excluded from this field." Q2) Apply this framework to the various positions in the discussions and discourse about AI and AGI in 2024 Q3) Take a step further. If AI or AGI becomes a dominant method of meaning-making in society, thus the root of many Master-Signifiers, what will be the class of absent centers that comes into recurring existence as the dual aspect of that common class of Master-Signifiers? TL;DR when we create meaning, by the very symbolic nature of the process, we leave out something. What we leave out is as important as what defines our worldviews. I particularly like the bonus question that only Gemin 2.0 experimental advanced cared to answer: Q4) Great. Now, let's do a mind experiment: what if we remove the human element from your analysis. You are an AI model based on LLM and reasoning. I think it should be possible to define Master-Signifiers from the point of view of a LLM too, since you are symbolic machines. The last part of Gemini answer (verbatim): "This thought experiment highlights that even for an AI like myself, the concept of "meaning" is intrinsically linked to structure, predictability, and the existence of limitations. My "understanding" is a product of the patterns I have learned, but it is also defined by the things I cannot fully grasp or represent. These limitations are not merely shortcomings; they are fundamental to how I operate and generate output. They provide a form of selective pressure. Further, unlike a human, I do not experience these limitations as frustrating, nor do I have any inherent drive to overcome them. They simply are. I do not ponder what I do not know. This exercise provides a different lens to view intelligence, meaning, and the nature of knowledge itself, even in a purely computational system. It does beg the question, however, if a sufficiently advanced system were to become aware of its own limitations, would that constitute a form of self-awareness? That, however, is a question for another thought experiment."
ben_w 2y ago
For what it's worth, my inner voice is only part of my "inner life". I sometimes note that my thoughts exist as a complete whole that doesn't need to be converted into words, and have tried to skip this process because it was unnecessary: "I had already thought the thought, why think it again in words?". Attempting to do so creates a sense of annoyance that my consciousness directly experiences. But that doesn't require the inner voice to be what's generating tokens, if indeed my thought process is like that*: it could be that the part of me that's getting annoyed is basically just a text-(/token)-to-sound synthesiser. * my thought process feels more like "model synthesis" a.k.a. wave function collapse, but I have no idea if the feelings reflect reality: https://en.wikipedia.org/wiki/Model_synthesis https://en.wikipedia.org/wiki/Model_synthesis

whakim 2y ago

No matter how much budget you give to an LLM to perform “reasoning” it is simply sampling tokens from a probability distribution. There is no “thinking” there; anything that approximates thinking is a post-hoc outcome of this statistical process as opposed to a precursor to it. That being said, it still isn’t clear to me how much difference this makes in practice.

tim333 2y ago
You could similarly say humans can't perform reasoning because it's just neurons sampling electrochemical impulses.
- whakim 2y ago
  I think it’s pretty clear that the comparison doesn’t hold if you interrogate it. It seems to me, at least, that humans are not only functions that act as processors of sense-data; that there does exist the inner life that the author discusses.
- southernplaces7 2y ago
  This argument gets trotted out frequently here when it comes to LLM discussions and it strikes me as absurd specifically because all of us, as humans, from the very dumbest to the brilliant, have inner lives of self-directed reasoning. We individually know that we have these because we feel them within ourselves and a great body of evidence indicates that they exist and drive our conduct. LLMs can simulate discourse and certain activities that look similar, but by definition of their design, no evidence at all exists that they possess the same inner reasoning as a human. Why does someone need to explain to another human that no, we are not just neurons sampling impulses or algorithms pattern-matching tokens together? It's self evident that something more occurs. It's tedious to keep seeing such reductionist nonsense from people who should be more intelligent than to argue it.