4 ms·

I sometimes see these reddit threads of people talking about the experience of having an internal monologue. I have no such monologue, at least not one that is

I sometimes see these reddit threads of people talking about the experience of having an internal monologue. I have no such monologue, at least not one that is accessible to the part of my mind that calls itself 'me', but I have often wondered if that monologue is something like a 'chain of thought'. I feel like maybe without access to that 'idea feed' maybe my planning and executive functioning is less effective than some other people. I do find myself quite more effective with those sort of tasks when I do a little 'chain of thought' notepad. I also suspect I spend less time ruminating and second-guessing myself and other anxious behaviours that I imagine would come with having someone talking in your ear all day, but that's probably off topic.

kla-s 2y ago
Genuine question, how does multi step reasoning work for you then? Like eg if you have some math problem that's trivially to solve individually but needs multiple steps, lets say 16 * 3 + 5? How does 16 * 3 = 48 land in some 'register' of your brain (short term memory), so that you can then add 5 to get to 53? Maybe 16 * 3 + 5 is to easy for you and you'll just 'see' it but the question still stands, just choose a more complex problem. Isn't the same meta process at play when thinking about more fuzzy topics?
- cgriswald 2y ago
  I believe I have an internal narrator but I’m not certain exactly what others mean by that so I don’t know for sure. However the way I think about math is different than the way I plan my day or other things. In my case, it is very much like I have registers that would hold the result of 16 x 3 in it so I can add the 5 to it later. I have a certain number of registers and with effort like repeating what Ive already solved I could temporarily create more. It also feels somewhat physical, as if the register is an actual box or has a “location” or like I’ve put the answer down on the desk like a part of something I’m building. Perhaps not coincidentally I am one of the many people who have a “calendar shape” for the months.
- crooked-v 2y ago
  Not that poster, but for me it's directly manipulating numbers (for example, "16×3 + 5" turns into "10×3 + 6×3 + 5" into "30 + 18 + 5" into "30 + 10 + 8 + 5" into "40 + 13" into "53"). There's no language involved, though in some cases I might use some spatial reasoning by doing something like associating given chunks of an equation with different fingers.
  - leroyrandolph 2y ago
    I don't really follow. Say, during the in-between step "10×3 + 6×3 + 5", how do you store and cognize the individual numeric and operator elements? Surely, even if the arithmetics can be simplified and "lookup-table'd", you are still aware of the numbers in Arabic form or whatever equivalent you're using, right? Or do you somehow have 53 individual blobs swirling inside your consciousness?
    - wizzwizz4 2y ago
      Language (human) and language (theoretical computer science) are different concepts.
    - idiotsecant 2y ago
      Not poster you replied to but it sounds like we might have a similar internal model. I store numbers as pictures of numbers, or a geometric representation depending on how big or precise the number is. Are you saying when you think of the concept of 'twelve plus twelve' you have the equivalent of someone in your head saying 'hmm, well twelve is 2 more than ten, so if I add up ten and ten and two and two I get twenty four?' That's wild if so. For your reference, I would follow the procedure above approximately, but visually with numbers that just do the thing that feels right. I think under the hood we're probably doing the same thing, just with a different interface layer
      - jacomoRodriguez 2y ago
        I can just speak for me, obviously, but yes, that is what's happening. But it's not someone, it is more like me explaining / telling it to myself. Depending on the complexity this can be more or less verbal - the more complex, the less verbal I would say.
- t-3 2y ago
  I speak out loud or write on paper, or just do it a tiny bit slower and sometimes have to redo steps when I forget a result.
- atwrk 2y ago
  It's also very probable that the verbalization the majority does internally is just that - a verbalization of the actual underlying thought process. That is what much of current cognitive linguistics points to as far as I have understood. (Also a reason why I'm very sceptical that the current LLM approach will eventually lead to AGI, BTW)
  - idiotsecant 2y ago
    I think you're probably right that the verbalization is the 'interface layer' but why does that mean LLMs can't approach AGI? They also only use words as an 'interface' layer. Underlying weights are vectors in an abstract space.
    - atwrk 2y ago
      With humans, it has been shown that reasoning processes for different aspects of human behavior are completely distinct from verbalization, they work fully autonomous, and the (inner) verbalization comes afterwards. For LLMs, the tokens (i.e. words) are what the weights are based on, as there isn't other input into them.
RGamma 2y ago
So you cannot think in language? Sounds kinda scary to be honest.
kennysoona 2y ago
You never form thoughts in your mind in a linguistic way? Can you read a sentence and be aware of it as a sentence in your mind, or are you unable to do that? I don't doubt you or anything like that, just very curious. As someone with a very strong internal monologue, it's hard for me to imagine not having one.
- PeterStuer 2y ago
  For me it has linguistics components for sure, but it is many in parallel and a lot less 'linear'. Where inner language most certainly comes into play is in the 'output' phase, be it spoken or written, as serialization is required there, but to be honest that often feels like a projection or even a reconstruction with an inherrent sense of loss as the inner is so much richer and nuanced. That is not to say linearization has no merits. Even if it loses so much it forces consistency and rigor in the lower dimensional reasoning.
  - kennysoona 2y ago
    If you were in a debate, and had to think through your reasoning and arguments, refining your argument, not the words you want to say but the substance of what you will say, would there be an inner voice in that situation?
    - HPsquared 2y ago
      Non-chain of thought models just "come up with" words to say. It's probably a bit like that.
      - TeMPOraL 2y ago
        So do chain-of-thoughts. The "come up with words to say" aka. selecting the best-fitting token to continue the sequence, is more-less exactly how my inner monologue feels. It sits at the boundary of the conscious and the unconscious, and from the point of view of my own introspection, the words just appear, complete with an emotional undertone of whether they "feel right" and how much.
ninetyninenine 2y ago
I do have an internal monologue. I can also think in pictures and I can also think in terms of neither, just pure raw thought. I would say most people are like me. They have 3 modes of thinking and they probably have a primary mode which they favor. I favor none and go into all 3 depending on whether I’m reading, writing or doing something else. The second bigger group has only one primary mode of thinking. The internal monologue. They can only think in terms of an inner voice and this inner voice is so powerful I often encountered people who think this inner voice is the definition of thought. They assumed thinking was COT. The even rarer versions you get people who assign colors to numbers or people who can’t even perceive to think in pictures. You’re the first person I’ve encountered who can’t even have an internal monologue.
- idiotsecant 2y ago
  I think you'd be surprised. I never knew that internal monologues were a thing until there was a HN thread about it and a lot of people had them and a lot of people didn't. I always thought it was something that we did in TV shows or books to give you a sense of what a character was feeling, I didn't know this was an actual literal experience people had. I can certainly have an internal monologue, in the way that you could put on a puppet show. I can conciously think to myself 'self, this is self. Clean your car out' I can form the feeling of those words in my head. But there's nobody 'saying' them if that makes sense. I'm playing back a design of my conscious self.
tdeck 2y ago
There's a fascinating thing called aphantasia where people can't picture things at all in their mind, but such people are able to lead normal lives and may never realize there's something different. This feels like a similar concept but for imaging speech. https://en.m.wikipedia.org/wiki/Aphantasia https://en.m.wikipedia.org/wiki/Aphantasia That said, most of my thinking is not done in the form of a linear monologue where I "talk through" steps to myself.

kla-s 2y ago

Genuine question, how does multi step reasoning work for you then? Like eg if you have some math problem that's trivially to solve individually but needs multiple steps, lets say 16 * 3 + 5? How does 16 * 3 = 48 land in some 'register' of your brain (short term memory), so that you can then add 5 to get to 53? Maybe 16 * 3 + 5 is to easy for you and you'll just 'see' it but the question still stands, just choose a more complex problem. Isn't the same meta process at play when thinking about more fuzzy topics?

cgriswald 2y ago
I believe I have an internal narrator but I’m not certain exactly what others mean by that so I don’t know for sure. However the way I think about math is different than the way I plan my day or other things. In my case, it is very much like I have registers that would hold the result of 16 x 3 in it so I can add the 5 to it later. I have a certain number of registers and with effort like repeating what Ive already solved I could temporarily create more. It also feels somewhat physical, as if the register is an actual box or has a “location” or like I’ve put the answer down on the desk like a part of something I’m building. Perhaps not coincidentally I am one of the many people who have a “calendar shape” for the months.
crooked-v 2y ago
Not that poster, but for me it's directly manipulating numbers (for example, "16×3 + 5" turns into "10×3 + 6×3 + 5" into "30 + 18 + 5" into "30 + 10 + 8 + 5" into "40 + 13" into "53"). There's no language involved, though in some cases I might use some spatial reasoning by doing something like associating given chunks of an equation with different fingers.
- leroyrandolph 2y ago
  I don't really follow. Say, during the in-between step "10×3 + 6×3 + 5", how do you store and cognize the individual numeric and operator elements? Surely, even if the arithmetics can be simplified and "lookup-table'd", you are still aware of the numbers in Arabic form or whatever equivalent you're using, right? Or do you somehow have 53 individual blobs swirling inside your consciousness?
  - wizzwizz4 2y ago
    Language (human) and language (theoretical computer science) are different concepts.
  - idiotsecant 2y ago
    Not poster you replied to but it sounds like we might have a similar internal model. I store numbers as pictures of numbers, or a geometric representation depending on how big or precise the number is. Are you saying when you think of the concept of 'twelve plus twelve' you have the equivalent of someone in your head saying 'hmm, well twelve is 2 more than ten, so if I add up ten and ten and two and two I get twenty four?' That's wild if so. For your reference, I would follow the procedure above approximately, but visually with numbers that just do the thing that feels right. I think under the hood we're probably doing the same thing, just with a different interface layer
    - jacomoRodriguez 2y ago
      I can just speak for me, obviously, but yes, that is what's happening. But it's not someone, it is more like me explaining / telling it to myself. Depending on the complexity this can be more or less verbal - the more complex, the less verbal I would say.
t-3 2y ago
I speak out loud or write on paper, or just do it a tiny bit slower and sometimes have to redo steps when I forget a result.
atwrk 2y ago
It's also very probable that the verbalization the majority does internally is just that - a verbalization of the actual underlying thought process. That is what much of current cognitive linguistics points to as far as I have understood. (Also a reason why I'm very sceptical that the current LLM approach will eventually lead to AGI, BTW)
- idiotsecant 2y ago
  I think you're probably right that the verbalization is the 'interface layer' but why does that mean LLMs can't approach AGI? They also only use words as an 'interface' layer. Underlying weights are vectors in an abstract space.
  - atwrk 2y ago
    With humans, it has been shown that reasoning processes for different aspects of human behavior are completely distinct from verbalization, they work fully autonomous, and the (inner) verbalization comes afterwards. For LLMs, the tokens (i.e. words) are what the weights are based on, as there isn't other input into them.

RGamma 2y ago

So you cannot think in language? Sounds kinda scary to be honest.

kennysoona 2y ago

You never form thoughts in your mind in a linguistic way? Can you read a sentence and be aware of it as a sentence in your mind, or are you unable to do that? I don't doubt you or anything like that, just very curious. As someone with a very strong internal monologue, it's hard for me to imagine not having one.

PeterStuer 2y ago
For me it has linguistics components for sure, but it is many in parallel and a lot less 'linear'. Where inner language most certainly comes into play is in the 'output' phase, be it spoken or written, as serialization is required there, but to be honest that often feels like a projection or even a reconstruction with an inherrent sense of loss as the inner is so much richer and nuanced. That is not to say linearization has no merits. Even if it loses so much it forces consistency and rigor in the lower dimensional reasoning.
- kennysoona 2y ago
  If you were in a debate, and had to think through your reasoning and arguments, refining your argument, not the words you want to say but the substance of what you will say, would there be an inner voice in that situation?
  - HPsquared 2y ago
    Non-chain of thought models just "come up with" words to say. It's probably a bit like that.
    - TeMPOraL 2y ago
      So do chain-of-thoughts. The "come up with words to say" aka. selecting the best-fitting token to continue the sequence, is more-less exactly how my inner monologue feels. It sits at the boundary of the conscious and the unconscious, and from the point of view of my own introspection, the words just appear, complete with an emotional undertone of whether they "feel right" and how much.

ninetyninenine 2y ago

I do have an internal monologue. I can also think in pictures and I can also think in terms of neither, just pure raw thought. I would say most people are like me. They have 3 modes of thinking and they probably have a primary mode which they favor. I favor none and go into all 3 depending on whether I’m reading, writing or doing something else. The second bigger group has only one primary mode of thinking. The internal monologue. They can only think in terms of an inner voice and this inner voice is so powerful I often encountered people who think this inner voice is the definition of thought. They assumed thinking was COT. The even rarer versions you get people who assign colors to numbers or people who can’t even perceive to think in pictures. You’re the first person I’ve encountered who can’t even have an internal monologue.

idiotsecant 2y ago
I think you'd be surprised. I never knew that internal monologues were a thing until there was a HN thread about it and a lot of people had them and a lot of people didn't. I always thought it was something that we did in TV shows or books to give you a sense of what a character was feeling, I didn't know this was an actual literal experience people had. I can certainly have an internal monologue, in the way that you could put on a puppet show. I can conciously think to myself 'self, this is self. Clean your car out' I can form the feeling of those words in my head. But there's nobody 'saying' them if that makes sense. I'm playing back a design of my conscious self.

tdeck 2y ago

There's a fascinating thing called aphantasia where people can't picture things at all in their mind, but such people are able to lead normal lives and may never realize there's something different. This feels like a similar concept but for imaging speech. https://en.m.wikipedia.org/wiki/Aphantasia https://en.m.wikipedia.org/wiki/Aphantasia That said, most of my thinking is not done in the form of a linear monologue where I "talk through" steps to myself.