3 ms·

That’s not how it feels to me. The way it feels to me is that I have a fully formed concept almost immediately upon hearing or reading something then I employ

by dools 4y ago

That’s not how it feels to me. The way it feels to me is that I have a fully formed concept almost immediately upon hearing or reading something then I employ language to try and convey the same concept into to someone else’s brain. During that process additional ideas may occur to me, and that may augment the final output.

TeMPOraL 4y ago
Do concepts always come to you fully formed? I experience some of what you describe, too, when dealing with ideas I already am familiar with, or explaining them to others. But when I'm trying to work through some novel thoughts, or am engaging in a conversation - including the one that has me writing this comment - I function in the way that's peculiarly similar to LLMs. That is, what pops into my head are words, phrases, sometimes entire sentences (often movie or videogame quotes). I pick one that fits best as an extension of what was already said - the previous words, the previous sentences, the response from the other party - and then repeat with the next completion, and the next, etc., until I feel it's time to stop. I've joked for many years that, based on my personal experience and introspection, human brains at the highest level may just be glorified Markov chains. I sometimes still say that about LLMs - but I do so more as a compliment now. GPT-3 and GPT-4 may be just fancy autocomplete, but so are we. I feel we've brute-forced our way into a turning point in our understanding of what intelligence is and how it works.
- dools 4y ago
  The concept is always fully formed (or at least SOME concept is fully formed) before I start typing. Like I read your comment. I had thoughts during that process, but I read until the end, then I made a decision about what I thought, and that decision contained all the information that I want to convey in my response. Now: when I am constructing my response, I am of course modulating which words might come next that would best fit into my language model to convey that concept, but the concept is an instantaneous understanding. That instantaneous understanding may change during the process of articulation, but not fundamentally -- I think that any changes during the process of articulation could be considered "elaboration" rather than a reconceptualisation. All the language that I just typed was designed to convey a concept that was more or less instantaneously formed and fully complete prior to my typing anything. The way in which I construct the language to articulate that concept may be very similar to the way in which a LLM constructs its responses, but I think the core difference is that original and instantaneous conceptual understanding of what it is trying to articulate.
  - withinboredom 4y ago
    I’m a mix between you two, I think. I know what thought I want to convey, before writing, but not the exact words I want to use. Even as I’m writing, I’m pausing for a few moments to figure out the best word for the thought, determining if it will lead to the next thought, how you might accept it, and then writing it down. And so on. Only moments before I write the word, do I know what that word will be, but I know the complete thought I want to convey. I don’t know if this is important, but my thoughts are not words, so I’m basically just describing them using words.
    - dools 4y ago
      Yeah I think that's basically what I'm saying. The concept is an elephant. You look at the elephant and know immediately what it is and all it's attributes, but if you were to describe that elephant to someone you'd have to go through all sorts of decisions about what to say and how to say it and you'd change your mind, and you'd say in one instance that the ears were flat, then you'd say no not really flat in the sense that they're level but very thin and floppy. And the person would ask questions and you'd have to reply to them. The difference of course is that the concepts can change and merge with each other, but what I'm referring to as an "instantaneous concept" that precedes linguistic expression is like an instantaneous graphical image of an elephant, that one must then put into words. I think that a LLM doesn't have that. I remember Kasparov had some criticism of Deep Blue that it wasn't really "playing chess" (although I can't find it now). In the same way I think that LLMs aren't really conveying a concept, they're only doing the part that assembles the language. The concepts are all pre-existing in the same way that all the chess move possibilities pre-exist in the programming of a chess computer. EDIT: In fact even the way I described concepts as "pre-existing" there might be giving it too much credit. Like the concept only arises on the receiving end, from a pool of concepts that were pre-existing but about which the LLM has no "awareness" whatsoever.
      - TeMPOraL 4y ago
        > The concept is an elephant. You look at the elephant and know immediately what it is and all it's attributes, but if you were to describe that elephant to someone you'd have to go through all sorts of decisions (...) The question is, do you know its attributes before you describe them to yourself? To me, at least, an "elephant" feels like a pointer, a handle, a promise - I know it's a thing, but to know more than that, I have to dereference the pointer. And that usually means something that's getting close to stringing words together. Possible confounder: I suffer from aphantasia[0]. I have nearly zero ability to perceive images in my mind, except when dreaming. Maybe it's easier to "know immediately what it [the elephant] is and all it's attributes", if it just pops into your head as a complete image. I, unfortunately, don't experience that. > The difference of course is that the concepts can change and merge with each other, but what I'm referring to as an "instantaneous concept" that precedes linguistic expression (...) > I think that a LLM doesn't have that. Crazy idea: what if a concept is... a partial set of coordinates in the latent space? Enough to describe a pretty narrow region. > I remember Kasparov had some criticism of Deep Blue that it wasn't really "playing chess" (although I can't find it now). In the same way I think that LLMs aren't really conveying a concept, they're only doing the part that assembles the language. My tentative guess is that language will turn out to not be the fundamental part of what makes LLMs so powerful/spectacular. I think the magic is the latent space itself - the high-dimensional constructs where all kinds of associations end up represented as spatial distance. Language is just a serialization format[1] for latent space associations. By being fed lots and lots of text in training, LLMs build up the latent space, piece by piece, until there's enough dimensions to it that it is no longer a simple mapping between words learned from the training data. I think this is how understanding and concepts look like. Also, I think it takes so much input text for LLMs to get to their current performance, because LLMs learn like babies - they soak up everything they can, letting the inputs self-arrange by correlations. We learn to understand what an elephant is by hearing stories about it, seeing it alive, and/or being told facts about it in a structured way (i.e. school). LLMs learn what elephant is by soaking up so much text that (the token representing) the word "elephant" is, in the latent space, close to all the words people used to describe its attributes - which themselves are close to other words with whatever various meanings and aspects we relate to those attributes, etc. EDIT: And connecting the two parts of the comment: LLMs are AI's poster child of aphantasia. If they have concepts, how else could they work with them if not by... articulating them? And when they articulate a concept, what makes us so sure they're just parroting associations, and not truly having a concept? -- [0] - https://en.wikipedia.org/wiki/Aphantasia https://en.wikipedia.org/wiki/Aphantasia. [1] - Quite literally, as communicating via any language happens via ingesting or producing a series of some kind of tokens (letters, words, ideographs, etc.) over time.

by dools 4y ago

TeMPOraL 4y ago

Do concepts always come to you fully formed? I experience some of what you describe, too, when dealing with ideas I already am familiar with, or explaining them to others. But when I'm trying to work through some novel thoughts, or am engaging in a conversation - including the one that has me writing this comment - I function in the way that's peculiarly similar to LLMs. That is, what pops into my head are words, phrases, sometimes entire sentences (often movie or videogame quotes). I pick one that fits best as an extension of what was already said - the previous words, the previous sentences, the response from the other party - and then repeat with the next completion, and the next, etc., until I feel it's time to stop. I've joked for many years that, based on my personal experience and introspection, human brains at the highest level may just be glorified Markov chains. I sometimes still say that about LLMs - but I do so more as a compliment now. GPT-3 and GPT-4 may be just fancy autocomplete, but so are we. I feel we've brute-forced our way into a turning point in our understanding of what intelligence is and how it works.

dools 4y ago
The concept is always fully formed (or at least SOME concept is fully formed) before I start typing. Like I read your comment. I had thoughts during that process, but I read until the end, then I made a decision about what I thought, and that decision contained all the information that I want to convey in my response. Now: when I am constructing my response, I am of course modulating which words might come next that would best fit into my language model to convey that concept, but the concept is an instantaneous understanding. That instantaneous understanding may change during the process of articulation, but not fundamentally -- I think that any changes during the process of articulation could be considered "elaboration" rather than a reconceptualisation. All the language that I just typed was designed to convey a concept that was more or less instantaneously formed and fully complete prior to my typing anything. The way in which I construct the language to articulate that concept may be very similar to the way in which a LLM constructs its responses, but I think the core difference is that original and instantaneous conceptual understanding of what it is trying to articulate.
- withinboredom 4y ago
  I’m a mix between you two, I think. I know what thought I want to convey, before writing, but not the exact words I want to use. Even as I’m writing, I’m pausing for a few moments to figure out the best word for the thought, determining if it will lead to the next thought, how you might accept it, and then writing it down. And so on. Only moments before I write the word, do I know what that word will be, but I know the complete thought I want to convey. I don’t know if this is important, but my thoughts are not words, so I’m basically just describing them using words.
  - dools 4y ago
    Yeah I think that's basically what I'm saying. The concept is an elephant. You look at the elephant and know immediately what it is and all it's attributes, but if you were to describe that elephant to someone you'd have to go through all sorts of decisions about what to say and how to say it and you'd change your mind, and you'd say in one instance that the ears were flat, then you'd say no not really flat in the sense that they're level but very thin and floppy. And the person would ask questions and you'd have to reply to them. The difference of course is that the concepts can change and merge with each other, but what I'm referring to as an "instantaneous concept" that precedes linguistic expression is like an instantaneous graphical image of an elephant, that one must then put into words. I think that a LLM doesn't have that. I remember Kasparov had some criticism of Deep Blue that it wasn't really "playing chess" (although I can't find it now). In the same way I think that LLMs aren't really conveying a concept, they're only doing the part that assembles the language. The concepts are all pre-existing in the same way that all the chess move possibilities pre-exist in the programming of a chess computer. EDIT: In fact even the way I described concepts as "pre-existing" there might be giving it too much credit. Like the concept only arises on the receiving end, from a pool of concepts that were pre-existing but about which the LLM has no "awareness" whatsoever.
    - TeMPOraL 4y ago
      > The concept is an elephant. You look at the elephant and know immediately what it is and all it's attributes, but if you were to describe that elephant to someone you'd have to go through all sorts of decisions (...) The question is, do you know its attributes before you describe them to yourself? To me, at least, an "elephant" feels like a pointer, a handle, a promise - I know it's a thing, but to know more than that, I have to dereference the pointer. And that usually means something that's getting close to stringing words together. Possible confounder: I suffer from aphantasia[0]. I have nearly zero ability to perceive images in my mind, except when dreaming. Maybe it's easier to "know immediately what it [the elephant] is and all it's attributes", if it just pops into your head as a complete image. I, unfortunately, don't experience that. > The difference of course is that the concepts can change and merge with each other, but what I'm referring to as an "instantaneous concept" that precedes linguistic expression (...) > I think that a LLM doesn't have that. Crazy idea: what if a concept is... a partial set of coordinates in the latent space? Enough to describe a pretty narrow region. > I remember Kasparov had some criticism of Deep Blue that it wasn't really "playing chess" (although I can't find it now). In the same way I think that LLMs aren't really conveying a concept, they're only doing the part that assembles the language. My tentative guess is that language will turn out to not be the fundamental part of what makes LLMs so powerful/spectacular. I think the magic is the latent space itself - the high-dimensional constructs where all kinds of associations end up represented as spatial distance. Language is just a serialization format[1] for latent space associations. By being fed lots and lots of text in training, LLMs build up the latent space, piece by piece, until there's enough dimensions to it that it is no longer a simple mapping between words learned from the training data. I think this is how understanding and concepts look like. Also, I think it takes so much input text for LLMs to get to their current performance, because LLMs learn like babies - they soak up everything they can, letting the inputs self-arrange by correlations. We learn to understand what an elephant is by hearing stories about it, seeing it alive, and/or being told facts about it in a structured way (i.e. school). LLMs learn what elephant is by soaking up so much text that (the token representing) the word "elephant" is, in the latent space, close to all the words people used to describe its attributes - which themselves are close to other words with whatever various meanings and aspects we relate to those attributes, etc. EDIT: And connecting the two parts of the comment: LLMs are AI's poster child of aphantasia. If they have concepts, how else could they work with them if not by... articulating them? And when they articulate a concept, what makes us so sure they're just parroting associations, and not truly having a concept? -- [0] - https://en.wikipedia.org/wiki/Aphantasia https://en.wikipedia.org/wiki/Aphantasia. [1] - Quite literally, as communicating via any language happens via ingesting or producing a series of some kind of tokens (letters, words, ideographs, etc.) over time.