4 ms·

Yeah I think that's basically what I'm saying. The concept is an elephant. You look at the elephant and know immediately what it is and all it's attributes, but

by dools 4y ago

Yeah I think that's basically what I'm saying. The concept is an elephant. You look at the elephant and know immediately what it is and all it's attributes, but if you were to describe that elephant to someone you'd have to go through all sorts of decisions about what to say and how to say it and you'd change your mind, and you'd say in one instance that the ears were flat, then you'd say no not really flat in the sense that they're level but very thin and floppy. And the person would ask questions and you'd have to reply to them. The difference of course is that the concepts can change and merge with each other, but what I'm referring to as an "instantaneous concept" that precedes linguistic expression is like an instantaneous graphical image of an elephant, that one must then put into words. I think that a LLM doesn't have that. I remember Kasparov had some criticism of Deep Blue that it wasn't really "playing chess" (although I can't find it now). In the same way I think that LLMs aren't really conveying a concept, they're only doing the part that assembles the language. The concepts are all pre-existing in the same way that all the chess move possibilities pre-exist in the programming of a chess computer. EDIT: In fact even the way I described concepts as "pre-existing" there might be giving it too much credit. Like the concept only arises on the receiving end, from a pool of concepts that were pre-existing but about which the LLM has no "awareness" whatsoever.

TeMPOraL 4y ago
> The concept is an elephant. You look at the elephant and know immediately what it is and all it's attributes, but if you were to describe that elephant to someone you'd have to go through all sorts of decisions (...) The question is, do you know its attributes before you describe them to yourself? To me, at least, an "elephant" feels like a pointer, a handle, a promise - I know it's a thing, but to know more than that, I have to dereference the pointer. And that usually means something that's getting close to stringing words together. Possible confounder: I suffer from aphantasia[0]. I have nearly zero ability to perceive images in my mind, except when dreaming. Maybe it's easier to "know immediately what it [the elephant] is and all it's attributes", if it just pops into your head as a complete image. I, unfortunately, don't experience that. > The difference of course is that the concepts can change and merge with each other, but what I'm referring to as an "instantaneous concept" that precedes linguistic expression (...) > I think that a LLM doesn't have that. Crazy idea: what if a concept is... a partial set of coordinates in the latent space? Enough to describe a pretty narrow region. > I remember Kasparov had some criticism of Deep Blue that it wasn't really "playing chess" (although I can't find it now). In the same way I think that LLMs aren't really conveying a concept, they're only doing the part that assembles the language. My tentative guess is that language will turn out to not be the fundamental part of what makes LLMs so powerful/spectacular. I think the magic is the latent space itself - the high-dimensional constructs where all kinds of associations end up represented as spatial distance. Language is just a serialization format[1] for latent space associations. By being fed lots and lots of text in training, LLMs build up the latent space, piece by piece, until there's enough dimensions to it that it is no longer a simple mapping between words learned from the training data. I think this is how understanding and concepts look like. Also, I think it takes so much input text for LLMs to get to their current performance, because LLMs learn like babies - they soak up everything they can, letting the inputs self-arrange by correlations. We learn to understand what an elephant is by hearing stories about it, seeing it alive, and/or being told facts about it in a structured way (i.e. school). LLMs learn what elephant is by soaking up so much text that (the token representing) the word "elephant" is, in the latent space, close to all the words people used to describe its attributes - which themselves are close to other words with whatever various meanings and aspects we relate to those attributes, etc. EDIT: And connecting the two parts of the comment: LLMs are AI's poster child of aphantasia. If they have concepts, how else could they work with them if not by... articulating them? And when they articulate a concept, what makes us so sure they're just parroting associations, and not truly having a concept? -- [0] - https://en.wikipedia.org/wiki/Aphantasia https://en.wikipedia.org/wiki/Aphantasia. [1] - Quite literally, as communicating via any language happens via ingesting or producing a series of some kind of tokens (letters, words, ideographs, etc.) over time.

by dools 4y ago

TeMPOraL 4y ago

> The concept is an elephant. You look at the elephant and know immediately what it is and all it's attributes, but if you were to describe that elephant to someone you'd have to go through all sorts of decisions (...) The question is, do you know its attributes before you describe them to yourself? To me, at least, an "elephant" feels like a pointer, a handle, a promise - I know it's a thing, but to know more than that, I have to dereference the pointer. And that usually means something that's getting close to stringing words together. Possible confounder: I suffer from aphantasia[0]. I have nearly zero ability to perceive images in my mind, except when dreaming. Maybe it's easier to "know immediately what it [the elephant] is and all it's attributes", if it just pops into your head as a complete image. I, unfortunately, don't experience that. > The difference of course is that the concepts can change and merge with each other, but what I'm referring to as an "instantaneous concept" that precedes linguistic expression (...) > I think that a LLM doesn't have that. Crazy idea: what if a concept is... a partial set of coordinates in the latent space? Enough to describe a pretty narrow region. > I remember Kasparov had some criticism of Deep Blue that it wasn't really "playing chess" (although I can't find it now). In the same way I think that LLMs aren't really conveying a concept, they're only doing the part that assembles the language. My tentative guess is that language will turn out to not be the fundamental part of what makes LLMs so powerful/spectacular. I think the magic is the latent space itself - the high-dimensional constructs where all kinds of associations end up represented as spatial distance. Language is just a serialization format[1] for latent space associations. By being fed lots and lots of text in training, LLMs build up the latent space, piece by piece, until there's enough dimensions to it that it is no longer a simple mapping between words learned from the training data. I think this is how understanding and concepts look like. Also, I think it takes so much input text for LLMs to get to their current performance, because LLMs learn like babies - they soak up everything they can, letting the inputs self-arrange by correlations. We learn to understand what an elephant is by hearing stories about it, seeing it alive, and/or being told facts about it in a structured way (i.e. school). LLMs learn what elephant is by soaking up so much text that (the token representing) the word "elephant" is, in the latent space, close to all the words people used to describe its attributes - which themselves are close to other words with whatever various meanings and aspects we relate to those attributes, etc. EDIT: And connecting the two parts of the comment: LLMs are AI's poster child of aphantasia. If they have concepts, how else could they work with them if not by... articulating them? And when they articulate a concept, what makes us so sure they're just parroting associations, and not truly having a concept? -- [0] - https://en.wikipedia.org/wiki/Aphantasia https://en.wikipedia.org/wiki/Aphantasia. [1] - Quite literally, as communicating via any language happens via ingesting or producing a series of some kind of tokens (letters, words, ideographs, etc.) over time.