8 ms·
Probably? I mean, if you were to ask me, why I wrote any of my comments the way I did, half of the time the honest answer would be "because it fits" or "it soun
by TeMPOraL 4y ago
Probably? I mean, if you were to ask me, why I wrote any of my comments the way I did, half of the time the honest answer would be "because it fits" or "it sounds right". Surely I'm not the only one who's aware they're speaking and writing mostly by generating completions and picking ones that sound best.
- dools 4y agoThat’s not how it feels to me. The way it feels to me is that I have a fully formed concept almost immediately upon hearing or reading something then I employ language to try and convey the same concept into to someone else’s brain. During that process additional ideas may occur to me, and that may augment the final output.
- TeMPOraL 4y agoDo concepts always come to you fully formed? I experience some of what you describe, too, when dealing with ideas I already am familiar with, or explaining them to others. But when I'm trying to work through some novel thoughts, or am engaging in a conversation - including the one that has me writing this comment - I function in the way that's peculiarly similar to LLMs. That is, what pops into my head are words, phrases, sometimes entire sentences (often movie or videogame quotes). I pick one that fits best as an extension of what was already said - the previous words, the previous sentences, the response from the other party - and then repeat with the next completion, and the next, etc., until I feel it's time to stop. I've joked for many years that, based on my personal experience and introspection, human brains at the highest level may just be glorified Markov chains. I sometimes still say that about LLMs - but I do so more as a compliment now. GPT-3 and GPT-4 may be just fancy autocomplete, but so are we. I feel we've brute-forced our way into a turning point in our understanding of what intelligence is and how it works.
- dools 4y agoThe concept is always fully formed (or at least SOME concept is fully formed) before I start typing. Like I read your comment. I had thoughts during that process, but I read until the end, then I made a decision about what I thought, and that decision contained all the information that I want to convey in my response. Now: when I am constructing my response, I am of course modulating which words might come next that would best fit into my language model to convey that concept, but the concept is an instantaneous understanding. That instantaneous understanding may change during the process of articulation, but not fundamentally -- I think that any changes during the process of articulation could be considered "elaboration" rather than a reconceptualisation. All the language that I just typed was designed to convey a concept that was more or less instantaneously formed and fully complete prior to my typing anything. The way in which I construct the language to articulate that concept may be very similar to the way in which a LLM constructs its responses, but I think the core difference is that original and instantaneous conceptual understanding of what it is trying to articulate.
- withinboredom 4y agoI’m a mix between you two, I think. I know what thought I want to convey, before writing, but not the exact words I want to use. Even as I’m writing, I’m pausing for a few moments to figure out the best word for the thought, determining if it will lead to the next thought, how you might accept it, and then writing it down. And so on. Only moments before I write the word, do I know what that word will be, but I know the complete thought I want to convey. I don’t know if this is important, but my thoughts are not words, so I’m basically just describing them using words.
- dools 4y agoYeah I think that's basically what I'm saying. The concept is an elephant. You look at the elephant and know immediately what it is and all it's attributes, but if you were to describe that elephant to someone you'd have to go through all sorts of decisions about what to say and how to say it and you'd change your mind, and you'd say in one instance that the ears were flat, then you'd say no not really flat in the sense that they're level but very thin and floppy. And the person would ask questions and you'd have to reply to them. The difference of course is that the concepts can change and merge with each other, but what I'm referring to as an "instantaneous concept" that precedes linguistic expression is like an instantaneous graphical image of an elephant, that one must then put into words. I think that a LLM doesn't have that. I remember Kasparov had some criticism of Deep Blue that it wasn't really "playing chess" (although I can't find it now). In the same way I think that LLMs aren't really conveying a concept, they're only doing the part that assembles the language. The concepts are all pre-existing in the same way that all the chess move possibilities pre-exist in the programming of a chess computer. EDIT: In fact even the way I described concepts as "pre-existing" there might be giving it too much credit. Like the concept only arises on the receiving end, from a pool of concepts that were pre-existing but about which the LLM has no "awareness" whatsoever.
- TeMPOraL 4y ago> The concept is an elephant. You look at the elephant and know immediately what it is and all it's attributes, but if you were to describe that elephant to someone you'd have to go through all sorts of decisions (...) The question is, do you know its attributes before you describe them to yourself? To me, at least, an "elephant" feels like a pointer, a handle, a promise - I know it's a thing, but to know more than that, I have to dereference the pointer. And that usually means something that's getting close to stringing words together. Possible confounder: I suffer from aphantasia[0]. I have nearly zero ability to perceive images in my mind, except when dreaming. Maybe it's easier to "know immediately what it [the elephant] is and all it's attributes", if it just pops into your head as a complete image. I, unfortunately, don't experience that. > The difference of course is that the concepts can change and merge with each other, but what I'm referring to as an "instantaneous concept" that precedes linguistic expression (...) > I think that a LLM doesn't have that. Crazy idea: what if a concept is... a partial set of coordinates in the latent space? Enough to describe a pretty narrow region. > I remember Kasparov had some criticism of Deep Blue that it wasn't really "playing chess" (although I can't find it now). In the same way I think that LLMs aren't really conveying a concept, they're only doing the part that assembles the language. My tentative guess is that language will turn out to not be the fundamental part of what makes LLMs so powerful/spectacular. I think the magic is the latent space itself - the high-dimensional constructs where all kinds of associations end up represented as spatial distance. Language is just a serialization format[1] for latent space associations. By being fed lots and lots of text in training, LLMs build up the latent space, piece by piece, until there's enough dimensions to it that it is no longer a simple mapping between words learned from the training data. I think this is how understanding and concepts look like. Also, I think it takes so much input text for LLMs to get to their current performance, because LLMs learn like babies - they soak up everything they can, letting the inputs self-arrange by correlations. We learn to understand what an elephant is by hearing stories about it, seeing it alive, and/or being told facts about it in a structured way (i.e. school). LLMs learn what elephant is by soaking up so much text that (the token representing) the word "elephant" is, in the latent space, close to all the words people used to describe its attributes - which themselves are close to other words with whatever various meanings and aspects we relate to those attributes, etc. EDIT: And connecting the two parts of the comment: LLMs are AI's poster child of aphantasia. If they have concepts, how else could they work with them if not by... articulating them? And when they articulate a concept, what makes us so sure they're just parroting associations, and not truly having a concept? -- [0] - https://en.wikipedia.org/wiki/Aphantasia https://en.wikipedia.org/wiki/Aphantasia. [1] - Quite literally, as communicating via any language happens via ingesting or producing a series of some kind of tokens (letters, words, ideographs, etc.) over time.
- maxbond 4y agoWhat's to stop a Markov chain or very simple language model (something that prompting doesn't work on, like an RNN with a modest hidden state) from meeting this definition? Because I wouldn't buy that either of these are aware, and both of them predict the next token.
- TeMPOraL 4y agoI think it's a matter of degree, not of kind. That is, the complexity of the model predicting the next token matters, and so does the space of possible tokens. Ultimately, I think the definite answer would require solving the halting problem. The approximate answer we should be able to get when we let models interact with some complex environment over time. That is, the model predicts the next token, which may cause some change of a shared state, and then the model receives a response.
- maxbond 4y agoI dunno. Is there a testable prediction of this hypothesis? I just don't really see why I should believe it other than that chatting to LLMs is compelling, which isn't enough for me. This seems like an answer to the question, "why is the LLM so compelling?" And not "what makes something conscious/aware?" Which is an important question but feels like not quite the right one to ask and then come to this conclusion. And it bothers me if I try to accept an idea where I don't know what conditions I'd reject it (eg, I don't know what evidence would cause me to reject this hypothesis, I can't see a test I can perform to next reject it).
- grumbel 4y agoSimple models will very quickly produce content that doesn't fit. Just stringing a bit of grammar and a dictionary together will produce locally correct sentences, but it will completely fail at generate larger texts with context. GPT in contrast knows what it was talking about five sentences ago and can refer to it in newly formed sentences.
- jodrellblank 4y agoGo and write a Markov chain, you need some text, a hashtable and a few minutes, you'll quickly see that they generate rubbish which doesn't fit anywhere, no sense of context, no sense of 'chat', no sense of intelligence to them. ChatGPT 'predicts the next token' so much better than other 'next token predictors' that using that as the comparison for a low effort dismissal is like saying "what's to stop a human tracking a criminal by scent, humans have noses and dogs have noses so they must be the same".