8 ms·
>what the model is "thinking" before writing its response An actual "thinking machine" would be constantly running computations on its accumulated experience i
by optimalsolver 2y ago
>what the model is "thinking" before writing its response
An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output and/or further compress its sensory history.
An LLM is doing exactly nothing while waiting for the next prompt.
- fassssst 2y agoWhy does the timing of the “thinking” matter?
- verdverm 2y agothinking is generally considered an internal process, without input/output (of tokens), though some people decide to output some of that thinking into a more permanent form I see thinking as less about "timing" and more about a "process" What this post seems to be describing is more about where attention is paid and what neurons fire for various stimuli
- sabrina_ramonov 2y agowe know so little about thinking and consciousness, these claims seem premature
- verdverm 2y agoThat one can fix the RNG and get consistent output indicates a lack of dynamics They certainly do not self update the weights in an online process as needed information is experienced
- whimsicalism 2y agoIf we could perfectly simulate the brain and there were quantum hidden variables, we too could “fix RNG and get deterministic output”
- byteknight 2y agoI disagree with this. That suggests that thinking requires persistent, malleable and non-static memory. That is not the case. You can reasonably reason about without increasing knowledge if you have a base set of logic. I think the thing you were looking for was more along the lines of a persistent autonomous agent.
- soma8088 2y agoLLMs can reasonably reason, however they differ in that once an output begins to be generated, it must continue along the same base set of logic. Correct me if I'm wrong, but I do not believe it can stop and think to itself that there is something wrong with the output and that it should start over at the beginning or backup to a previous state before it outputted something incorrect. Once its output begins to hallucinate it has no choice but continue down the same path since its next token is also based on previous tokens it has just outputted
- HarHarVeryFunny 2y agoSure you can reason over a fixed "base set of logic", although there's another word for that - an expert system with a fixed set of rules, which IMO is really the right way to view an LLM. Still, what current LLMs are doing with their fixed rules is only a very limited form of reasoning since they just use a fixed N-steps of rule application to generate each word. People are looking to techniques such "group of experts" prompting to improve reasoning - step-wise generate multiple responses then evaluate them and proceed to next step.
- whimsicalism 2y agoif you zoom in enough, all thinking is an expert system with a fixed set of rules.
- byteknight 2y agoExactly. You can't reason with that you do not currently posses.
- HarHarVeryFunny 2y agoRight, it's not doing anything between prompts, but each prompt is fed through each of the transformer layers (I think it was 96 layers for GPT-3) in turn, so we can think of this as a fixed N-steps of "thought" (analyzing prompt in hierarchical fashion) to generate each token.
- whimsicalism 2y agoIf we figured out how to freeze and then revive brains, would that mean that all of the revived brains were no longer thinking because they had previously been paused at some point? Frankly this objection seems very weak
- verdverm 2y agoThere are many more features that would be needed, such as a peer comment pointed out, being able to recognize you are saying something incorrect, pausing, and then starting a new stream of output. This is currently done with multiple LLMs and calls, not within the running of a single model i/o Another example would be to input a single token or gibberish, the models we have today are more than happy to spit out fantastic numbers of tokens. They really only stop because we look for stop words they are trained to generate and we do the actual stopping action
- whimsicalism 2y agoi don’t see why any of the things you’re describing are criteria for thinking, it seems just arbitrarily picking things humans do and saying this is somehow constitutive to thought
- verdverm 2y agoIt's more to point out how far the LLMs we have today are from anything that ought to be considered thoughts. They are far more mechanical than anything else
- whimsicalism 2y agoyou’re just retreating into tautologies - my question was why these are the criteria for thought it’s fine though, this was as productive as i expected
- verdverm 2y ago
- viking123 2y agoI might be a complete brainlet so excuse my take, but when animals think and do things, the weights in the brain are constantly being adjusted, old connections pruned out and new ones made right? But once LLM is trained, that's kind of it? Nothing there changes when we discuss with it. As far as I understand from what I read, even our memories are just somehow in the connections between the neurons
- whimsicalism 2y agomy understanding was that once you are of age, brain pruning and malleability is relatively small