4 ms·
Right, to express it another way: The real-world LLM just makes documents longer. It is being run against a document which resembles a movie script, where "User
by Terr_ 2y ago
Right, to express it another way: The real-world LLM just makes documents longer. It is being run against a document which resembles a movie script, where "User says X, Computer says" is inserted whenever I type X, and then the LLM just makes it a bit longer to complete the line for the "Computer" character.
These models marketed as "reasoning" are just changing the style of the script to film noir detective, where the protagonist "Computer" has extra observations and commentary that aren't "spoken" to other characters.
While that change may help keep the story on track, it does not affect the fundamental nature of the "thinking" going on: It's still just growing a document and acting out whatever story emerges.
- hnaccount_rng 2y agoThat’s a very good analogy, which works astonishingly well in giving people a reasonable expectation. The question I’m having though is: How different are we from that? The nature of a Markov-Chain is that you can describe just about anything with it. The following description is correct, isn’t it? Our “thoughts” are generated as some function of our integrated past sensory input (scare quotes because I don’t want to talk about what exactly a thought is)
- Terr_ 2y ago> The question I’m having though is: How different are we from that? It's obvious we are different, but if we could answer exactly how with sufficient rigor, we would already have better AI and be asking different questions. :p I readily admit that LLMs are an exciting potential piece of a much bigger puzzle, but this could easily be like trying to parse HTML only using regular expressions: It works great on trivial input, but no amount of minor would let it truly solve the problem, because it lacks some higher structure of organization or meaning.
- hnaccount_rng 2y agoI too thought that it was obvious. But then I spend a lot of time in neuroscience discussions… and… it got less obvious