4 ms·
The argument isn't that there is something more than next token prediction happening. The argument is that next token prediction does not imply an upper bound
by cjbprime 2y ago
The argument isn't that there is something more than next token prediction happening.
The argument is that next token prediction does not imply an upper bound on intelligence, because an improved next token prediction will pull increasingly more of the world that is described in the training data into itself.
- unoti 2y ago> The argument isn't that there is something more than next token prediction happening. > The argument is that next token prediction does not imply an upper bound on intelligence, because an improved next token prediction will pull increasingly more of the world that is described in the training data into itself. Well said! There's a philosophical rift appearing in the tech community over this issue semi-neatly dividing people between naysayers, "disbelievers" and believers over this very issue.
- nuancebydefault 2y agoI fully agree. Some people fully disagree though on the 'pull of the world' part, let alone 'intelligence' part, which are in fact impossible to define.