4 ms·
The idea that it's modelling cognition at all is wishful thinking. We know what it's doing: predicting the next token given a certain amount of context. There's
by Planktonne 4y ago
The idea that it's modelling cognition at all is wishful thinking. We know what it's doing: predicting the next token given a certain amount of context. There's no reason to assume that it's doing more than that.
Not everyone anticipated that 'predict the next token' would be a good enough method to mimic as much communication as it can, but ML is full of such surprises. With sufficient data, an awful lot of things can be solved in other ways than we used to. We used to think that only birds could fly, but that doesn't make a helicopter a bird, it makes it another way to solve a problem.
- dTal 4y agoYou keep saying "predicting the next token" like it's somehow a small task, but predicting the future by modeling the problem space is fundamentally what intelligence is. "The next token" can be influenced by anything from facts about the world to quirks in the way humans think. For it to perform at anything like the level it does, it must have built models of all those things.