3 ms·
Indeed, they are next token predictors, but this is a vacuous statement because the predictor can be arbitrary complex.
by gpderetta 1y ago
Indeed, they are next token predictors, but this is a vacuous statement because the predictor can be arbitrary complex.
- HarHarVeryFunny 1y agoSure, but a complex predictor is still a predictor. It would be a BAD predictor if everything it output was not based on "what would the training data say?". If you ask it to innovate and come up with something not in it's training data, what do you think it will do .... it'll "look at" it's training data and regurgitate (predict) something labelled as innovative You can put a reasoning cap on a predictor, but it's still a predictor.
- deleted 1y ago[deleted]