3 ms·
Agree. Token predicting machines will continue to be token predicting machines by nature. Continued size and tuning will have the effect of making them more and
by calinet6 28d ago
Agree. Token predicting machines will continue to be token predicting machines by nature. Continued size and tuning will have the effect of making them more and more perfect at being average.
- breuleux 28d agoNext-token prediction is a general paradigm, though. In principle, there isn't really anything a sufficiently advanced token predictor couldn't do.
- anthonyrstevens 28d agoThis. People treat "token prediction" dismissively, as if it were a limit and not a foundational skill. Human brains do "token prediction" in all sorts of contexts.
- Dlemlo 28d agoThey learned generic concepts like our brain does to optimize for this particular surprisngly perfect task: You have to be able to respond to a very generic question in a way that the other entity thinks this is good, comprehensive, etc. You can call us situation predicting machines as well if you want. But you undermine what the latent space of an LLM is representing.