> There was a whole bunch of people who claimed LLMs can't reason at all and that everything is a regurgitation. I wonder what they have to say about this. I d

> There was a whole bunch of people who claimed LLMs can't reason at all and that everything is a regurgitation. I wonder what they have to say about this. I don't see that as a refutation of the former actually: model trained to be stochastic parrots with next-token prediction as only learning target were indeed stochastic parrots and now we've moved to a completely different technology that features reinforcement learning in its training so it will go farther and farther from stochastic parrots and more and more towards “intelligence”. If anything, the fact that the entire industry has now moved to RL instead of just cramming through trillions of tokens to make progress is a pretty strong acknowledgement that the “stochastic parrots” crowd was right.