In principle you would expect reasoning to eventually emerge. During training LLMs predict large volumes of text, and that text contains a lot of arguments and

In principle you would expect reasoning to eventually emerge. During training LLMs predict large volumes of text, and that text contains a lot of arguments and chains of thought. You can achieve some success predicting these with pattern matching, but the best way to complete arguments at scale is a reasoning process. But I think we are only scratching the surface of achieving this. Based on my admittedly casual observation of current model capabilities you need a model that's fairly big (closer to 70B than 7B) and trained far beyond what's implied by the Chinchilla metric.