Simply scaling to larger models, but retaining naive next token prediction, won’t be enough to get to reliable reasoning on truly novel tasks. Any gains will b

Simply scaling to larger models, but retaining naive next token prediction, won’t be enough to get to reliable reasoning on truly novel tasks. Any gains will be reasoning by analogy to a larger number of reasoning examples. But I don’t think any of the big closed source models are relying entirely on next token prediction anymore. They are using reinforcement learning to add new (more complex) objectives to the training. This might allow for better reasoning abilities within the same architecture.