I don't agree that LLMS can't reason, but literally saying "Make up your own math problems and go for it", him doing that and it failing really isn't moving the

I don't agree that LLMS can't reason, but literally saying "Make up your own math problems and go for it", him doing that and it failing really isn't moving the goal posts. LLMs are not good at math. But this is a subset of reasoning. Chain-of-thought on logical inference tasks (using fake labels so we are outside the training set) shows they can do reasonably well at these. Nevertheless, it's likely that the best approach for pure reasoning tasks will be to connect a LLM to a real inference engine (datalog or something) and rely on the LLM to perform the mapping to the inference engine inputs and outputs. This is similar to the "System 1" and "System 2" models of human thought.