Not being able to "do math" is an obvious, IMO uninteresting limitation of how LLMs inherently work, and the more advanced models have already figured out good

Not being able to "do math" is an obvious, IMO uninteresting limitation of how LLMs inherently work, and the more advanced models have already figured out good solutions. E.g. while an LLM may stumble on "How many R's are in Strawberry", at this point any code model can easily correctly implement "write me a program in Python that counts the number of R's in strawberry". GPT 3 would essentially always fail on higher math problems/arithmetic, but whenever I've used it for math GPT 4 has always delegated out to executing code where necessary.