> Meanwhile, GPT4 will cheerfully write a program to carry out the required arithmetic operations (and then some.) A study that doesn't acknowledge that is wort

> Meanwhile, GPT4 will cheerfully write a program to carry out the required arithmetic operations (and then some.) A study that doesn't acknowledge that is worthless at best. We don't want to see if the LLM can be used as a tool to do arithmetics, but whether it can learn complex data relationships like arithmetics. Arithmetics is a stepping stone, not a goal so that the model can solve it by invoking a calculator isn't relevant. The problems we want it to solve doesn't have tools like calculators so it doesn't help getting us there.