I would argue that ChatGPT is actually quite good at "mathematics", in the sense of helping me formulate a problem in an appropriate mathematical structure, and

I would argue that ChatGPT is actually quite good at "mathematics", in the sense of helping me formulate a problem in an appropriate mathematical structure, and coming up with a good sequence of steps to solve / simplify it. Not perfect by any means, but not bad at all. What it is bad at is actually performing the steps accurately, but as others mentioned, that's where Wolfram and/or a code interpreter would come in.