Since ChatGPT doesn't have a central place to do math its ability to know whether 3 is bigger than 2 depends on the context, your examples thus doesn't prove an

Since ChatGPT doesn't have a central place to do math its ability to know whether 3 is bigger than 2 depends on the context, your examples thus doesn't prove anything. Every time I've tried gpt-4 etc there are such problems. I don't use it for trivial things since then what's the point, and for anything non-trivial it screws up too much. It is useful for translations and brainstorming, but not much else. Translating from natural language to code is a part of that, but not letting it code itself.