I believe there is a meaningful difference between an answer that's wrong and an answer that's nonsense. If I were to multiply those numbers, it's likely that

I believe there is a meaningful difference between an answer that's wrong and an answer that's nonsense. If I were to multiply those numbers, it's likely that I'd get the wrong outcome because it's a long computation and at every step I have a nonzero chance of making a mistake. My solution, written out, would look like the correct algorithm, but with computational mistakes along the way. My result could be quite far off, but it would be in roughly the same order of magnitude. If I'd get an answer that's not roughly in the order of magnitude that I expect, I would spot it and -- if so motivated -- start over. If you'd look at my work, you would be able to conclude that I understand multiplication but made a computational error. ChatGPT also diligently describes its work, and it's just nonsense. The final result is smaller than either of the factors. The algorithm it uses makes no sense. On smaller numbers, the algorithm also doesn't make sense, but it can ballpark the outcome. Therefore, it seems to be mostly using estimation rather than computation to get to the answer, and that estimation breaks down completely for very large numbers.