It seems clear that the AI is not being trained on logic, physics or chemistry. Yet it's read and internalized (whatever that means) those subjects somehow. It

It seems clear that the AI is not being trained on logic, physics or chemistry. Yet it's read and internalized (whatever that means) those subjects somehow. It does seem simplistic to say "AI isn't programmed with logic or physics" when it's likely read several textbooks on those subjects. It might be expected to gain something from that, similar to any other intelligence that read those texts. Maybe the confusion stems from "what the AI was programmed to do". It certainly wasn't programmed to do anything but make a model of text and maybe deeper structure of that text (meaning) using an enormous corpus. But when we're exercising an AI, we're not exercising that programming. Not at all. We're exercising the model it made. And for that reason, the AI-model may indeed have some 'understanding' of physics and chemistry, though it had never been 'programmed' to know that. All our previous instincts about computers is, they do what we've programmed them to do. When we try to apply that to AI, we go pretty far off the mark. Because the computer is perhaps the least interesting part of the AI. It's that model we should be concerned about.