I wonder how much this is GPT 4 actually inferring the details about what's on the screen vs GPT 4 recognizing the "shape" of Blender, getting the task (which i

I wonder how much this is GPT 4 actually inferring the details about what's on the screen vs GPT 4 recognizing the "shape" of Blender, getting the task (which is simple and has most likely a myriad tutorials online), and just proceeding step by step. Basically, whether just asking GPT 4 the same question with "in blender" in text mode didn't result in the same effect. Ideally, could be tested with something that's less straightforward, and requires understanding the data presented in some windows on the screen. Like "how can I fix this error"?