I have no doubt that it's possible to design an algorithm that can produce diagrams in a given style, given a sufficient description. But a GPT is only going to

I have no doubt that it's possible to design an algorithm that can produce diagrams in a given style, given a sufficient description. But a GPT is only going to learn that sort of thing via sufficiently-many examples. This problem is fundamentally different to (e.g.) computer chess, in that we have no algorithmic win condition. You can't brute-force a solution space when your only way of assessing whether candidates are any good is to ask a human. (Especially not when humans use heuristics to assess the quality of something at a glance, and find it tiring to process large quantities of data: that's a simple recipe for specification gaming if ever I saw one.)