You probably could but what would you see? A bunch of weights, connections between layers and more numbers. You don't see any meaningful, understandable code.

You probably could but what would you see? A bunch of weights, connections between layers and more numbers. You don't see any meaningful, understandable code. For example, if prompt begins with draw me a picture then jump to layer X. I'm no expert but I can imagine that to be the problem when one attempts to debug an Algorithmic Intelligence black box.