The whole debate is that when you decode the weights (i.e. "run AI on some prompt") you do in fact get training code reproduced verbatim. The fact that we do no

The whole debate is that when you decode the weights (i.e. "run AI on some prompt") you do in fact get training code reproduced verbatim. The fact that we do not have tools to analyze this decoding function analytically is orthogonal.