I want to be clear that I still find it impressive, in the same way I find Riffusion impressive. If anything, I'm looking at Norvig's pointing out that "the big

I want to be clear that I still find it impressive, in the same way I find Riffusion impressive. If anything, I'm looking at Norvig's pointing out that "the biggest issue is that there is no explanation of why the code is the way it is, no justification for how it works." The model can't and won't; it's an unreasonable expectation, and I can't tell whether Norvig is asking for it in good faith. If I assume he is, and his proposed suggestions that the model "participate in a conversation that leads to the kind of questions and answers we discussed here, thereby building trust in the program" and "generate documentation or tests that would build trust in the code" are also in good faith, then I maintain that he's still missing a fundamental limitation of these models even as he outlines its shape with great specificity. They literally and demonstrably are incapable of coherently doing what he wants; they can't be trained to engender trust, only to mimic actions that might by generating novel responses based on patterns. That would still not be reasoning through the problem to engineer a solution, it's just an extremely effective, superhuman con of novel mimicry. Which, again, is still really, really impressive, and even potentially useful, but in a different way than we might want or expect it to be, and in a dangerous way to use as a stable foundation for iteration toward AGI.