I think Lattner was too generous and missed a couple of crucial points in the CCC experiment. He wrote: > CCC shows that AI systems can internalize the textboo

I think Lattner was too generous and missed a couple of crucial points in the CCC experiment. He wrote: > CCC shows that AI systems can internalize the textbook knowledge of a field and apply it coherently at scale. Except that's not what happened. There was neither (just) textbook knowledge nor a "coherent application at scale": 1. The agents relied on thousands of human written tests embodying many person-years of "preparation effort", not to mention a complete spec. Furthermore, their models were also trained not only on the spec (and on the tests) but also on a reference implementation and the agents were given access to the reference implementation as a test oracle. None of that is found in a textbook. 2. Despite the extraordinary effort required to help the agents in this case - something that isn't available for most software - the models ultimately failed to write a workable C compiler, and couldn't converge. They reached a point where any bug fix caused another bug and that's when the people running the agents stopped the experiment. The main issue wasn't that there was nothing innovative in the code but that even after embibing textbooks and relying on an impractical amount of preparation effort of help, the agents couldn't write a workable C compiler (which isn't some humongous task to begin with).