Like always this ignores that the space programs live in is not only high dimensional but hyperbolic, meaning the search space for even trivial programs explode

Like always this ignores that the space programs live in is not only high dimensional but hyperbolic, meaning the search space for even trivial programs explodes much faster than you'd expect. Neural networks are best thought of as oracles, simple single purpose tools which can do amazing things impossible with regular software in human time frames, but ultimately limited by the fact there isn't enough bits in the universe to encode enough training data even for medium difficulty tasks. Put another way, all the LLMs have been trained on every book ever written. If we put a human being on an island with every book ever written and effective immortality we'd have someone a lot more interesting than gpt4 when we came back in a few million years or so.