The obvious point that you're missing is that there are literally infinite ways to assemble software systems from the pieces that an LLM is able to manipulate d

The obvious point that you're missing is that there are literally infinite ways to assemble software systems from the pieces that an LLM is able to manipulate due to its training. With minor guidance, LLMs can put together an unlimited number of novel combinations. The idea that the entire end product has to be in the training set is trivially false.