Are LLMs / AI attaining better results than the data they were trained on? For me, the answer is no: LLMs are always imperfectly modeling the underlying distrib

Are LLMs / AI attaining better results than the data they were trained on? For me, the answer is no: LLMs are always imperfectly modeling the underlying distribution of the training dataset. Do we have sufficient data that spans the entire problem space that SWE deals with? Probably not, and even if we did it would still be imperfectly modeled. Do we have sufficient data to span the space of many routine tasks in SWE? It seems so, and this is where the LLMs are really nice: e.g., scripting, regurgitating examples, etc. So to me, much like previous innovation, it will just shift job focus away from the things the innovation can do well, rather than replacing the field as a whole. One pet theory I have is that we currently suck at assessing model performance. Sure, vibes-based analysis of the outputs of the model make them look amazing. But is that not the literal point of RLHF? But how good are these outputs really?