Verifiability makes it easier to understand how well the LLM works, but this doesn't counter my hypothesis. If X number of instances get 99.0% on an objective,

Verifiability makes it easier to understand how well the LLM works, but this doesn't counter my hypothesis. If X number of instances get 99.0% on an objective, verifiable metric, is there any guarantee that 10X will get 99.9%? The fact that we are reliant on new model releases to push capability in big ways, and that people running gigantic clusters of LLMs end up beaten by new models implies that the capabilities of a given model have a hard upper limit, and that it may not even take much to reach it.