They could even persist the inference parameterization, e.g. "use top-8 for 'compute the sum of 42 and 17'", for every prompt ever seen and reuse it later to pr

They could even persist the inference parameterization, e.g. "use top-8 for 'compute the sum of 42 and 17'", for every prompt ever seen and reuse it later to prevent people doing "poor man's regression testing" and make the model look more stable.