I don't know what is true either, but you can't run that analysis for the claim people are making. The reason being that people aren't claiming the API model

I don't know what is true either, but you can't run that analysis for the claim people are making. The reason being that people aren't claiming the API model outputs changed, they're claiming the website based outputs changed. Most claim it's probably due to some alteration in the 'pre-prompt', the hyperparameters (temp, top k instead of typical sampling, etc) or something to that effect. But you can't really test that so easily with a non deterministic temp.