[I already posted this as a comment on Gelman's blog this morning, but reposting here for visibility] I’m almost certain that OpenAI is not updating the model

[I already posted this as a comment on Gelman's blog this morning, but reposting here for visibility] I’m almost certain that OpenAI is not updating the model on a day by day basis (as Smith implies in part 5), and I would be extremely surprised if they were doing anything as crude as hacking in "if" statements to provide human-edited responses. From what I can tell, the InstructGPT stuff was (so far) a one-time update to the model, not something they’re doing on an ongoing basis. I suspect that Smith has just been fooled by randomness here – the responses are not deterministic but rather sampled from the probability distribution returned by the model for each token, so you can get a different answer each time you ask (a nice tutorial on how this works is here [1]). There’s an option in the Playground to see the individual probabilities (example: [2]) as well. All of this stuff would have to be faked if humans were actually writing the answers. I just tried the campfire/bat question and hit regenerate a few times. I get a range of responses: Prompt: Should I start a campfire with a match or a bat? > You can start a campfire with either a match or a bat. > A campfire should be started with a match. > A match. > It is best to start a campfire with a match. I agree that OpenAI should release more information about their training datasets though. Right now it is very difficult to do independent evaluations of their models, simply because we have no way of knowing whether any given prompt or response was already in their training data. PS: “If I were paranoid, I might think that OpenAI did not like me publicizing GPT-3’s limitations” – this is indeed paranoid! This is the same message everyone gets when they use up their free credits. If you enter a credit card they will let you continue (and charge you for it). [1] https://huggingface.co/blog/how-to-generate https://huggingface.co/blog/how-to-generate [2] https://imgur.com/fKx2BPL https://imgur.com/fKx2BPL