The art and science of building these models is not disputed, but I think that the scientific value of prompts is tightly linked to reproducibility. If you’ve

The art and science of building these models is not disputed, but I think that the scientific value of prompts is tightly linked to reproducibility. If you’ve developed a new prompt for a model whose weights you can directly access, then this prompt could have scientific value because its utility will not diminish over time or be erased with a new model update. I’m even generally of the view that a closed API endpoint whose expiration date is years into the future could have some value (but much less so). But simply finding a prompt for something like ChatGPT is not useful for science because we don’t even have certainty about which model it’s executing against. Note that some of the best uses of these models and prompting have nothing to do with academics; this is a comment focused on the idea about writing academic papers about prompts.