I think you would have to do it by hand. Assuming that the prompt is stripped off before the text is sent to the user, there's no obvious way to me that you wou

I think you would have to do it by hand. Assuming that the prompt is stripped off before the text is sent to the user, there's no obvious way to me that you would reverse-engineer it except by having an experienced GPT-3 user use their intuition to think about what kind of prompt would elicit observed responses. Since you don't have access to the GPT-3 model itself, you can't directly use gradient ascent or MCMC to try to reverse it to get the prompt. You might be able to blackbox it, but the only approach that comes to mind is using the logprobs, and your target won't give you those any more than they will give you the prompt itself. I'm not sure what he has in mind; just because model-stealing has been demonstrated for CNN classifiers doesn't mean you can feasibly steal GPT-3 prompts...