4 ms·
This part piqued my interest: > GPT-3 seed prompts can be reverse-engineered, which may become a rude awakening for entrepreneurs and the venture capitalists w
by cryptoz 6y ago
This part piqued my interest:
> GPT-3 seed prompts can be reverse-engineered, which may become a rude awakening for entrepreneurs and the venture capitalists who fund them.
Is there any more information about this? Is the reverse engineering done by hand or is this something that can be coded? Super curious about this point.
- gwern 6y agoI think you would have to do it by hand. Assuming that the prompt is stripped off before the text is sent to the user, there's no obvious way to me that you would reverse-engineer it except by having an experienced GPT-3 user use their intuition to think about what kind of prompt would elicit observed responses. Since you don't have access to the GPT-3 model itself, you can't directly use gradient ascent or MCMC to try to reverse it to get the prompt. You might be able to blackbox it, but the only approach that comes to mind is using the logprobs, and your target won't give you those any more than they will give you the prompt itself. I'm not sure what he has in mind; just because model-stealing has been demonstrated for CNN classifiers doesn't mean you can feasibly steal GPT-3 prompts...
- cryptoz 6y agoSuper interesting, thanks for the reply!