5 ms·
The question is how you know you’ve found a prompt that will give reliable results, at least for questions in some domain? Although it’s possible such a prompt
by skybrian 6y ago
The question is how you know you’ve found a prompt that will give reliable results, at least for questions in some domain? Although it’s possible such a prompt exists, it’s not clear that anyone has found one, and based on architecture, there is some reason to expect that apparently-promising prompts will prove unreliable if investigated seriously.
It’s fine to do some exploratory experiments to find a good prompt, but this needs to be followed up with a separate, rigorous evaluation phase with a fixed prompt. If you keep changing the prompt then you will never know what you have and can easily be fooled by randomness.