3 ms·
I suggest you spend 20 hours evaluating the results of 10 prompts vs 1 prompt with multiple perspectives to learn the truth about the matter rather than trying
by CuriouslyC 2y ago
I suggest you spend 20 hours evaluating the results of 10 prompts vs 1 prompt with multiple perspectives to learn the truth about the matter rather than trying to armchair expert.
Edit in response to your wall of text: I have *extensively* tested the results of multi-shot prompting vs repeated single shot prompting, and the differences between them are not material to the outcome of "averaging" results, or selecting the best result. You can theorize all you want, but the real world would like a word.
- wokwokwok 2y agoJust click on the 'regenerate result' button a few times and see what happens before you change the prompt. That's all it takes. It's an easy adjustment to workflows that people often either forget to do or don't realise they should be doing. Sorry; I'm not trying to criticize; I'm just telling you that's how it works.
- CuriouslyC 2y agoThat's an early step that matters more when hitting chat with hidden temperature. Once you get a prompt dialed in you usually want to lower the temperature to the minimum value that still produces the desired results.
- dartos 2y agoI think the two of you are arguing different things. OP is saying that you can’t evaluate any prompt by using just one generation using that prompt. You need to run that prompt several times to approximate any prompt’s performance. That’s just how probability works. I don’t believe OP is arguing the effectiveness of running 10 different prompts vs a single multiple perspective prompt.