4 ms·
I agree prompts make a huge difference in how well GPT-3 performs. The relevant questions are: 1) how easy is it to replicate best practices? If it's simple re
by allencheng 6y ago
I agree prompts make a huge difference in how well GPT-3 performs. The relevant questions are:
1) how easy is it to replicate best practices? If it's simple reformatting and everyone knows how to do it, then the playing field levels off quickly.
2) what does the improvement curve look like - how far can you push performance through better prompts before you get diminishing marginal returns?
It's too early to tell and we'll need more samples to figure this out.
- deleted 6y ago[deleted]
- deleted 6y ago[deleted]
- mlb_hn 6y agoI don't think it's going to be about a single prompt; reverse engineering multiple prompts interacting with themselves is hard. There's a lot of cool things to be done with: (a) creating a pipeline of prompts that combine outputs of previous prompts into new prompts in a predefined manner and (b) designing prompts to generate other prompts
- goldenkey 6y agoWith the right type of online learning and possibly some of the weights frozen, GPT-3 could gain an unlimited memory instead of the fixed 2048 token memory.