3 ms·
The approach I've been using is that I test against a corpus of questions with known answers. Using sample-and-vote, that is asking for some number of variable
by williamcotton 4y ago
The approach I've been using is that I test against a corpus of questions with known answers. Using sample-and-vote, that is asking for some number of variable completions with a non-zero temperature and treating the explicitly computed solutions as votes, has smoothed over issues with basic syntax and logic errors.
This is pretty similar to the approach used in the Toolformer paper, other than sample-and-vote, which I believe is novel.
- triyambakam 4y agoIn this context what does temperature mean?
- williamcotton 4y agoTemperature is an option passed into the LLM. A temperature of 0 means the response is deterministic, that is that it always returns the same completion. A temperature of greater than zero results in increasing variation in the completion.