3 ms·
Actually, the key point is that they aren't using perplexity. They built a chatbot that aims to maximize "SSA" (Sensibleness and Specificity Average), and then
by bkanber 7y ago
Actually, the key point is that they aren't using perplexity. They built a chatbot that aims to maximize "SSA" (Sensibleness and Specificity Average), and then found that SSA is correlated with perplexity.
- phreeza 7y ago"The training objective is to minimize perplexity, the uncertainty of predicting the next toke"" SSA is used as an evaluation metric. It is generated by human raters, and as such is not differentiable, making I hard to directly optimize for with current methods.