3 ms·
Correct me if I'm wrong, but the problem is that it is almost impossible to evaluate sampling methods. You can't just look at perplexity and conclude that A is
by bjourne 10mo ago
Correct me if I'm wrong, but the problem is that it is almost impossible to evaluate sampling methods. You can't just look at perplexity and conclude that A is better than B. So you need large-scale expensive human evaluations. Even if you have those it is difficult to extrapolate results since what sampling method works best depends on the dataset(s).
- programjames 10mo agoI think you can try maximizing the free energy E[reward] + temperature*entropy?
- bjourne 10mo agoHow do you know that generates high quality text?
- programjames 9mo agoIt generalizes better, so it ought to produce higher quality text.