2 ms·
To the previous poster's point, soft distributions are useful, even saving the top 10 logits is significantly more training signal than just the final token.
by rao-v 5mo ago
To the previous poster's point, soft distributions are useful, even saving the top 10 logits is significantly more training signal than just the final token.