buneskamin

3 ms·

1. 1.▲
   by buneskamin 2y ago

   Yea it does seem pretty clear that it's mainly Terrance's contributions to the context window that is bringing the model to the right answer
2. 2.▲
   by buneskamin 2y ago

   Seriously? those are pretty simple HS math questions, I find it a bit hard to believe most average people cant solve them. Don't most people graduate HS?
3. 3.▲
   by buneskamin 2y ago

   I think he's saying if you set temp to 0 and answers become deterministic, it will appear that the model is just memorising and reciting. The randomness is a hack that 'forces' the model to generalise by deliberately knocking