3 ms·
> if nobody's surprised, then nobody has learned anything Really? You're saying that as long as you assume something is true, there's no value in finding out i
by digging 2y ago
> if nobody's surprised, then nobody has learned anything
Really? You're saying that as long as you assume something is true, there's no value in finding out if it's actually true or not?
- hamburga 2y agoI was taught in biology that a good scientific experiment is one in which you learn something whether or not the null hypothesis is confirmed. I am equating learning to surprise, though you could disagree with semantics.
- digging 2y agoYes, it is an enormous mistake to equate learning with surprise. I'd ask you to consider answering my above question directly, as I think it will resolve this issue.
- hamburga 2y agoI agree with you, of course, that we should test our assumptions empirically as a general point. However, there isn't time to test out every single assumption we could generally have. Therefore, the more worthwhile experiments are ones where we learn something interesting no matter what happens. I'm equating this with "surprise," as in, we have done some meaningful gradient descent or Bayesian update, we've changed our views, we know something that wasn't obvious before. You could disagree with semantics there, but hopefully we agree with the idea of more vs. less valuable experiments. I'm just not sure whose model of LLM dynamics was updated by this paper. Then again, I only listened to a couple minutes of their linked YouTube discussion before getting bored.