4 ms·
Nondeterminism strikes again! But yes, I would expect GPT-4 to get this right most of the time.
by phillipcarter 3y ago
Nondeterminism strikes again!
But yes, I would expect GPT-4 to get this right most of the time.
- tgv 3y agoSaying "Sorry, I was non-deterministic" to your teacher won't do much for your grade.
- phillipcarter 3y agoSure, but what is your point? This is about evaluating LLM outputs, not grade school.
- pixl97 3y agoIt may turn out that one of the key factors in human intelligence and success is probabilistic nondeterminism. That is most of the time we'll come up with close to the same answer, but possibly worded different (which can lead to other interpretations of our answer), but occasionally our answers are so widely different they lead to escape of the local maxima.
- jameshart 3y agoThe reason we use averages of performance over multiple tests and papers as a way to grade human students is precisely because we know that human performance is nondeterministic.