2 ms·
Are these LLMs deterministic or is this comparison rather useless?
by croes 3y ago
Are these LLMs deterministic or is this comparison rather useless?
- chris_st 3y agoI'm voting pretty useless -- I've dorked around with Llama and CodeLlama a bit, and wow, the differences between runs are pretty substantial.
- sp332 3y agoThe notes at the bottom say all the tests were run with a temperature of 0, so they should be deterministic except for GPT-4 which has other sources of randomness.
- rvz 3y agoThey are all non-deterministic; hence the term of stochastic parrots always referred to LLMs. Thus these tests are close to useless.