2 ms·
Was the temperature 0? Cause unless I don't understand it right, any non-zero temperature implies probabilistic next token prediction. You did mention, seed, w
by Iolaum 3mo ago
Was the temperature 0? Cause unless I don't understand it right, any non-zero temperature implies probabilistic next token prediction.
You did mention, seed, which I haven't seen available anywhere else (in my limited experience) and it can explain a reproducible inference result, but I feel like this matches the "letter of the law" meaning of deterministic rather than the "spirit of the law".
- olmo23 3mo agoIf you send the seed along, you can have a non-zero temperature: when it's time to select tokens, a PRNG will be seeded and used for the selection.
- bnjemian 3mo agoUsing a PRNG and a seed, you can have paeudo-randomness and determinism. The pseudo-randomness is the key piece, it’s random to the extent that it cannot be distinguished from true randomness in polynomial time classically.
- bruce343434 3mo agoPerhaps they just used the exact same random seed everywhere?