4 ms·
But having the same "seed" doesn't guarantee the same response from an LLM, hence the question above.
by thomasfromcdnjs 2y ago
But having the same "seed" doesn't guarantee the same response from an LLM, hence the question above.
- wavemode 2y agoI fail to understand how an LLM could produce two different responses from the same seed. Same seed implies all random numbers generated will be the same. So where is the source of nondeterminism?
- furyofantares 2y agoI believe people are confused because ChatGPT's API exposes a seed parameter which is not guaranteed to be deterministic. But that's due to the possibility model configuration changes on the service end and not relevant here.
- dragonwriter 2y agoBarring subtle incompatibilities in underlying implementations on different environments, it does, assuming all other generation settings (temperature, etc.) are held constant.