3 ms·
Even if that's true, they are still not reliable. The same question can produce different answers each time.
by ygritte 1y ago
Even if that's true, they are still not reliable. The same question can produce different answers each time.
- hhh 1y agoThis isn't really true when you control the stack, no? If you have all of your parameters set to be reproducible (e.g. temp 0, same seed), the output should be the same as long as everything further down the stack is the same, no?
- imiric 1y agoThat's not a usable workaround. In most cases it doesn't actually produce full determinism[1]. And even if it did, a certain degree of non-determinism is actually desirable. The most probable tokens might not be correct, and randomness is partly responsible for what humans interpret as "creativity". Even hallucinations are desirable in some applications (art, entertainment, etc.). [1]: https://medium.com/google-cloud/is-a-zero-temperature-deterministic-c4a7faef4d20 https://medium.com/google-cloud/is-a-zero-temperature-determ...
- jowea 1y agoIs that critical? Doesn't it just need to be better than the alternative? Unless it's a safety-critical system.