4 ms·
It's very strange that they don't guarantee determinism. We trained a model at Codeium that does have determinism at inference time - it's not super hard if you
by varunkmohan 4y ago
It's very strange that they don't guarantee determinism. We trained a model at Codeium that does have determinism at inference time - it's not super hard if you don't just use random seeds everywhere.
This makes it a lot harder to rely on OpenAI's API if you are doing any sort of caching / evaluation workloads.
- xyzzy123 4y agoNot promising determinism gives them a lot more implementation wiggle room to update / change things. My guess is that promising it right now is not a priority. It would be a great "pro" or "enterprise" type feature tho.
- swyx 4y agoPytorch/CUDA is apparently sometimes nondeterministic https://twitter.com/taliaringer/status/1511411984398487564?s=46&t=L75UJ1fe78VCiBPUK6PddA https://twitter.com/taliaringer/status/1511411984398487564?s...
- varunkmohan 4y agoInteresting, I guess we implemented this op properly :) It is strange that cub does not guarantee determinism here - it's not too hard an op to implement in CUDA.