3 ms·
Yes, probably. At temperature zero the model will be completely deterministic, so a particular prompt will always produce the same result (ignoring for a second
by DougBTX 2y ago
Yes, probably. At temperature zero the model will be completely deterministic, so a particular prompt will always produce the same result (ignoring for a second that some fairly common optimisations introduce data races in the GPU).
On the other hand, does it really matter? With a slight tweak to the prompt, ChatGPT generates some serviceable code:
> Run a function to produce a random number between 1 and 10. What is the number?
import random
# Generate a random number between 1 and 10
random_number = random.randint(1, 10)
random_number
The random number generated between 1 and 10 is 9.
- nextaccountic 2y ago> (ignoring for a second that some fairly common optimisations introduce data races in the GPU). Okay so are any GPU compilers intentionally introducing data races in programs that previously exhibited no data races?
- DougBTX 2y agoNot really compilers, but the underlying GPU libraries. Here’s a good jumping off point: https://pytorch.org/docs/stable/generated/torch.use_deterministic_algorithms.html https://pytorch.org/docs/stable/generated/torch.use_determin...