LLMs can be run in a mostly deterministic mode (see https://docs.pytorch.org/docs/stable/notes/randomness.html https://docs.pytorch.org/docs/stable/notes/random

LLMs can be run in a mostly deterministic mode (see https://docs.pytorch.org/docs/stable/notes/randomness.html https://docs.pytorch.org/docs/stable/notes/randomness.html for some info on running PyTorch programs). Varying the deployment type (chip model, number of chips, batch size, ...) can also change the output due to rounding errors. See https://arxiv.org/abs/2506.09501 https://arxiv.org/abs/2506.09501 for some details on that.