> if the temperature is fixed at a value where the top logit is always picked, then because they weights are fixed, the exact same input should produce the exac

> if the temperature is fixed at a value where the top logit is always picked, then because they weights are fixed, the exact same input should produce the exact same output. Am I missing something? Yes. Your input is part of a batch, and you don't know where in the batch it is. By default batches are not invariant and VLLM only supports invariance at all on some Huwaei Ascend hardware. See https://docs.vllm.ai/projects/ascend/en/latest/user_guide/feature_guide/batch_invariance.html https://docs.vllm.ai/projects/ascend/en/latest/user_guide/fe...