Have you tried running it against a quantized model on HuggingFace with identical inputs and deterministic sampling to check if the outputs you're getting are i

Have you tried running it against a quantized model on HuggingFace with identical inputs and deterministic sampling to check if the outputs you're getting are identical? I think that should confirm/eliminate any concern of the model being evaluated incorrectly.