3 ms·
I did large scale tests temp 0 and there was still randomness with the same prompt inputs coming in. I did this with several model apis. GPU processing is not
by johnsmith1840 3mo ago
I did large scale tests temp 0 and there was still randomness with the same prompt inputs coming in.
I did this with several model apis.
GPU processing is not going to be the same from what I read but also the AI backend is doing a lot of fancy batching resulting in another layer of randomness.