3 ms·
>Temp should be effectively 0 so it's atleast deterministic in it's random probabilities. Is this a thing? I read an article about how due to some implementati
by andai 7mo ago
>Temp should be effectively 0 so it's atleast deterministic in it's random probabilities.
Is this a thing? I read an article about how due to some implementation detail of GPUs, you don't actually get deterministic outputs even with temp 0.
But I don't understand that, and haven't experimented with it myself.
- kingstnap 7mo agoBy default CUDA isn't deterministic because of thread scheduling. The main difference comes from rounding order of reduction difference. It does make a small difference. Unless you have an unstable floating point algorithm, but if you have an unstable floating point algorithm on a GPU at low precision you were doomed from the start.