3 ms·
You may be running into jensen (huang)’s inequality, E(loss).cuda() <= E(loss.cuda())
by QuadmasterXLII 1y ago
You may be running into jensen (huang)’s inequality,
E(loss).cuda() <= E(loss.cuda())
- CaptainOfCoit 1y agoWould make sense I suppose if I was using two different GPUs for the same thing and get two different outcomes. But instead I have two implementations (one naive, one tensor cores) running on the same GPU, but getting different outcomes, where they should be the same. But then this joke might be flying above my head as well.
- p1esk 1y agoTensor cores use lower precision, so small numerical differences should be expected.
- deleted 1y ago[deleted]