5 ms·
Can someone explain the difference between fp16 and fp32 in these benchmarks because the difference is pretty dramatic. I assume it's floating point precision(?
by bryan0 6y ago
Can someone explain the difference between fp16 and fp32 in these benchmarks because the difference is pretty dramatic. I assume it's floating point precision(?) but why would lower precision be slower relatively on the 3090? For training jobs how does the precision impact accuracy of the model?
Edit: clarified that I am referring to slower relative performance
- smallnamespace 6y agoFP16 is faster (units are images per second)
- bryan0 6y agoSorry I was referring to the relative performance, I edited my question to be clearer
- riku_iki 6y ago> FP16 is faster (units are images per second) But does model get quality hit: need to train for more steps before converging to the similar performance and have more parameters? FM16 obviously contains less information than FP32.
- bitL 6y ago3090 opted for bundling 2x FP32 units Bulldozer-style and now FP16 is processed by those cores as well, so FP16 and FP32 have the same performance (35.58TFlops). https://www.techpowerup.com/gpu-specs/geforce-rtx-3090.c3622 https://www.techpowerup.com/gpu-specs/geforce-rtx-3090.c3622
- bufo 6y agoNvidia nerfed at the software level the FP16 performance to disincentivize people from using this card as a TITAN / datacenter ML card replacement.
- my123 6y agoIt isn't at the software level, FP16 goes through the tensor cores on Turing onwards: https://www.anandtech.com/show/13973/nvidia-gtx-1660-ti-review-feat-evga-xc-gaming/2 https://www.anandtech.com/show/13973/nvidia-gtx-1660-ti-revi...)
- dogma1138 6y agoThe ALUs are capable of half precision regardless of the tensor cores and aren’t restricted. For “tensor ops” in GeForce cards FP16 with FP32 accumulate is done at half rate so you don’t get double the performance which you do get in Quadro and Titan cards using the same die.
- bufo 6y agoSee this thread https://twitter.com/wightmanr/status/1309583916362117120 https://twitter.com/wightmanr/status/1309583916362117120
- Der_Einzige 6y agoFp16 is faster in this article on most models...
- bufo 6y agoThat's because of the improved memory bandwidth. See https://timdettmers.com/2020/09/07/which-gpu-for-deep-learning/#The_Most_Important_GPU_Specs_for_Deep_Learning_Processing_Speed https://timdettmers.com/2020/09/07/which-gpu-for-deep-learni...
- deleted 6y ago[deleted]