4 ms·
I don't quite understand the benefit of mixed precision. It seems like using high precision is useful for training, but if not training, why not just use float
by WatchDog 4y ago
I don't quite understand the benefit of mixed precision.
It seems like using high precision is useful for training, but if not training, why not just use float16 weights and save the memory?
- NavinF 4y agoConverting weights to float16 after training will reduce quality/accuracy whereas mixed precision has a negligible effect on quality/accuracy and dramatically improves performance. If you really just want to save memory, there's plenty of other low hanging fruit. It's just not a priority for most devs since mid tier GPUs start at 10GB whereas a typical model only has 0.5GB weights. Activations and intermediate calculations use way more memory.
- zone411 4y agoYou usually can. But it can take some work if you're using any libraries that expect FP32 and it might be slower, depending on the GPU. The FP16 support isn't quite as good as FP32.