On the topic of FP- https://dennisforbes.ca/index.php/2017/04/11/floating-point-numbers-an-infinite-number-of-mathematicians-enter-a-bar/ https://dennisforbes.

On the topic of FP- https://dennisforbes.ca/index.php/2017/04/11/floating-point-numbers-an-infinite-number-of-mathematicians-enter-a-bar/ https://dennisforbes.ca/index.php/2017/04/11/floating-point-... Most desktop chips offer little advantage for FP16 (at best that the source memory footprint is smaller, maybe better cache hit rates), but newer GPUs can actually blaze. If it has the magnitude/precision needs for a task, it's a big win.