3 ms·
I did not know about unnormalized floats. I sometimes wonder about it staring at my GPU when it multiplies matrices.
by standy13 2y ago
I did not know about unnormalized floats. I sometimes wonder about it staring at my GPU when it multiplies matrices.
- ashvardanian 2y agoI've made a bunch of bad measurements, until someone reminded me to: #if defined(__AVX512F__) || defined(__AVX2__) void configure_x86_denormals(void) { _MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON); // Flush results to zero _MM_SET_DENORMALS_ZERO_MODE(_MM_DENORMALS_ZERO_ON); // Treat denormal inputs as zero } #endif It had a 50x performance impact on Intel. As benchmarked on `c7i.metal-48xl` instances: - `f64` throughput grew from 0.2 to 8.2 TFLOPS. - `f32` throughput grew from 0.6 to 15.1 TFLOPS. Here is that section in the repo with more notes AVX-512, AMX, and other instructions: <https://github.com/ashvardanian/less_slow.cpp/blob/8f32d65cc8976ad20c8cdf645caa5b39a1cf488b/less_slow.cpp#L1984-L2004 https://github.com/ashvardanian/less_slow.cpp/blob/8f32d65cc...>.
- cwzwarich 2y agoIntel has always had terrible subnormal performance. It's not that difficult to implement in HW, and even if you still want to optimize for the normalized case, we're talking about a 1 cycle penalty, not an order(s)-of-magnitude penalty.