2 ms·

if there were big general-case speedups still possible from auto-vectorization that juice would have been squeezed already, even if it was only possible at runt

by paulmd 4y ago

if there were big general-case speedups still possible from auto-vectorization that juice would have been squeezed already, even if it was only possible at runtime the core would watch for those patterns and pull those onto the units. it's like the old joke about economists: an economist is walking down the street with his friend, the friend says "look, a 20 dollar bill laying on the ground!" and bends to pick it up. But the economist keeps walking, and says "it couldn't be, or someone would already have picked it up!". That joke but with computer architecture. we are inherently talking about a world where that's not possible anymore, that juice has been squeezed. Throwing 10% more silicon at some specific problems and getting 2.5-3x speedup (real-world numbers from SIMD-JSON) is better than throwing 10% more silicon at the general case and getting 1% more performance. If 10-20% of real-world code gets that 3x speedup (read: probably 2x perf/w) that's great, that's much better than the general-case speedups!

fulafel 4y ago
Depends on what abstraction level we are discussing. Existing applications without code changes in mainstream languages with parallelism-restrictive semantics, I agree we're probably close to the limits. Beyond that we know that most apps are amenable to human perf work and reformulation to get very big speedups. Besides human reformulation expressed with only low level parallelism primitives like SIMD instrinsics, the field has been using parallelism geared languages like Cuda, Futhark, ISPC etc. And there's a lot of untapped potential in data representation flexibility etc that even those languages aren't tapping, like for example Halide can do. Human perf work also involves a lot of trial and error, it's a search type process, automation of which hasn't been explored that much. Some work to automating this are approaches like Atlas BLAS.

by paulmd 4y ago

fulafel 4y ago

Depends on what abstraction level we are discussing. Existing applications without code changes in mainstream languages with parallelism-restrictive semantics, I agree we're probably close to the limits. Beyond that we know that most apps are amenable to human perf work and reformulation to get very big speedups. Besides human reformulation expressed with only low level parallelism primitives like SIMD instrinsics, the field has been using parallelism geared languages like Cuda, Futhark, ISPC etc. And there's a lot of untapped potential in data representation flexibility etc that even those languages aren't tapping, like for example Halide can do. Human perf work also involves a lot of trial and error, it's a search type process, automation of which hasn't been explored that much. Some work to automating this are approaches like Atlas BLAS.