3 ms·
SIMD-compute is a totally different model of compute than what most programmers are familiar with. That's the biggest problem. If you write optimal SIMD-code,
by dragontamer 6y ago
SIMD-compute is a totally different model of compute than what most programmers are familiar with.
That's the biggest problem. If you write optimal SIMD-code, no one else in your team will understand it. Since we have so much compute these days (to the point where O(n^2) scanf parsers are all over the place), its become increasingly obvious that few modern programmers care about performance at all.
Nonetheless, the more and more I study SIMD-compute, the more I realize that these expert programmers have figured out a ton of good and fast solutions to a wide-variety of problems... decades ago and then somehow forgotten until recently.
Seriously: that Data Parallel Algorithms paper is just WTF to me. Linked list traversal (scan-reduced sum from a linked list in SIMD-parallel), Regular Expressions and more.
--------
Then I look at the GPU-graphics guys, and they're doing like BVH tree traversal in parallel so that their raytracers work.
Its like "Yeah, Raytracing is clearly a parallel operation cause GPUs can do it". So I look it up and wtf? Its not easy. Someone really thought things through. Its non-obvious how they managed to get a recursive / sequential operation to operate in highly parallel SIMD operations while avoiding branch divergence issues.
Really: think about it: Raytracing is effectively:
If(ray hit object) recursively bounce ray.
How the hell did they make that parallel? A combination of stream-compaction and very intelligent data-structures, as well as a set of new SIMD-assembly instructions to cover some obscure cases.
There's some really intelligent stuff going on in the SIMD-compute world, that clearly applies beyond just the machine-learning crowd.