3 ms·
Lockless concurrency was always an "interesting" subject to me, but I never thought it was practical for me to use them. Mostly because multithreaded programmin
by dragontamer 9y ago
Lockless concurrency was always an "interesting" subject to me, but I never thought it was practical for me to use them. Mostly because multithreaded programming generally has easier-to-use and higher-level constructs (ie: Reader-Writer Queues already written) that can be used.
However, I've recently grown to respect Lockless Programming as I have begun to experiment with GPU programming. OpenCL doesn't give you Mutexes, and indeed, you really shouldn't "lock" an instruction pointer in a GPU ever. (On AMD systems, a singular instruction pointer runs 64 different "threads". So if one of these "threads" lock up, ALL 64 of them lock. This "Single Instruction, Multiple Data" is the key to why GPUs have so many "cores")
As such, the only practical way to write decently high-performance code with concurrency on a GPU is through Atomics (ie: Compare and Swap). Which of course, requires an understanding of memory barriers as well.
- blattimwind 9y ago> Single Instruction, Multiple Data *multiple threads (preferably both, in the case of GPUs, but what you described is SIMT)
- djmips 9y agoSome would say SIMT is a marketing term but at the end of the day SIMT is a subset of SIMD
- dragontamer 9y agoI know its what NVidia calls SIMT, but its what AMD calls SIMD. ¯\_(ツ)_/¯ NVidia is somewhat correct to call their "SIMT" concept something new, because Intel's original SIMD implementations (MMX and SSE) couldn't do the thread-divergence thing very easily (AVX 512 adds a few more features to make thread-divergence easier to handle at the CPU level). But as far as I can tell, AMD GPUs can do thread-divergence, "constant broadcasts", and all that jazz too, but AMD still calls it SIMD.