3 ms·

Here is an example: NN-512 uses pthreads. It generates a tiny, standalone, highly scalable C99 work-stealing thread pool in every C file. The work items are si

Here is an example: NN-512 uses pthreads. It generates a tiny, standalone, highly scalable C99 work-stealing thread pool in every C file. The work items are simply coordinates, similar to the threadIdx/blockIdx coordinates in Nvidia's CUDA. For example, all the structs and functions starting with "Example32Threader" in this file: https://nn-512.com/example/32#3 https://nn-512.com/example/32#3 The implementation ends at Example32ThreaderDo1 The key is to avoid writing to shared cache lines whenever possible, and to keep a bitmap in the central struct (the "hub") that avoids rediscovery of the fact that a thread has no remaining work to be stolen (thereby avoiding a "thundering herd" type problem: https://en.m.wikipedia.org/wiki/Thundering_herd_problem https://en.m.wikipedia.org/wiki/Thundering_herd_problem) Another key is to never take any short-cuts in synchronization. Rather than doing something subtle/clever to avoid mutexes (etc.), instead make sure the units of work are big enough so that synchronization costs are negligible with respect to the overall computation

gary_0 6y ago
> Rather than doing something subtle/clever to avoid mutexes (etc.), instead make sure the units of work are big enough so that synchronization costs are negligible with respect to the overall computation This seems to be working well in my current project, even though I originally planned to refactor things to be "cleverer". A momentary Big Stupid Mutex lock followed by a good chunk of independent work is a nicely boring design for worker threads that seems to work fine. Another trick I previously used was just leaving work involving contended resources aside until after all the worker threads finish, instead of trying to resolve contention with waits or more complex code. If the leftovers from each thread are <1% of the total work, it can just be cleaned up single-threaded style later. Sometimes it's enough to just detect potential contention and leave the trickier work to simpler code.

gary_0 6y ago

> Rather than doing something subtle/clever to avoid mutexes (etc.), instead make sure the units of work are big enough so that synchronization costs are negligible with respect to the overall computation This seems to be working well in my current project, even though I originally planned to refactor things to be "cleverer". A momentary Big Stupid Mutex lock followed by a good chunk of independent work is a nicely boring design for worker threads that seems to work fine. Another trick I previously used was just leaving work involving contended resources aside until after all the worker threads finish, instead of trying to resolve contention with waits or more complex code. If the leftovers from each thread are <1% of the total work, it can just be cleaned up single-threaded style later. Sometimes it's enough to just detect potential contention and leave the trickier work to simpler code.