The only "design" bottleneck is where the main thread has to read processed data from the other threads via a shared mem buffer and add it to any existing value

The only "design" bottleneck is where the main thread has to read processed data from the other threads via a shared mem buffer and add it to any existing value in another shared mem buffer. Reading from the source buffer is sequential but the write to the destination is effectively random.