4 ms·

Your point is especially important as most of the GPGPU performance gains come from really clever use of shared memory. That is the very model of shared state w

Your point is especially important as most of the GPGPU performance gains come from really clever use of shared memory. That is the very model of shared state with concurrent acces. Usually, your shader or kernel is required to synchronize threads that need to see each other's changes. I'm sure that there are at least some kernels out there that skip synchronization and still work because there are enough other instructions between write and read accesses that cover the issue up.

dragontamer 7y ago
> shared memory. I've done enough Hacker News discussions to note that the lay-reader won't understand what this means, and this probably needs more elaboration. "Shared Memory" is a special memory area inside of GPUs where grids (NVidia) or workgroups (AMD)... a group of 32x to 1024x SIMD threads... can perform inter-thread communications in an outrageously fast way. Shared memory is extremely small: roughly 64kB in size. Optimizing shared memory access involves resolving bank-conflicts, and lots of very-low level thought. At a minimum, you need to consider how you fill shared memory (the memcpy in) as well as how to get the final data out (the memcpy out). --------- In many cases, synchronization to-and-from shared memory only requires a __threadfence() instruction. Maybe only a __syncwarp() instruction in some cases. A lot of thought goes into the "ordering" of memory accesses, to make shared-memory as fast as possible. See here for further details: https://developer.nvidia.com/gpugems/GPUGems3/gpugems3_ch39.html https://developer.nvidia.com/gpugems/GPUGems3/gpugems3_ch39.... See "39.2.3 Avoiding Bank Conflicts" for how shared memory is optimized in practice. You're not only thinking about the shared-state, but also the average number of other threads hitting any particular bank. On a say... 32-bank system, you'll want all 32-banks to be utilized as much as possible. If all 1024 threads are accessing bank#0, you'll be 1/32th the speed (and banks #1 through #31 are all wasted). A "bank" is basically the implicitly "RAID0" arrangement of shared-memory. The precise number of banks is somewhere between 16x banks or 32x banks, depending on architecture. But regardless, if all GPU-threads access memory location #0, that will all hit bank#0, which is slow. Instead, you want GPU-threads to "spread out" over the banks (maybe GPU-thread#0 should access Bank0 / Memory location #0. GPU-thread#1 should access bank1 / memory-location #16. GPU thread #2 should access bank2 / memory-location #32. Etc. etc.)
- AndrewGaspar 7y ago
  > I've done enough Hacker News discussions to note that the lay-reader won't understand what this means, and this probably needs more elaboration. This is why I didn't mention it, but I agree that shared memory and typical uses present a particularly challenging problem for Rust.
skohan 7y ago
Yeah it's a little bit difficult to imagine Rust's concurrency primitives being used in the context of GPGPU programming. Like are we talking about using channels to pass state between threads instead of using tools like barriers for synchronization? Seems like it would negate the advantages GPU's are designed to enable.
- steveklabnik 7y ago
  Rust does not force you to use channels, and lets you use stuff like https://doc.rust-lang.org/std/sync/atomic/fn.fence.html https://doc.rust-lang.org/std/sync/atomic/fn.fence.html if you'd like.

dragontamer 7y ago

> shared memory. I've done enough Hacker News discussions to note that the lay-reader won't understand what this means, and this probably needs more elaboration. "Shared Memory" is a special memory area inside of GPUs where grids (NVidia) or workgroups (AMD)... a group of 32x to 1024x SIMD threads... can perform inter-thread communications in an outrageously fast way. Shared memory is extremely small: roughly 64kB in size. Optimizing shared memory access involves resolving bank-conflicts, and lots of very-low level thought. At a minimum, you need to consider how you fill shared memory (the memcpy in) as well as how to get the final data out (the memcpy out). --------- In many cases, synchronization to-and-from shared memory only requires a __threadfence() instruction. Maybe only a __syncwarp() instruction in some cases. A lot of thought goes into the "ordering" of memory accesses, to make shared-memory as fast as possible. See here for further details: https://developer.nvidia.com/gpugems/GPUGems3/gpugems3_ch39.html https://developer.nvidia.com/gpugems/GPUGems3/gpugems3_ch39.... See "39.2.3 Avoiding Bank Conflicts" for how shared memory is optimized in practice. You're not only thinking about the shared-state, but also the average number of other threads hitting any particular bank. On a say... 32-bank system, you'll want all 32-banks to be utilized as much as possible. If all 1024 threads are accessing bank#0, you'll be 1/32th the speed (and banks #1 through #31 are all wasted). A "bank" is basically the implicitly "RAID0" arrangement of shared-memory. The precise number of banks is somewhere between 16x banks or 32x banks, depending on architecture. But regardless, if all GPU-threads access memory location #0, that will all hit bank#0, which is slow. Instead, you want GPU-threads to "spread out" over the banks (maybe GPU-thread#0 should access Bank0 / Memory location #0. GPU-thread#1 should access bank1 / memory-location #16. GPU thread #2 should access bank2 / memory-location #32. Etc. etc.)

AndrewGaspar 7y ago
> I've done enough Hacker News discussions to note that the lay-reader won't understand what this means, and this probably needs more elaboration. This is why I didn't mention it, but I agree that shared memory and typical uses present a particularly challenging problem for Rust.

skohan 7y ago

Yeah it's a little bit difficult to imagine Rust's concurrency primitives being used in the context of GPGPU programming. Like are we talking about using channels to pass state between threads instead of using tools like barriers for synchronization? Seems like it would negate the advantages GPU's are designed to enable.

steveklabnik 7y ago
Rust does not force you to use channels, and lets you use stuff like https://doc.rust-lang.org/std/sync/atomic/fn.fence.html https://doc.rust-lang.org/std/sync/atomic/fn.fence.html if you'd like.