5 ms·
It doesn't really describe the fundamental difference between a GPU and a 4000-core CPU, which is that the GPU has a shared program counter. All the cores must
by IshKebab 8y ago
It doesn't really describe the fundamental difference between a GPU and a 4000-core CPU, which is that the GPU has a shared program counter. All the cores must execute the same instruction at each cycle.
- deleted 8y ago[deleted]
- twtw 8y agoThe conceptually independent threads are executed on 32 wide SIMD cores. "All the cores" within a warp/wavefront must execute the same instruction each cycle. GPU threads got even more independent in nvidia'a volta architecture - search Volta ITS for details.
- why_only_15 8y agoAre you sure? I'm pretty sure I can branch on core-specific measures like (in CUDA) TheadIdx and BlockIdx
- gpuhacker 8y agoIt's true that the programming model allows that, but underneath all threads within the warp will execute the same instructions. However if there is a branch, some threads can be predicated so their instructions have no effect. This is called warp divergence and can become a performance issue. If possible branch only on threadidx using multiples of the warp size. There's a cool slide deck on implementing a parallel sum algorithm that explains this really well.
- dialecticDolt 8y agoDo you have any idea what the source of the slide deck could have been? It sounds very interesting and I'd love to see it.
- gpuhacker 8y agoSure! I meant this one: https://developer.download.nvidia.com/assets/cuda/files/reduction.pdf https://developer.download.nvidia.com/assets/cuda/files/redu...
- 0-_-0 8y agoYes but can you do that from Julia without writing CUDA/OpenCL code? I'm happy that Julia supports GPU programming for simple code, but I don't see how you can run algorithms with inter-thread communication.
- simondanisch 8y agoThat's how you do it: https://github.com/JuliaGPU/GPUArrays.jl/blob/master/src/linalg.jl#L29 https://github.com/JuliaGPU/GPUArrays.jl/blob/master/src/lin... Or directly with cuda shfl intrinsics: https://github.com/JuliaGPU/CUDAnative.jl/blob/b249dfd145501632695cc6e03cdf98e49f5c8708/examples/reduce/reduce.jl#L22 https://github.com/JuliaGPU/CUDAnative.jl/blob/b249dfd145501...
- 0-_-0 8y agoOh cool! Thanks!
- byt143 8y agoYes you can.
- llukas 8y agoOn NVIDIA Volta architecture threads have individual PC.