3 ms·

>But if you care enough about performance that you focus on the performance benefits of SIMD over ordinary execution, you probably should be using the GPU inste

>But if you care enough about performance that you focus on the performance benefits of SIMD over ordinary execution, you probably should be using the GPU instead. That's not true. There's a lot of heavily serial algorithms which do not always work well on the GPU unless they are of a very large size, larger than most applications. This happens a lot in optimization and differential equations, two of the pillars of scientific computing. >Thus, a much fairer comparison, if we care so much about performance that non-SIMD execution is a non-starter, would be CUDA vs. Julia targeting the GPU. Tim already wrote that article where it shows how Julia code generation can be faster in some instances for GPUs. https://devblogs.nvidia.com/gpu-computing-julia-programming-language/ https://devblogs.nvidia.com/gpu-computing-julia-programming-... >but idiomatic Julia code still has low overhead Where?

ummonk 8y ago
>That's not true. There's a lot of heavily serial algorithms which do not always work well on the GPU unless they are of a very large size, larger than most applications. This happens a lot in optimization and differential equations, two of the pillars of scientific computing. And those situations tend to get a very minimal speedup from SIMD. >Tim already wrote that article where it shows how Julia code generation can be faster in some instances for GPUs. https://devblogs.nvidia.com/gpu-computing-julia-programming-.. https://devblogs.nvidia.com/gpu-computing-julia-programming-.... Yes, that's great, and would have made for an interesting challenge, rather than focusing on SIMD instead of serial or GPU performance. >Where? I didn't word it well, but I meant to say that Julia doesn't have much overhead compared to C/C++, unlike Python (what overhead it does have comes from garbage collection).

ummonk 8y ago

>That's not true. There's a lot of heavily serial algorithms which do not always work well on the GPU unless they are of a very large size, larger than most applications. This happens a lot in optimization and differential equations, two of the pillars of scientific computing. And those situations tend to get a very minimal speedup from SIMD. >Tim already wrote that article where it shows how Julia code generation can be faster in some instances for GPUs. https://devblogs.nvidia.com/gpu-computing-julia-programming-.. https://devblogs.nvidia.com/gpu-computing-julia-programming-.... Yes, that's great, and would have made for an interesting challenge, rather than focusing on SIMD instead of serial or GPU performance. >Where? I didn't word it well, but I meant to say that Julia doesn't have much overhead compared to C/C++, unlike Python (what overhead it does have comes from garbage collection).