3 ms·
This seems very finely tuned to the Julia language (i.e. relying heavily on meta-programming). It's not clear to me that strong support for meta-programming is
by ummonk 8y ago
This seems very finely tuned to the Julia language (i.e. relying heavily on meta-programming). It's not clear to me that strong support for meta-programming is actually that necessary for every day use cases.
I'd be curious to see what kinds of implementations someone could come up with in CUDA using templates, though. It could easily blow the Julia performance out of the water.
- ChrisRackauckas 8y agoHow could it blow the Julia performance out of the water? Julia is just manipulating the generated LLVM IR, and LLVM generates the .ptx. If C++ and Julia are both using LLVM for this compilation, you'll get to pretty much the same place. And this is tuned towards package development, specifically generic package development. This is quite a hot topic in scientific computing. The challenge isn't for people who use packages, it's for people who develop packages and are looking to make them generally useful (i.e. not just single project).
- ummonk 8y agoYou can certainly use a GPU package for Julia with macros for generating ptx code. That's not what the OP did though. He merely optimized for using SIMD, which provides Julia a distinct advantage over other languages with strong metaprogramming, such as C++. But if you care enough about performance that you focus on the performance benefits of SIMD over ordinary execution, you probably should be using the GPU instead. By using CUDA (not C++) as the language, one can get far faster performance right out of the box. Thus, a much fairer comparison, if we care so much about performance that non-SIMD execution is a non-starter, would be CUDA vs. Julia targeting the GPU. Note that I absolutely love Julia because I find it far easier to write than a language like C++, but idiomatic Julia code still has low overhead, unlike, say, Python, which introduces orders of magnitude of overhead compared to low level languages, and you have to move away from simple idiomatic code to achieve reasonable performance. However, I find it unreasonable to focus on SIMD performance - the performance gain over non-vectorized code isn't that big, and if you do want to eek out that much extra performance, you should focus on using the GPU.
- ChrisRackauckas 8y ago>But if you care enough about performance that you focus on the performance benefits of SIMD over ordinary execution, you probably should be using the GPU instead. That's not true. There's a lot of heavily serial algorithms which do not always work well on the GPU unless they are of a very large size, larger than most applications. This happens a lot in optimization and differential equations, two of the pillars of scientific computing. >Thus, a much fairer comparison, if we care so much about performance that non-SIMD execution is a non-starter, would be CUDA vs. Julia targeting the GPU. Tim already wrote that article where it shows how Julia code generation can be faster in some instances for GPUs. https://devblogs.nvidia.com/gpu-computing-julia-programming-language/ https://devblogs.nvidia.com/gpu-computing-julia-programming-... >but idiomatic Julia code still has low overhead Where?
- ummonk 8y ago>That's not true. There's a lot of heavily serial algorithms which do not always work well on the GPU unless they are of a very large size, larger than most applications. This happens a lot in optimization and differential equations, two of the pillars of scientific computing. And those situations tend to get a very minimal speedup from SIMD. >Tim already wrote that article where it shows how Julia code generation can be faster in some instances for GPUs. https://devblogs.nvidia.com/gpu-computing-julia-programming-.. https://devblogs.nvidia.com/gpu-computing-julia-programming-.... Yes, that's great, and would have made for an interesting challenge, rather than focusing on SIMD instead of serial or GPU performance. >Where? I didn't word it well, but I meant to say that Julia doesn't have much overhead compared to C/C++, unlike Python (what overhead it does have comes from garbage collection).
- oxinabox 8y agoI'ld also like to see what someone great at using templates could come up with.
- simondanisch 8y agoWell, that'd be comparing apples and oranges... But fun fact: Julia actually has excellent GPU support! I will soon update the challenge with a GPU accelerated version. Spoiler: it won't need much more than 5 lines of code to get it to the GPU. Good luck trying to beat that with pure CUDA C++ ;)