4 ms·

There is no reason that you can't create both OpenCL (i.e. SPIR-V) and PTX from the same source. Thats what I do for [DCompute](https://github.com/libmir/dcompu

There is no reason that you can't create both OpenCL (i.e. SPIR-V) and PTX from the same source. Thats what I do for [DCompute](https://github.com/libmir/dcompute https://github.com/libmir/dcompute). Agnosticism at the library level/CPU side is next on my list.

roel_v 9y ago
The problem isn't getting it to run, the problem is getting similar performance. You need self-tweaking algos to get some level of performance portability using just opencl, let alone across two apis.
- nicwilson 9y ago
  Indeed, there is a very large amount of parameterisation you can do in D with ease. You can, if you have prior knowledge of the hardware that it will run on, create different versions of the kernels to exploit that (e.g. with template kernels, which we support directly unlike OpenCL C++). Do you mean adaptive algorithms or dynamic recompilation? And yes I do expect that cross API will be difficult, for both running and getting good perf. But it is not just the room at the high performance end of the spectrum that is important, but also the lower end that is stifled by the barrier to entry that would benefit from the extra compute.
  - roel_v 9y ago
    I meant dynamic (re)compilation yes - like adapting the stride in your kernel when you vary work group size for OpenCL kernels. I mean, I have something like that (a primitive self-tuning pre-run calibration step) and I only vary work group-, local- and vector size; and that's already a massive pain in the ass. It's ok on toy kernels but once you move beyond that, it just seeps into all your kernel code, making it almost into a meta-language. And then I'm not even talking about differences like using image types vs arrays for data that is 2d by nature. I don't see how I can generalize that; I just write various versions of my kernels. Which doesn't scale very much, to put it mildly. My point is - was drawn to OpenCL for its 'portability' claim, and yes kernels will 'run' on various hardware, but with massive differences in speed. what good does that portability do me? My workloads (scientific computing, branch heavy) are different from the typical ML half- or single precision MulDiv()/linear algebra applications, so all the hand-tuned CUDA libraries aren't even my concern. The elephant in the room is that performance doesn't depend on this API or that; it's in how you tune your algorithm to the actual hardware you're running on. Which is the direct opposite of portability. To come back to the post I was replying to - yes it's 'trivial' (I mean, a lot of work, but technically not hard, not to belittle your work) to compile almost any statically typed code into either SPIR-V or PTX or any other future format for that matter. But that doesn't mean that it will work 'well' (not even 'optimal') on other hardware. In the real world, you're almost always better off just spending a few hundred to get the same GPU as whoever wrote the kernels tested them on (or if you're running your kernels on existing clusters, to focus on optimizing them for what you know you'll be running them on). Oh and all of that is just considering GPU's. I mean, when you read an OpenCL book they make it seem (in the first few chapters) that you don't even have to think about whether you'll be running on a CPU or a GPU. And then you accidentally use USE_HOST_PTR instead of COPY_HOST_PTR (or the other way around) in the wrong place, and all of a sudden your code is 10x slower than the sequential version of your algo even. What I'm saying is - I no longer believe in easy to use abstractions for these purposes. If you want speed, you code to the metal, and/or you tweak your abstractions to your specific use case. Yes this is a lot of work. And if you don't need speed, you just throw in a few std::thread's here and there and call it a day. But that's just my conclusion for my use cases.
    - nicwilson 9y ago
      Thanks for you insight. I agree that it is not easy to 'parameterise the metal', but it is certainly doable in D[1], in C++ (guessed from the std::thread) good lord no: D's is orders of magnitude ahead of C+'s meta-programming. Writing different versions of a kernel is a poor mans parameterisation ;) LDC, the LLVM D Compiler, will be getting a dynamic re-optimisation, which I hope to get to play nicely with DCompute. Well PTX, because SPIR-V doesn't yet have a jit backend. Dynamic re-optimisation from what I understand is freeze some variables as constants and rerun the optimisation passes. This is as opposed to recompilation with complete restructuring of the kernel. Not quite the same but for things like loop counts and branch "prediction" this should help a lot. w.r.t USE_HOST_PTR/COPY_HOST_PTR, that's not a part of the kernel parameterisation, that's part of the host and is easily adjustable. Yes you need to figure our which one to use, but that's just part of tuning. > What I'm saying is - I no longer believe in easy to use abstractions for these purposes. I hope to be able to show you otherwise :) [1](https://github.com/libmir/mir-glas#porting-to-a-new-target https://github.com/libmir/mir-glas#porting-to-a-new-target)

roel_v 9y ago

The problem isn't getting it to run, the problem is getting similar performance. You need self-tweaking algos to get some level of performance portability using just opencl, let alone across two apis.

nicwilson 9y ago
Indeed, there is a very large amount of parameterisation you can do in D with ease. You can, if you have prior knowledge of the hardware that it will run on, create different versions of the kernels to exploit that (e.g. with template kernels, which we support directly unlike OpenCL C++). Do you mean adaptive algorithms or dynamic recompilation? And yes I do expect that cross API will be difficult, for both running and getting good perf. But it is not just the room at the high performance end of the spectrum that is important, but also the lower end that is stifled by the barrier to entry that would benefit from the extra compute.
- roel_v 9y ago
  I meant dynamic (re)compilation yes - like adapting the stride in your kernel when you vary work group size for OpenCL kernels. I mean, I have something like that (a primitive self-tuning pre-run calibration step) and I only vary work group-, local- and vector size; and that's already a massive pain in the ass. It's ok on toy kernels but once you move beyond that, it just seeps into all your kernel code, making it almost into a meta-language. And then I'm not even talking about differences like using image types vs arrays for data that is 2d by nature. I don't see how I can generalize that; I just write various versions of my kernels. Which doesn't scale very much, to put it mildly. My point is - was drawn to OpenCL for its 'portability' claim, and yes kernels will 'run' on various hardware, but with massive differences in speed. what good does that portability do me? My workloads (scientific computing, branch heavy) are different from the typical ML half- or single precision MulDiv()/linear algebra applications, so all the hand-tuned CUDA libraries aren't even my concern. The elephant in the room is that performance doesn't depend on this API or that; it's in how you tune your algorithm to the actual hardware you're running on. Which is the direct opposite of portability. To come back to the post I was replying to - yes it's 'trivial' (I mean, a lot of work, but technically not hard, not to belittle your work) to compile almost any statically typed code into either SPIR-V or PTX or any other future format for that matter. But that doesn't mean that it will work 'well' (not even 'optimal') on other hardware. In the real world, you're almost always better off just spending a few hundred to get the same GPU as whoever wrote the kernels tested them on (or if you're running your kernels on existing clusters, to focus on optimizing them for what you know you'll be running them on). Oh and all of that is just considering GPU's. I mean, when you read an OpenCL book they make it seem (in the first few chapters) that you don't even have to think about whether you'll be running on a CPU or a GPU. And then you accidentally use USE_HOST_PTR instead of COPY_HOST_PTR (or the other way around) in the wrong place, and all of a sudden your code is 10x slower than the sequential version of your algo even. What I'm saying is - I no longer believe in easy to use abstractions for these purposes. If you want speed, you code to the metal, and/or you tweak your abstractions to your specific use case. Yes this is a lot of work. And if you don't need speed, you just throw in a few std::thread's here and there and call it a day. But that's just my conclusion for my use cases.
  - nicwilson 9y ago
    Thanks for you insight. I agree that it is not easy to 'parameterise the metal', but it is certainly doable in D[1], in C++ (guessed from the std::thread) good lord no: D's is orders of magnitude ahead of C+'s meta-programming. Writing different versions of a kernel is a poor mans parameterisation ;) LDC, the LLVM D Compiler, will be getting a dynamic re-optimisation, which I hope to get to play nicely with DCompute. Well PTX, because SPIR-V doesn't yet have a jit backend. Dynamic re-optimisation from what I understand is freeze some variables as constants and rerun the optimisation passes. This is as opposed to recompilation with complete restructuring of the kernel. Not quite the same but for things like loop counts and branch "prediction" this should help a lot. w.r.t USE_HOST_PTR/COPY_HOST_PTR, that's not a part of the kernel parameterisation, that's part of the host and is easily adjustable. Yes you need to figure our which one to use, but that's just part of tuning. > What I'm saying is - I no longer believe in easy to use abstractions for these purposes. I hope to be able to show you otherwise :) [1](https://github.com/libmir/mir-glas#porting-to-a-new-target https://github.com/libmir/mir-glas#porting-to-a-new-target)