4 ms·

so what happens if you go ptx -> llvm -> opencl -> nvidia gpu? how does the speed change?

by andrewcooke 17y ago

Raphael_Amiard 17y ago
this shouldn't happen but it would be an interresting comparison nevertheless
- andrewcooke 17y ago
  it seems possible from the diagram on the linked page. why shouldn't it happen? i am curious whether the analysis stages can improve the code.
  - sparky 17y ago
    I see that "OpenCL" is listed above the LLVM box on the Ocelot page, but I'm not sure why; It is known that several OpenCL toolchains (Nvidia, ATI, RapidMind) make use of LLVM, but it is unclear in what capacity they are used. For the sequence you described (PTX->OpenCL->GPU), there would have to be an OpenCL backend for LLVM. As far as I know, no such backend is publically available, and another compiler would be necessary to take the OpenCL source code down to PTX (from whence it came) and then the driver would JIT that PTX for your specific GPU model. > i am curious whether the analysis stages can improve the code. Ihe conventional wisdom is that any series of analyses that takes you from representation X, through one or more other representations, and back to X can only make things worse, assuming that the JIT from PTX to GPU machine code isn't horrendous. This is because any analyses, optimizations, and transformations need to be conservative to maintain correctness, and high-level semantic information about the parallelism inherent in the application is usually lost in each translation step. In this particular case it might not be so bad, as long as the LLVM IR is rich enough to faithfully represent the Cooperative Thread Array (CTA) semantics in PTX and not flatten them to SPMD code. My intuition, however, is that it's not; LLVM was designed as a fairly generic virtual machine that would faithfully represent most CPU-like execution models, and hardware CTAs (also called 'warps' in Nvidia parliance) are mostly a GPU-only phenomenon. CPUs have SIMD units (e.g. SSE, MMX, Altivec, NEON), but the execution model there is fundamentally different than the GPU.
    - jacquesm 17y ago
      Once the GT300 series hits the shelves that problem will be largely mitigated though, they're supposed to be mostly independent, with a 'variable warp' size. Of course that will introduce a new level of complexity to the optimization problem.
      - andrewcooke 17y ago
        i don't understand this either. "wariable warp size" sounds like a small efficiency fix for when things aren't multiples of 32, or when they exceed 512. a "variable warp size" doesn't alter the fundamental problem SIMD approach - you've still got a multiprocessor with slave processors that are doing very similar work. for me, the big advances in fermi are a unified address space and some kind of cache for the global memory. neither of those change the paradigm, but they may make life significantly simpler when programming the thing.
        jacquesm 17y ago
        Multiples of 32 are nice, multiples of 1 are better :) (and let's hope it goes that far down), that would make things a lot easier as well. Unified address space I assume you mean across multiple GPUs ? Global memory cache is a double edged sword, that eats in to the transistor budget at a very rapid pace, effectively you already have a cache, you just have to fill it yourself. GPU programming is definitely a step back in the ease with which you can write programs, but if your problem maps well on to a GPU the speed increases are simply astounding. What would have taken you a cluster with 100 boxes now sits under your desk and consumes 250 watt tops. That's really very impressive. The way intel seems to edge in to gpu territory and nvidia into cpu territory will make for some interesting stuff happening in the next couple of years.
        cjenkins 17y ago
        I believe the reference to unified address space is in reference to #6 in the PDF linked below and caches #4. I agree that the unified memory address space will be wonderful as managing all the various hierarchies by hand is a pain. http://www.nvidia.com/content/PDF/fermi_white_papers/D.Patterson_Top10InnovationsInNVIDIAFermi.pdf http://www.nvidia.com/content/PDF/fermi_white_papers/D.Patte...
profquail 17y ago
nVidia's latest OpenCL driver actually uses LLVM for compiling device code. If you want to read more: http://llvm.org/devmtg/2009-10/Grover_PLANG.pdf http://llvm.org/devmtg/2009-10/Grover_PLANG.pdf

so what happens if you go ptx -> llvm -> opencl -> nvidia gpu? how does the speed change?

Raphael_Amiard 17y ago

this shouldn't happen but it would be an interresting comparison nevertheless

andrewcooke 17y ago
it seems possible from the diagram on the linked page. why shouldn't it happen? i am curious whether the analysis stages can improve the code.
- sparky 17y ago
  I see that "OpenCL" is listed above the LLVM box on the Ocelot page, but I'm not sure why; It is known that several OpenCL toolchains (Nvidia, ATI, RapidMind) make use of LLVM, but it is unclear in what capacity they are used. For the sequence you described (PTX->OpenCL->GPU), there would have to be an OpenCL backend for LLVM. As far as I know, no such backend is publically available, and another compiler would be necessary to take the OpenCL source code down to PTX (from whence it came) and then the driver would JIT that PTX for your specific GPU model. > i am curious whether the analysis stages can improve the code. Ihe conventional wisdom is that any series of analyses that takes you from representation X, through one or more other representations, and back to X can only make things worse, assuming that the JIT from PTX to GPU machine code isn't horrendous. This is because any analyses, optimizations, and transformations need to be conservative to maintain correctness, and high-level semantic information about the parallelism inherent in the application is usually lost in each translation step. In this particular case it might not be so bad, as long as the LLVM IR is rich enough to faithfully represent the Cooperative Thread Array (CTA) semantics in PTX and not flatten them to SPMD code. My intuition, however, is that it's not; LLVM was designed as a fairly generic virtual machine that would faithfully represent most CPU-like execution models, and hardware CTAs (also called 'warps' in Nvidia parliance) are mostly a GPU-only phenomenon. CPUs have SIMD units (e.g. SSE, MMX, Altivec, NEON), but the execution model there is fundamentally different than the GPU.
  - jacquesm 17y ago
    Once the GT300 series hits the shelves that problem will be largely mitigated though, they're supposed to be mostly independent, with a 'variable warp' size. Of course that will introduce a new level of complexity to the optimization problem.
    - andrewcooke 17y ago
      i don't understand this either. "wariable warp size" sounds like a small efficiency fix for when things aren't multiples of 32, or when they exceed 512. a "variable warp size" doesn't alter the fundamental problem SIMD approach - you've still got a multiprocessor with slave processors that are doing very similar work. for me, the big advances in fermi are a unified address space and some kind of cache for the global memory. neither of those change the paradigm, but they may make life significantly simpler when programming the thing.
      - jacquesm 17y ago
        Multiples of 32 are nice, multiples of 1 are better :) (and let's hope it goes that far down), that would make things a lot easier as well. Unified address space I assume you mean across multiple GPUs ? Global memory cache is a double edged sword, that eats in to the transistor budget at a very rapid pace, effectively you already have a cache, you just have to fill it yourself. GPU programming is definitely a step back in the ease with which you can write programs, but if your problem maps well on to a GPU the speed increases are simply astounding. What would have taken you a cluster with 100 boxes now sits under your desk and consumes 250 watt tops. That's really very impressive. The way intel seems to edge in to gpu territory and nvidia into cpu territory will make for some interesting stuff happening in the next couple of years.
        cjenkins 17y ago
        I believe the reference to unified address space is in reference to #6 in the PDF linked below and caches #4. I agree that the unified memory address space will be wonderful as managing all the various hierarchies by hand is a pain. http://www.nvidia.com/content/PDF/fermi_white_papers/D.Patterson_Top10InnovationsInNVIDIAFermi.pdf http://www.nvidia.com/content/PDF/fermi_white_papers/D.Patte...

profquail 17y ago

nVidia's latest OpenCL driver actually uses LLVM for compiling device code. If you want to read more: http://llvm.org/devmtg/2009-10/Grover_PLANG.pdf http://llvm.org/devmtg/2009-10/Grover_PLANG.pdf