3 ms·

this shouldn't happen but it would be an interresting comparison nevertheless

andrewcooke 17y ago
it seems possible from the diagram on the linked page. why shouldn't it happen? i am curious whether the analysis stages can improve the code.
- sparky 17y ago
  I see that "OpenCL" is listed above the LLVM box on the Ocelot page, but I'm not sure why; It is known that several OpenCL toolchains (Nvidia, ATI, RapidMind) make use of LLVM, but it is unclear in what capacity they are used. For the sequence you described (PTX->OpenCL->GPU), there would have to be an OpenCL backend for LLVM. As far as I know, no such backend is publically available, and another compiler would be necessary to take the OpenCL source code down to PTX (from whence it came) and then the driver would JIT that PTX for your specific GPU model. > i am curious whether the analysis stages can improve the code. Ihe conventional wisdom is that any series of analyses that takes you from representation X, through one or more other representations, and back to X can only make things worse, assuming that the JIT from PTX to GPU machine code isn't horrendous. This is because any analyses, optimizations, and transformations need to be conservative to maintain correctness, and high-level semantic information about the parallelism inherent in the application is usually lost in each translation step. In this particular case it might not be so bad, as long as the LLVM IR is rich enough to faithfully represent the Cooperative Thread Array (CTA) semantics in PTX and not flatten them to SPMD code. My intuition, however, is that it's not; LLVM was designed as a fairly generic virtual machine that would faithfully represent most CPU-like execution models, and hardware CTAs (also called 'warps' in Nvidia parliance) are mostly a GPU-only phenomenon. CPUs have SIMD units (e.g. SSE, MMX, Altivec, NEON), but the execution model there is fundamentally different than the GPU.
  - jacquesm 17y ago
    Once the GT300 series hits the shelves that problem will be largely mitigated though, they're supposed to be mostly independent, with a 'variable warp' size. Of course that will introduce a new level of complexity to the optimization problem.
    - andrewcooke 17y ago
      i don't understand this either. "wariable warp size" sounds like a small efficiency fix for when things aren't multiples of 32, or when they exceed 512. a "variable warp size" doesn't alter the fundamental problem SIMD approach - you've still got a multiprocessor with slave processors that are doing very similar work. for me, the big advances in fermi are a unified address space and some kind of cache for the global memory. neither of those change the paradigm, but they may make life significantly simpler when programming the thing.
      - jacquesm 17y ago
        Multiples of 32 are nice, multiples of 1 are better :) (and let's hope it goes that far down), that would make things a lot easier as well. Unified address space I assume you mean across multiple GPUs ? Global memory cache is a double edged sword, that eats in to the transistor budget at a very rapid pace, effectively you already have a cache, you just have to fill it yourself. GPU programming is definitely a step back in the ease with which you can write programs, but if your problem maps well on to a GPU the speed increases are simply astounding. What would have taken you a cluster with 100 boxes now sits under your desk and consumes 250 watt tops. That's really very impressive. The way intel seems to edge in to gpu territory and nvidia into cpu territory will make for some interesting stuff happening in the next couple of years.
        cjenkins 17y ago
        I believe the reference to unified address space is in reference to #6 in the PDF linked below and caches #4. I agree that the unified memory address space will be wonderful as managing all the various hierarchies by hand is a pain. http://www.nvidia.com/content/PDF/fermi_white_papers/D.Patterson_Top10InnovationsInNVIDIAFermi.pdf http://www.nvidia.com/content/PDF/fermi_white_papers/D.Patte...
  - andrewcooke 17y ago
    so the analysis is in the llvm? the diagram is very confusing; it looks to me as though the analysis is before the llvm and applies to three different backends. but then looking some more, it seems to imply you can go ptx -> ocelot -> nvidia gpu (no llvm at all). (ie i agree with everything else you say - i just don't understand what that graphic is trying to show).
    - sparky 17y ago
      My interpretation, which I will further inform later today by scraping through the source, is that Ocelot takes in PTX from the Nvidia compiler and spits out LLVM bytecode, which can be JITed to run on an x86 (using LLVM's JIT) or compiled by LLVM to run on any other architectures it supports now or in the future. I think the diagram may mean that Ocelot presents a single CUDA device which may do any of these things (x86, GPU, ???-via-LLVM) to execute a CUDA kernel behind-the-scenes, transparently to whatever made the kernel call.
      - jacquesm 17y ago
        Please post your findings!
        sparky 17y ago
        So far it appears as though my initial suspicion was correct. Ocelot seems to present a single CUDA device interface which can use either the Nvidia driver or the LLVM JIT to run CUDA kernels. The eventual plan is to integrate Ocelot into a runtime system, Harmony, which presents a very high-level interface to parallel resources, and manages multiple backends to keep whatever mix of CPU and GPU cores you have busy. The plan is here (http://code.google.com/p/gpuocelot/wiki/Roadmap http://code.google.com/p/gpuocelot/wiki/Roadmap). Harmony is part of Greg Diamos' (http://gdiamos.net/ http://gdiamos.net/) ongoing PhD thesis work at Georgia Tech.
        andrewcooke 17y ago
        thanks!
  - profquail 17y ago
    I posted a link for PLANG (PTX frontend for LLVM) in my earlier comment above...that PDF has some good info on how nVidia is integrating LLVM into their toolchain.

it seems possible from the diagram on the linked page. why shouldn't it happen? i am curious whether the analysis stages can improve the code.

sparky 17y ago
I see that "OpenCL" is listed above the LLVM box on the Ocelot page, but I'm not sure why; It is known that several OpenCL toolchains (Nvidia, ATI, RapidMind) make use of LLVM, but it is unclear in what capacity they are used. For the sequence you described (PTX->OpenCL->GPU), there would have to be an OpenCL backend for LLVM. As far as I know, no such backend is publically available, and another compiler would be necessary to take the OpenCL source code down to PTX (from whence it came) and then the driver would JIT that PTX for your specific GPU model. > i am curious whether the analysis stages can improve the code. Ihe conventional wisdom is that any series of analyses that takes you from representation X, through one or more other representations, and back to X can only make things worse, assuming that the JIT from PTX to GPU machine code isn't horrendous. This is because any analyses, optimizations, and transformations need to be conservative to maintain correctness, and high-level semantic information about the parallelism inherent in the application is usually lost in each translation step. In this particular case it might not be so bad, as long as the LLVM IR is rich enough to faithfully represent the Cooperative Thread Array (CTA) semantics in PTX and not flatten them to SPMD code. My intuition, however, is that it's not; LLVM was designed as a fairly generic virtual machine that would faithfully represent most CPU-like execution models, and hardware CTAs (also called 'warps' in Nvidia parliance) are mostly a GPU-only phenomenon. CPUs have SIMD units (e.g. SSE, MMX, Altivec, NEON), but the execution model there is fundamentally different than the GPU.
- jacquesm 17y ago
  Once the GT300 series hits the shelves that problem will be largely mitigated though, they're supposed to be mostly independent, with a 'variable warp' size. Of course that will introduce a new level of complexity to the optimization problem.
  - andrewcooke 17y ago
    i don't understand this either. "wariable warp size" sounds like a small efficiency fix for when things aren't multiples of 32, or when they exceed 512. a "variable warp size" doesn't alter the fundamental problem SIMD approach - you've still got a multiprocessor with slave processors that are doing very similar work. for me, the big advances in fermi are a unified address space and some kind of cache for the global memory. neither of those change the paradigm, but they may make life significantly simpler when programming the thing.
    - jacquesm 17y ago
      Multiples of 32 are nice, multiples of 1 are better :) (and let's hope it goes that far down), that would make things a lot easier as well. Unified address space I assume you mean across multiple GPUs ? Global memory cache is a double edged sword, that eats in to the transistor budget at a very rapid pace, effectively you already have a cache, you just have to fill it yourself. GPU programming is definitely a step back in the ease with which you can write programs, but if your problem maps well on to a GPU the speed increases are simply astounding. What would have taken you a cluster with 100 boxes now sits under your desk and consumes 250 watt tops. That's really very impressive. The way intel seems to edge in to gpu territory and nvidia into cpu territory will make for some interesting stuff happening in the next couple of years.
      - cjenkins 17y ago
        I believe the reference to unified address space is in reference to #6 in the PDF linked below and caches #4. I agree that the unified memory address space will be wonderful as managing all the various hierarchies by hand is a pain. http://www.nvidia.com/content/PDF/fermi_white_papers/D.Patterson_Top10InnovationsInNVIDIAFermi.pdf http://www.nvidia.com/content/PDF/fermi_white_papers/D.Patte...
- andrewcooke 17y ago
  so the analysis is in the llvm? the diagram is very confusing; it looks to me as though the analysis is before the llvm and applies to three different backends. but then looking some more, it seems to imply you can go ptx -> ocelot -> nvidia gpu (no llvm at all). (ie i agree with everything else you say - i just don't understand what that graphic is trying to show).
  - sparky 17y ago
    My interpretation, which I will further inform later today by scraping through the source, is that Ocelot takes in PTX from the Nvidia compiler and spits out LLVM bytecode, which can be JITed to run on an x86 (using LLVM's JIT) or compiled by LLVM to run on any other architectures it supports now or in the future. I think the diagram may mean that Ocelot presents a single CUDA device which may do any of these things (x86, GPU, ???-via-LLVM) to execute a CUDA kernel behind-the-scenes, transparently to whatever made the kernel call.
    - jacquesm 17y ago
      Please post your findings!
      - sparky 17y ago
        So far it appears as though my initial suspicion was correct. Ocelot seems to present a single CUDA device interface which can use either the Nvidia driver or the LLVM JIT to run CUDA kernels. The eventual plan is to integrate Ocelot into a runtime system, Harmony, which presents a very high-level interface to parallel resources, and manages multiple backends to keep whatever mix of CPU and GPU cores you have busy. The plan is here (http://code.google.com/p/gpuocelot/wiki/Roadmap http://code.google.com/p/gpuocelot/wiki/Roadmap). Harmony is part of Greg Diamos' (http://gdiamos.net/ http://gdiamos.net/) ongoing PhD thesis work at Georgia Tech.
        andrewcooke 17y ago
        thanks!
- profquail 17y ago
  I posted a link for PLANG (PTX frontend for LLVM) in my earlier comment above...that PDF has some good info on how nVidia is integrating LLVM into their toolchain.