5 ms·

>Being able to exploit one says nothing about the other. data parallelism is a partial case of instruction level parallelism - N instances of the same instruct

>Being able to exploit one says nothing about the other. data parallelism is a partial case of instruction level parallelism - N instances of the same instruction run for different pieces of data. A very frequent case in high performance computing or enterprise data crunching tasks supposedly targeted by the Itanic

scott_s 16y ago
What you described is not what we mean by the term "instruction level parallelism." Yes, it is parallelism that involves instructions, but it is not ILP. Instruction level parallelism (ILP), data level parallelism (DLP) and task (sometimes thread) level parallelism (TLP) are all orthogonal. I can have data level parallelism that is not at the instruction level.
- VladRussian 16y ago
  so, according to you, the data level parallelism at the instruction level i described isn't a case of instruction level parallelism? http://en.wikipedia.org/wiki/Instruction-level_parallelism http://en.wikipedia.org/wiki/Instruction-level_parallelism and returning to the original specific context of Itanium vs. GPU : http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter35.html http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter35... Take your stub at what can be classified as what :)
  - scott_s 16y ago
    > so, according to you, the data level parallelism at the instruction level i described isn't a case of instruction level parallelism? Correct. The GPU article on NVIDIA's website uses ILP incorrectly. They are describing SIMD operations - single instruction, multiple data - which is data parallelism at the instruction level. This is inherently different than ILP, which is when you extract parallelism from a sequential stream of instructions by executing them out-of-order. Of course, it's possible to exploit ILP on a stream of SIMD instructions.
    - VladRussian 16y ago
      ok, it's an enlightening discussion on terminology. Back to my main point - GPU vs. Itanic. NVIDIA GPU Tesla have 512 SIMD cores organized into 16 SM ("streaming multiprocessors") with 2 independent instructions (from 2 different threads) issued per SM per clock (each instruction goes to its' half of the SM, ie. to 16 cores wide SIMD group): http://www.nvidia.com/content/PDF/fermi_white_papers/NVIDIA_Fermi_Compute_Architecture_Whitepaper.pdf http://www.nvidia.com/content/PDF/fermi_white_papers/NVIDIA_... That gives us [16 SM x 2 instructions x [1-16 cores]] - anywhere between 32 to 512 ops / clock - 1000 GFlops in the best case. The Itanic - 8 cores x 6 independent instructions - something like 200GFlops.
    - VladRussian 16y ago
      >The GPU article on NVIDIA's website uses ILP incorrectly. That depends on the implementation. Lets say there is a program f3=OP1(f1) f4=OP1(f2) OP2(f3) OP2(f4) Some possible ILP forms: OP1(f1)OP1(f2) OP2(f3)OP2(f4) or OP1(f1) OP1(f2)OP2(f3) OP2(f4) A DLP form: OP1(f1, f2) OP2(f3, f4) or may be their DLP implemented as OP1(f1)OP1(f2) OP2(f3)OP2(f4) and if it is the case i don't see why they can't call it an ILP.
      - scott_s 16y ago
        The concepts are orthogonal. Which means you can apply both at the same time. If you have a stream of SIMD instructions - which is data parallelism - and you can determine that you can execute some of them in parallel, then you are able to extract instruction level parallelism out of a stream of data parallel instructions. ILP means something very specific. This is a discussion of semantics, but semantics are important so that we can communicate easily. If I stick a Hershey's bar in the oven, it is literally hot chocolate, but it is not what we normally mean when we say "hot chocolate." You are talking about parallelism at the instruction level. I'm trying to explain that that is note what people mean when they say "instruction level parallelism."
        VladRussian 16y ago
        >You are talking about parallelism at the instruction level. I'm trying to explain that that is note what people mean when they say "instruction level parallelism." ok, just point which of the 2 ILP executions i mentioned above isn't really ILP, and which of the 2 DLP executions isn't really a DLP.

scott_s 16y ago

What you described is not what we mean by the term "instruction level parallelism." Yes, it is parallelism that involves instructions, but it is not ILP. Instruction level parallelism (ILP), data level parallelism (DLP) and task (sometimes thread) level parallelism (TLP) are all orthogonal. I can have data level parallelism that is not at the instruction level.

VladRussian 16y ago
so, according to you, the data level parallelism at the instruction level i described isn't a case of instruction level parallelism? http://en.wikipedia.org/wiki/Instruction-level_parallelism http://en.wikipedia.org/wiki/Instruction-level_parallelism and returning to the original specific context of Itanium vs. GPU : http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter35.html http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter35... Take your stub at what can be classified as what :)
- scott_s 16y ago
  > so, according to you, the data level parallelism at the instruction level i described isn't a case of instruction level parallelism? Correct. The GPU article on NVIDIA's website uses ILP incorrectly. They are describing SIMD operations - single instruction, multiple data - which is data parallelism at the instruction level. This is inherently different than ILP, which is when you extract parallelism from a sequential stream of instructions by executing them out-of-order. Of course, it's possible to exploit ILP on a stream of SIMD instructions.
  - VladRussian 16y ago
    ok, it's an enlightening discussion on terminology. Back to my main point - GPU vs. Itanic. NVIDIA GPU Tesla have 512 SIMD cores organized into 16 SM ("streaming multiprocessors") with 2 independent instructions (from 2 different threads) issued per SM per clock (each instruction goes to its' half of the SM, ie. to 16 cores wide SIMD group): http://www.nvidia.com/content/PDF/fermi_white_papers/NVIDIA_Fermi_Compute_Architecture_Whitepaper.pdf http://www.nvidia.com/content/PDF/fermi_white_papers/NVIDIA_... That gives us [16 SM x 2 instructions x [1-16 cores]] - anywhere between 32 to 512 ops / clock - 1000 GFlops in the best case. The Itanic - 8 cores x 6 independent instructions - something like 200GFlops.
  - VladRussian 16y ago
    >The GPU article on NVIDIA's website uses ILP incorrectly. That depends on the implementation. Lets say there is a program f3=OP1(f1) f4=OP1(f2) OP2(f3) OP2(f4) Some possible ILP forms: OP1(f1)OP1(f2) OP2(f3)OP2(f4) or OP1(f1) OP1(f2)OP2(f3) OP2(f4) A DLP form: OP1(f1, f2) OP2(f3, f4) or may be their DLP implemented as OP1(f1)OP1(f2) OP2(f3)OP2(f4) and if it is the case i don't see why they can't call it an ILP.
    - scott_s 16y ago
      The concepts are orthogonal. Which means you can apply both at the same time. If you have a stream of SIMD instructions - which is data parallelism - and you can determine that you can execute some of them in parallel, then you are able to extract instruction level parallelism out of a stream of data parallel instructions. ILP means something very specific. This is a discussion of semantics, but semantics are important so that we can communicate easily. If I stick a Hershey's bar in the oven, it is literally hot chocolate, but it is not what we normally mean when we say "hot chocolate." You are talking about parallelism at the instruction level. I'm trying to explain that that is note what people mean when they say "instruction level parallelism."
      - VladRussian 16y ago
        >You are talking about parallelism at the instruction level. I'm trying to explain that that is note what people mean when they say "instruction level parallelism." ok, just point which of the 2 ILP executions i mentioned above isn't really ILP, and which of the 2 DLP executions isn't really a DLP.