4 ms·

The monster is back. >Itanium relies on the compiler to optimize code at run-time Thats sums it up for me :) But seriously - in case when compiler is able to

The monster is back. >Itanium relies on the compiler to optimize code at run-time Thats sums it up for me :) But seriously - in case when compiler is able to do parallelization, NVidia GPU seems to be a better - cheaper, more accessible and performant - target.

jbri 16y ago
There are serious limitations on what you can actually do on a GPU - some operations are terribly slow, and flipping back to the CPU to carry them out is also slow. They're great for embarrassingly-parallel simple operations, but once you break the ceiling on complexity you're often better off trying to vectorize it on the CPU rather than try and manage all this chatter between the CPU and GPU. Itanium seems like it fits nicely in that space - operations complex enough to be very painful on a graphics processor, but parallel enough for you to actually consider using a GPU in the first place.
scott_s 16y ago
Not all parallelism is the same. In this article, parallelism usually means instruction level parallelism (http://en.wikipedia.org/wiki/Instruction_level_parallelism http://en.wikipedia.org/wiki/Instruction_level_parallelism).GPUs are fantastic at data parallelism (http://en.wikipedia.org/wiki/Data_parallelism http://en.wikipedia.org/wiki/Data_parallelism). Being able to exploit one says nothing about the other. Itanium differs from other processor architectures is how it handles instruction level parallelism. The processor in the computer in front of you probably uses out-of-order execution (http://en.wikipedia.org/wiki/Out-of-order_execution http://en.wikipedia.org/wiki/Out-of-order_execution) to exploit ILP. This happens on the fly, as a program executes. Itanium depends on the compiler to determine where ILP is.
- VladRussian 16y ago
  >Being able to exploit one says nothing about the other. data parallelism is a partial case of instruction level parallelism - N instances of the same instruction run for different pieces of data. A very frequent case in high performance computing or enterprise data crunching tasks supposedly targeted by the Itanic
  - scott_s 16y ago
    What you described is not what we mean by the term "instruction level parallelism." Yes, it is parallelism that involves instructions, but it is not ILP. Instruction level parallelism (ILP), data level parallelism (DLP) and task (sometimes thread) level parallelism (TLP) are all orthogonal. I can have data level parallelism that is not at the instruction level.
    - VladRussian 16y ago
      so, according to you, the data level parallelism at the instruction level i described isn't a case of instruction level parallelism? http://en.wikipedia.org/wiki/Instruction-level_parallelism http://en.wikipedia.org/wiki/Instruction-level_parallelism and returning to the original specific context of Itanium vs. GPU : http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter35.html http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter35... Take your stub at what can be classified as what :)
      - scott_s 16y ago
        > so, according to you, the data level parallelism at the instruction level i described isn't a case of instruction level parallelism? Correct. The GPU article on NVIDIA's website uses ILP incorrectly. They are describing SIMD operations - single instruction, multiple data - which is data parallelism at the instruction level. This is inherently different than ILP, which is when you extract parallelism from a sequential stream of instructions by executing them out-of-order. Of course, it's possible to exploit ILP on a stream of SIMD instructions.
        VladRussian 16y ago
        ok, it's an enlightening discussion on terminology. Back to my main point - GPU vs. Itanic. NVIDIA GPU Tesla have 512 SIMD cores organized into 16 SM ("streaming multiprocessors") with 2 independent instructions (from 2 different threads) issued per SM per clock (each instruction goes to its' half of the SM, ie. to 16 cores wide SIMD group): http://www.nvidia.com/content/PDF/fermi_white_papers/NVIDIA_Fermi_Compute_Architecture_Whitepaper.pdf http://www.nvidia.com/content/PDF/fermi_white_papers/NVIDIA_... That gives us [16 SM x 2 instructions x [1-16 cores]] - anywhere between 32 to 512 ops / clock - 1000 GFlops in the best case. The Itanic - 8 cores x 6 independent instructions - something like 200GFlops.

jbri 16y ago

There are serious limitations on what you can actually do on a GPU - some operations are terribly slow, and flipping back to the CPU to carry them out is also slow. They're great for embarrassingly-parallel simple operations, but once you break the ceiling on complexity you're often better off trying to vectorize it on the CPU rather than try and manage all this chatter between the CPU and GPU. Itanium seems like it fits nicely in that space - operations complex enough to be very painful on a graphics processor, but parallel enough for you to actually consider using a GPU in the first place.

scott_s 16y ago

Not all parallelism is the same. In this article, parallelism usually means instruction level parallelism (http://en.wikipedia.org/wiki/Instruction_level_parallelism http://en.wikipedia.org/wiki/Instruction_level_parallelism).GPUs are fantastic at data parallelism (http://en.wikipedia.org/wiki/Data_parallelism http://en.wikipedia.org/wiki/Data_parallelism). Being able to exploit one says nothing about the other. Itanium differs from other processor architectures is how it handles instruction level parallelism. The processor in the computer in front of you probably uses out-of-order execution (http://en.wikipedia.org/wiki/Out-of-order_execution http://en.wikipedia.org/wiki/Out-of-order_execution) to exploit ILP. This happens on the fly, as a program executes. Itanium depends on the compiler to determine where ILP is.

VladRussian 16y ago
>Being able to exploit one says nothing about the other. data parallelism is a partial case of instruction level parallelism - N instances of the same instruction run for different pieces of data. A very frequent case in high performance computing or enterprise data crunching tasks supposedly targeted by the Itanic
- scott_s 16y ago
  What you described is not what we mean by the term "instruction level parallelism." Yes, it is parallelism that involves instructions, but it is not ILP. Instruction level parallelism (ILP), data level parallelism (DLP) and task (sometimes thread) level parallelism (TLP) are all orthogonal. I can have data level parallelism that is not at the instruction level.
  - VladRussian 16y ago
    so, according to you, the data level parallelism at the instruction level i described isn't a case of instruction level parallelism? http://en.wikipedia.org/wiki/Instruction-level_parallelism http://en.wikipedia.org/wiki/Instruction-level_parallelism and returning to the original specific context of Itanium vs. GPU : http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter35.html http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter35... Take your stub at what can be classified as what :)
    - scott_s 16y ago
      > so, according to you, the data level parallelism at the instruction level i described isn't a case of instruction level parallelism? Correct. The GPU article on NVIDIA's website uses ILP incorrectly. They are describing SIMD operations - single instruction, multiple data - which is data parallelism at the instruction level. This is inherently different than ILP, which is when you extract parallelism from a sequential stream of instructions by executing them out-of-order. Of course, it's possible to exploit ILP on a stream of SIMD instructions.
      - VladRussian 16y ago
        ok, it's an enlightening discussion on terminology. Back to my main point - GPU vs. Itanic. NVIDIA GPU Tesla have 512 SIMD cores organized into 16 SM ("streaming multiprocessors") with 2 independent instructions (from 2 different threads) issued per SM per clock (each instruction goes to its' half of the SM, ie. to 16 cores wide SIMD group): http://www.nvidia.com/content/PDF/fermi_white_papers/NVIDIA_Fermi_Compute_Architecture_Whitepaper.pdf http://www.nvidia.com/content/PDF/fermi_white_papers/NVIDIA_... That gives us [16 SM x 2 instructions x [1-16 cores]] - anywhere between 32 to 512 ops / clock - 1000 GFlops in the best case. The Itanic - 8 cores x 6 independent instructions - something like 200GFlops.