4 ms·

"The Itanium is a noteworthy example of what happens when one designs an architecture exclusively for parallelism to the exclusion of all else, and leaves all i

"The Itanium is a noteworthy example of what happens when one designs an architecture exclusively for parallelism to the exclusion of all else, and leaves all instruction scheduling to the compiler. " Well no, it would be more accurate to say "a noteworthy example of when you release a chip that isn't in line with what common programming languages can do". That is, there is nothing wrong with doing the above. There's something wrong with doing the above when you most people are writing sequential C++ :) There simply wasn't enough ILP to extract :) IE no amount of compiler advance would have saved it.

chrisseaton 9y ago
But conventional Intel processors do manage to dynamically extract quite a good deal of ILP don't they? That's been the source of most of our performance increases over the last decade I think. Why does it work dynamically but not statically?
- DannyBee 9y ago
  They can extract some ILP, but most of that is because it has no limitations but the semantics of the instruction set. It also cheats and can speculate just about everything legally, because it can always undo it. A large amount of the ILP is, AFAIK, from being able to prefetch accurately (IE guess what memory will be accessed) and guess branch directions correctly. This is not possible to do as well statically. You actually can do it well dynamically (IE i could make a branch guessing thread and a prefetching thread), but you can't do it fast enough. In a lot of workloads, most time is spent stalling for memory. Also note that in VLIW, you have to extract ILP or you are just wasting time (IE are executing NOPs). You end up with a half or 3/4 empty pipeline. So unless you can predict you are going to stall for memory well, you are wasting the pipeline. On the other hand, modern OOO cores can just extract more ILP when possible. IE when stalling for memory, they can figure out what they could be doing otherwise.
  - chrisseaton 9y ago
    Maybe it's like JITs vs static compilers - we're better off with JIT even if your language is static, because you can adapt to real runtime conditions.
    - CyberDildonics 9y ago
      But we aren't better off with JIT in software. x64 processors work because they are doing things dynamically in separate hardware. It is part of how they use ever increasing transistor budgets to speed up serial processing.
      - bonzini 9y ago
        Really, at this point the transistor budget is mostly going to bigger caches and more cores. It's really little short of a miracle that Intel still managed to extract double digit percent improvements in serial code for all of Sandy Bridge, Haswell and Skylake.
        deepnotderp 9y ago
        It's less of a miracle and more like "larger OoO window, K done"
        bonzini 9y ago
        No, that's not so easy. You cannot fill that out-of-order window if you mispredict branches. So the branch prediction has gotten better and better, to an absurd level of accuracy. You also need more execution units, to accommodate more kinds of serial code without stalling the pipeline. In Skylake they also improved the retirement unit, apparently to improve hyperthreading performance. There are also better prefetchers and decoders, while the decoders also become more complicated because this is good old x86's variable-length encoding, with further complications for x86-64, AVX, and so on. TLBs also get more complex (there have been two-level TLBs for a while now), though I guess that is part of the "bigger caches" part.
- CyberDildonics 9y ago
  One reason is that the idea behind 'just need a smart compiler' was doomed from the start. If you build something that won't shine unless everyone that builds a compiler for it makes sure to extract plenty of instruction level parallelism, it is doomed to fail. Maybe these days with LLVM it could have been a little more practical, but Intel has dominated largely by making sure poorly written programs run as fast as possible. To put all that aside and hope that compilers and programmers become much more sophisticated was never going to happen, especially because so much software is already set and won't change.
  - trsohmers 9y ago
    LLVM is the godsend that any new architecture (especially a VLIW) to compete with Intel needed. I'm the founder of a startup that has made a new VLIW processor with our toolchain built on top of LLVM... We get the front end and all of the languages it supports virtually for free (we've only had to make small tweaks to clang), and get to focus our effort on just the parts important to us, our actual target backend and the unique optimization steps. Right now we can take any code that compiles through LLVM and run it on a single one of our cores while finding a good chunk of ILP (we are quad issue, averaging close to 2.5 IPC, sometimes higher), and be beating out the performance of some ARM and DSP cores that are 10x the size. Multi core parallelism requires including our hand tuned libraries, but our true MIMD style of task and data level parallelism is a lot more flexible than SIMD/SIMT/SMP that you are stuck with on existing chips.
    - chrisseaton 9y ago
      Is it the case that you can implement just a new backend and produce good machine code for your processor, or really do you need to modify the frontend to get it to generate code IR from which you can generate good machine code, because otherwise useful information has been lost by time you get the IR from the frontend?
      - trsohmers 9y ago
        We have played around with having "hints" in the original source code that will be preserved through the IR, but the vast majority of the optimizations and scheduling decisions we need to do are happening once it has been converted into machine code by the backend. I may be slightly off (I'm the dumb CEO/hardware architect), but I'm fairly sure 85%-90% of our compiler work has been done on the backend... I just remember being told some tweaks were made to clang due to some weird issue.
thesz 9y ago
As a developer of out-of-order CPU (model, but studied the effects on real code and did my best to make it synthesable) I can assure you there's a lot of ILP to extract even from ordinary programs. The problem with Itanium is that it wasn't OoO and can't be made such. Just to give you a hint - just computing various kinds of addresses in advance can make huge difference. Compute branch address, issue a fetch, compute load address, issue a fetch (even when destination register is blocked). Voila! Your program is sped up significantly.
- gsg 9y ago
  Itanium did have features for various kinds of speculation, though. You could load from an address even if it were potentially aliased with a subsequent store, with the processor doing a dynamic check to make sure the right value was observed (eg, performing a late load and blocking if it turned out there was an overlapping store). It wasn't as effective as OoO in the end, but not for lack of attention to the problem.

chrisseaton 9y ago

But conventional Intel processors do manage to dynamically extract quite a good deal of ILP don't they? That's been the source of most of our performance increases over the last decade I think. Why does it work dynamically but not statically?

DannyBee 9y ago
They can extract some ILP, but most of that is because it has no limitations but the semantics of the instruction set. It also cheats and can speculate just about everything legally, because it can always undo it. A large amount of the ILP is, AFAIK, from being able to prefetch accurately (IE guess what memory will be accessed) and guess branch directions correctly. This is not possible to do as well statically. You actually can do it well dynamically (IE i could make a branch guessing thread and a prefetching thread), but you can't do it fast enough. In a lot of workloads, most time is spent stalling for memory. Also note that in VLIW, you have to extract ILP or you are just wasting time (IE are executing NOPs). You end up with a half or 3/4 empty pipeline. So unless you can predict you are going to stall for memory well, you are wasting the pipeline. On the other hand, modern OOO cores can just extract more ILP when possible. IE when stalling for memory, they can figure out what they could be doing otherwise.
- chrisseaton 9y ago
  Maybe it's like JITs vs static compilers - we're better off with JIT even if your language is static, because you can adapt to real runtime conditions.
  - CyberDildonics 9y ago
    But we aren't better off with JIT in software. x64 processors work because they are doing things dynamically in separate hardware. It is part of how they use ever increasing transistor budgets to speed up serial processing.
    - bonzini 9y ago
      Really, at this point the transistor budget is mostly going to bigger caches and more cores. It's really little short of a miracle that Intel still managed to extract double digit percent improvements in serial code for all of Sandy Bridge, Haswell and Skylake.
      - deepnotderp 9y ago
        It's less of a miracle and more like "larger OoO window, K done"
        bonzini 9y ago
        No, that's not so easy. You cannot fill that out-of-order window if you mispredict branches. So the branch prediction has gotten better and better, to an absurd level of accuracy. You also need more execution units, to accommodate more kinds of serial code without stalling the pipeline. In Skylake they also improved the retirement unit, apparently to improve hyperthreading performance. There are also better prefetchers and decoders, while the decoders also become more complicated because this is good old x86's variable-length encoding, with further complications for x86-64, AVX, and so on. TLBs also get more complex (there have been two-level TLBs for a while now), though I guess that is part of the "bigger caches" part.
CyberDildonics 9y ago
One reason is that the idea behind 'just need a smart compiler' was doomed from the start. If you build something that won't shine unless everyone that builds a compiler for it makes sure to extract plenty of instruction level parallelism, it is doomed to fail. Maybe these days with LLVM it could have been a little more practical, but Intel has dominated largely by making sure poorly written programs run as fast as possible. To put all that aside and hope that compilers and programmers become much more sophisticated was never going to happen, especially because so much software is already set and won't change.
- trsohmers 9y ago
  LLVM is the godsend that any new architecture (especially a VLIW) to compete with Intel needed. I'm the founder of a startup that has made a new VLIW processor with our toolchain built on top of LLVM... We get the front end and all of the languages it supports virtually for free (we've only had to make small tweaks to clang), and get to focus our effort on just the parts important to us, our actual target backend and the unique optimization steps. Right now we can take any code that compiles through LLVM and run it on a single one of our cores while finding a good chunk of ILP (we are quad issue, averaging close to 2.5 IPC, sometimes higher), and be beating out the performance of some ARM and DSP cores that are 10x the size. Multi core parallelism requires including our hand tuned libraries, but our true MIMD style of task and data level parallelism is a lot more flexible than SIMD/SIMT/SMP that you are stuck with on existing chips.
  - chrisseaton 9y ago
    Is it the case that you can implement just a new backend and produce good machine code for your processor, or really do you need to modify the frontend to get it to generate code IR from which you can generate good machine code, because otherwise useful information has been lost by time you get the IR from the frontend?
    - trsohmers 9y ago
      We have played around with having "hints" in the original source code that will be preserved through the IR, but the vast majority of the optimizations and scheduling decisions we need to do are happening once it has been converted into machine code by the backend. I may be slightly off (I'm the dumb CEO/hardware architect), but I'm fairly sure 85%-90% of our compiler work has been done on the backend... I just remember being told some tweaks were made to clang due to some weird issue.

thesz 9y ago

As a developer of out-of-order CPU (model, but studied the effects on real code and did my best to make it synthesable) I can assure you there's a lot of ILP to extract even from ordinary programs. The problem with Itanium is that it wasn't OoO and can't be made such. Just to give you a hint - just computing various kinds of addresses in advance can make huge difference. Compute branch address, issue a fetch, compute load address, issue a fetch (even when destination register is blocked). Voila! Your program is sped up significantly.

gsg 9y ago
Itanium did have features for various kinds of speculation, though. You could load from an address even if it were potentially aliased with a subsequent store, with the processor doing a dynamic check to make sure the right value was observed (eg, performing a late load and blocking if it turned out there was an overlapping store). It wasn't as effective as OoO in the end, but not for lack of attention to the problem.