8 ms·

I’d be interested in understanding why the compilers never panned out but have never seen a good writeup on that. Or why people thought the compilers would be a

by gregw2 3y ago

I’d be interested in understanding why the compilers never panned out but have never seen a good writeup on that. Or why people thought the compilers would be able to succeed in the first place at the mission.

clausecker 3y ago
There are a number of reasons for the Itanium's poor performance, and it's the combination of these various factors that did it in. I wasn't present back in the Itanium's heyday, but this is what I gathered. As a quick recap, superscalar processors have multiple execution units, each of which can execute one instruction each cycle. So if you have three execution units, your CPU can execute up to three instructions every cycle. The conventional way to make use of the power of more than one execution unit is to have an out-of-order design, where a complicated mechanism (Tomasulo algorithm) decodes multiple instructions in parallel, tracks their dependencies and dispatches them onto execution units as they can be executed. Dependencies are resolved by having a large physical register file, which is dynamically mapped onto the programmer-visible logical register file (register renaming). This works well, but is notoriously complex to implement and requires a couple of extra pipeline stages before decode and execution, increasing the latency of mispredicted branches. The idea of VLIW architectures was to improve on this idea by moving the decision which instruction to execute on which port to the compiler. The compiler, having prescient knowledge about what your code is going to do next, can compute the optimal assignment of instructions to execution units. Each instruction word is a pack of multiple instructions, one for each port, that are executed simultaneously (these words become very wide, hence VLIW for Very Long Instruction Word). In essence, all the bits of the out-of-order mechanism between decoding and execution ports can be done away with and the decoder is much simpler, too. However, things fail in practice: * the whole idea hinges on the compiler being able to figure out the correct instruction schedule ahead of time. While feasible for Intel's/HP's in house compiler team, the authors of other toolchains largely did not bother, instead opting for more conventional code generation that did not performed all too well. * This issue was exacerbated by the Itanium's dreadful model for fast memory loads. You see, loads can take a long time to finish, especially if cache misses or page faults occur. To fix that, the Itanium has the option to do a speculative load, which may or may not succeed at a later point. So you can do a load from a dubious pointer, then check if the pointer is fine (e.g. is it in bounds? Is it a null pointer?), and only once it has been validated you make use of the result. This allows you to hide the latency of the load, significantly speeding up typical business logic. However, the load can still fail (e.g. due to to pagefault), in which case your code has to roll back to where the load should be performed and then do a conventional load as a back-up. Understandably, few, if any compilers ever made use of this feature and load latency was dealt with rather poorly. * Relatedly, the latency of some instructions like loads and division is variable and cannot easily be predicted. So there usually isn't even the one perfect schedule the compiler could find. Turns out the schedule is much better when you leave it to the Tomasulo mechanism, which has accurate knowledge of the latency of already executing long-latency instructions. * By design, VLIW instruction sets encode a lot about how the execution units work in the instruction format. For example, Itanium is designed for a machine with three execution units and each instruction pack has up to three instructions, one for each of them. But what if you want to put more execution units into the CPU in a future iteration of the design? Well, it's not straightforward. One approach is to ship executables in a bytecode, which is only scheduled and encoded on the machine it is installed on, allowed the instruction encoding and thus number of ports to vary. Intel had chosen a different approach and instead implemented later Itanium CPUs as out-of-order designs, combining the worst of both worlds. * Due to not having register renaming, VLIW architectures conventionally have a large register file (128 registers in the case of the Itanium). This slows down context switches, further reducing performance. Out-of-order CPUs can cheat by having a comparably small programmer-visible state, with most of the state hidden in the bowels of the processor and consequently not in need of saving or restoring. * Branch prediction rapidly grew more and more accurate shortly after the Itanium's release, reducing the importance of fast recovery from mispredictions. These days, branch prediction is up to 99% accurate and out-of-order CPUs can evaluate multiple branches per cycle using speculative execution. A feature, that is not possible with a straightforward VLIW design due to the lack of register renaming. So Intel locked itself out of one of the most crucial strategies for better performance with this approach. * Another enginering issue was that x86 simulation on the Itanium performed quite poorly, giving existing customers no incentive to switch. And those that did decide to switch found that if they invest into porting their software, they might as well make it fully portable and be independent of the architecture. This is the same problem that led to the death of DEC: by forcing their customers to rewrite all the VAX software for the Alpha, the created a bunch of customers that were no longer locked into their ecosystem and could now buy whatever UNIX box was cheapest on the free market.
- Kon-Peki 3y ago
  > To fix that, the Itanium has the option to do a speculative load, which may or may not succeed at a later point. So you can do a load from a dubious pointer, then check if the pointer is fine (e.g. is it in bounds? Is it a null pointer?), and only once it has been validated you make use of the result. Way back in the day, as a fairly young engineer, I was assigned to a project to get a bunch of legacy code migrated from Alpha to Itanium. The assignment was to "make it compile, run, and pass the tests. Do nothing else. At all." We were using the Intel C compiler on OpenVMS and every once in a while would encounter a crash in a block of code that looked something like this: if(ptr != NULL && ptr->val > 0) { //do something } else { //init the ptr } It was evaluating both parts of the if statement simultaneously and crashing on the second. Not being allowed to spend too much time debugging or investigating the compiler options, we did the following: if(ptr != NULL) { if(ptr->val > 0) { //do something } } else { //init the ptr } Which resolved the problem! EDIT - I recognize that the above change introduces a potential bug in the program ;) Obviously I wasn't copying code verbatim - it was 10-15 years ago! But you get the picture - the compiler was wonky, even the one you paid money for.
  - deleted 3y ago
    [deleted]
  - jandrese 3y ago
    When I was learning C many years ago I was warned that some compilers don't support boolean short circuiting and thus you had to be careful with it.
  - quux 3y ago
    Is this one of those rare cases where using a goto would be reasonable?
    - mjevans 3y ago
      The main case I ever found was implement missing language features. E.G. break 3; // Break 3 levels up break LABEL; // Break to a named label - safer-ish than goto goto LABEL; // When you have no other option. Usually for breaking out of a really deep set of loops to an outer loop. Such as a data stream reset, end of data, or for an error so bad a different language might E.G. throw an error and usually die.
_chris_ 3y ago
> I’d be interested in understanding why the compilers never panned out but have never seen a good writeup on that. Or why people thought the compilers would be able to succeed in the first place at the mission. It's a fundamentally impossible ask. Compilers are being asked to look at a program (perhaps watch it run a sample set) and guess the bias of each branch to construct a most-likely 'trace' path through the program, and then generate STATIC code for that path. But programs (and their branches) are not statically biased! So it simply doesn't work out for general-purpose codes. However, programs are fairly predictable, which means a branch predictor can dynamically learn the program path and regurgitate it on command. And if the program changes phases, the branch predictor can re-learn the new program path very quickly. Now if you wanted to couple a VLIW design with a dynamically re-executing compiler (dynamic binary translation), then sure, that can be made to work.
- yvdriess 3y ago
  > Now if you wanted to couple a VLIW design with a dynamically re-executing compiler (dynamic binary translation), then sure, that can be made to work. RIP Transmeta
  - andromeduck 3y ago
    Transmeta lived on in Nvidia's Project Denver but Denver was optimized for x86 and the Intel settlement precluded that. It ended up being too buggy/inefficient to compete in the market and effectively abandoned after the second generation.
- gregw2 3y ago
  This makes a lot of sense to me, thanks for boiling it down. Compilers can predict the code instructions coming up decently, but not really the data coming up, so VLIW doesn't work that well compared to branch prediction and speculative and out of order execution complexities which VLIW tried to simplify away on branching-heavy commercial/database server workloads. Does that sound right?
- actionfromafar 3y ago
  I think it could have worked if the IDE had performance instrumentation (some kind of tracing) which would have been fed in to the next build. (And perhaps several iterations of this.) Another way to leverage the Itanium power would have been to make a Java Virtual Machine go really fast, with dynamic binary translation. This way you'd sidestep all the C UB optimization caveats.
dfox 3y ago
One big reason is that it was 20 years ago. At that time, gcc only did rudimentary data flow analysis and full SSA dataflow was at best an experimental feature. Also, the market would not really accept a C compiler that does the kind of agressive UB exploitation needed to extract the paralelism from C code (and instead people mostly tended to pass -Wno-strict-aliasing and friends in order to reduce "warning noise"). This issue is somewhat C specific and Fortran compilers produced decidedly better IA-64 code than C compilers. Which is what together with respectable FP performance of Itanium made it somewhat popular for HPC.

by gregw2 3y ago

clausecker 3y ago

There are a number of reasons for the Itanium's poor performance, and it's the combination of these various factors that did it in. I wasn't present back in the Itanium's heyday, but this is what I gathered. As a quick recap, superscalar processors have multiple execution units, each of which can execute one instruction each cycle. So if you have three execution units, your CPU can execute up to three instructions every cycle. The conventional way to make use of the power of more than one execution unit is to have an out-of-order design, where a complicated mechanism (Tomasulo algorithm) decodes multiple instructions in parallel, tracks their dependencies and dispatches them onto execution units as they can be executed. Dependencies are resolved by having a large physical register file, which is dynamically mapped onto the programmer-visible logical register file (register renaming). This works well, but is notoriously complex to implement and requires a couple of extra pipeline stages before decode and execution, increasing the latency of mispredicted branches. The idea of VLIW architectures was to improve on this idea by moving the decision which instruction to execute on which port to the compiler. The compiler, having prescient knowledge about what your code is going to do next, can compute the optimal assignment of instructions to execution units. Each instruction word is a pack of multiple instructions, one for each port, that are executed simultaneously (these words become very wide, hence VLIW for Very Long Instruction Word). In essence, all the bits of the out-of-order mechanism between decoding and execution ports can be done away with and the decoder is much simpler, too. However, things fail in practice: * the whole idea hinges on the compiler being able to figure out the correct instruction schedule ahead of time. While feasible for Intel's/HP's in house compiler team, the authors of other toolchains largely did not bother, instead opting for more conventional code generation that did not performed all too well. * This issue was exacerbated by the Itanium's dreadful model for fast memory loads. You see, loads can take a long time to finish, especially if cache misses or page faults occur. To fix that, the Itanium has the option to do a speculative load, which may or may not succeed at a later point. So you can do a load from a dubious pointer, then check if the pointer is fine (e.g. is it in bounds? Is it a null pointer?), and only once it has been validated you make use of the result. This allows you to hide the latency of the load, significantly speeding up typical business logic. However, the load can still fail (e.g. due to to pagefault), in which case your code has to roll back to where the load should be performed and then do a conventional load as a back-up. Understandably, few, if any compilers ever made use of this feature and load latency was dealt with rather poorly. * Relatedly, the latency of some instructions like loads and division is variable and cannot easily be predicted. So there usually isn't even the one perfect schedule the compiler could find. Turns out the schedule is much better when you leave it to the Tomasulo mechanism, which has accurate knowledge of the latency of already executing long-latency instructions. * By design, VLIW instruction sets encode a lot about how the execution units work in the instruction format. For example, Itanium is designed for a machine with three execution units and each instruction pack has up to three instructions, one for each of them. But what if you want to put more execution units into the CPU in a future iteration of the design? Well, it's not straightforward. One approach is to ship executables in a bytecode, which is only scheduled and encoded on the machine it is installed on, allowed the instruction encoding and thus number of ports to vary. Intel had chosen a different approach and instead implemented later Itanium CPUs as out-of-order designs, combining the worst of both worlds. * Due to not having register renaming, VLIW architectures conventionally have a large register file (128 registers in the case of the Itanium). This slows down context switches, further reducing performance. Out-of-order CPUs can cheat by having a comparably small programmer-visible state, with most of the state hidden in the bowels of the processor and consequently not in need of saving or restoring. * Branch prediction rapidly grew more and more accurate shortly after the Itanium's release, reducing the importance of fast recovery from mispredictions. These days, branch prediction is up to 99% accurate and out-of-order CPUs can evaluate multiple branches per cycle using speculative execution. A feature, that is not possible with a straightforward VLIW design due to the lack of register renaming. So Intel locked itself out of one of the most crucial strategies for better performance with this approach. * Another enginering issue was that x86 simulation on the Itanium performed quite poorly, giving existing customers no incentive to switch. And those that did decide to switch found that if they invest into porting their software, they might as well make it fully portable and be independent of the architecture. This is the same problem that led to the death of DEC: by forcing their customers to rewrite all the VAX software for the Alpha, the created a bunch of customers that were no longer locked into their ecosystem and could now buy whatever UNIX box was cheapest on the free market.

Kon-Peki 3y ago
> To fix that, the Itanium has the option to do a speculative load, which may or may not succeed at a later point. So you can do a load from a dubious pointer, then check if the pointer is fine (e.g. is it in bounds? Is it a null pointer?), and only once it has been validated you make use of the result. Way back in the day, as a fairly young engineer, I was assigned to a project to get a bunch of legacy code migrated from Alpha to Itanium. The assignment was to "make it compile, run, and pass the tests. Do nothing else. At all." We were using the Intel C compiler on OpenVMS and every once in a while would encounter a crash in a block of code that looked something like this: if(ptr != NULL && ptr->val > 0) { //do something } else { //init the ptr } It was evaluating both parts of the if statement simultaneously and crashing on the second. Not being allowed to spend too much time debugging or investigating the compiler options, we did the following: if(ptr != NULL) { if(ptr->val > 0) { //do something } } else { //init the ptr } Which resolved the problem! EDIT - I recognize that the above change introduces a potential bug in the program ;) Obviously I wasn't copying code verbatim - it was 10-15 years ago! But you get the picture - the compiler was wonky, even the one you paid money for.
- deleted 3y ago
  [deleted]
- jandrese 3y ago
  When I was learning C many years ago I was warned that some compilers don't support boolean short circuiting and thus you had to be careful with it.
- quux 3y ago
  Is this one of those rare cases where using a goto would be reasonable?
  - mjevans 3y ago
    The main case I ever found was implement missing language features. E.G. break 3; // Break 3 levels up break LABEL; // Break to a named label - safer-ish than goto goto LABEL; // When you have no other option. Usually for breaking out of a really deep set of loops to an outer loop. Such as a data stream reset, end of data, or for an error so bad a different language might E.G. throw an error and usually die.

_chris_ 3y ago

> I’d be interested in understanding why the compilers never panned out but have never seen a good writeup on that. Or why people thought the compilers would be able to succeed in the first place at the mission. It's a fundamentally impossible ask. Compilers are being asked to look at a program (perhaps watch it run a sample set) and guess the bias of each branch to construct a most-likely 'trace' path through the program, and then generate STATIC code for that path. But programs (and their branches) are not statically biased! So it simply doesn't work out for general-purpose codes. However, programs are fairly predictable, which means a branch predictor can dynamically learn the program path and regurgitate it on command. And if the program changes phases, the branch predictor can re-learn the new program path very quickly. Now if you wanted to couple a VLIW design with a dynamically re-executing compiler (dynamic binary translation), then sure, that can be made to work.

yvdriess 3y ago
> Now if you wanted to couple a VLIW design with a dynamically re-executing compiler (dynamic binary translation), then sure, that can be made to work. RIP Transmeta
- andromeduck 3y ago
  Transmeta lived on in Nvidia's Project Denver but Denver was optimized for x86 and the Intel settlement precluded that. It ended up being too buggy/inefficient to compete in the market and effectively abandoned after the second generation.
gregw2 3y ago
This makes a lot of sense to me, thanks for boiling it down. Compilers can predict the code instructions coming up decently, but not really the data coming up, so VLIW doesn't work that well compared to branch prediction and speculative and out of order execution complexities which VLIW tried to simplify away on branching-heavy commercial/database server workloads. Does that sound right?
actionfromafar 3y ago
I think it could have worked if the IDE had performance instrumentation (some kind of tracing) which would have been fed in to the next build. (And perhaps several iterations of this.) Another way to leverage the Itanium power would have been to make a Java Virtual Machine go really fast, with dynamic binary translation. This way you'd sidestep all the C UB optimization caveats.

dfox 3y ago

One big reason is that it was 20 years ago. At that time, gcc only did rudimentary data flow analysis and full SSA dataflow was at best an experimental feature. Also, the market would not really accept a C compiler that does the kind of agressive UB exploitation needed to extract the paralelism from C code (and instead people mostly tended to pass -Wno-strict-aliasing and friends in order to reduce "warning noise"). This issue is somewhat C specific and Fortran compilers produced decidedly better IA-64 code than C compilers. Which is what together with respectable FP performance of Itanium made it somewhat popular for HPC.