11 ms·

Arch: Remove Itanium (IA-64) architecture

69 points by ndesaulniers 3y ago

twoodfin 3y ago
Imagine HP & Intel execute on Itanium to perfection: Were they still doomed to failure, or is VLIW/EPIC actually a viable competitive approach for high-performance general purpose CPUs? The market evidence suggests it’s not, but there haven’t been that many well-funded, competent attempts (Itanium, Transmeta, ???) so the sample size is small.
- KMag 3y ago
  Code density is an issue, requiring huge caches for Itanium to remain competitive. These large caches consume lots of die area, making the chips both expensive and power-hungry. As I and others have pointed out, instead of a constant 3 instructions in 128 bits, a variable number of instructions in 64 bits should get you competitive code densities. (Say, have a 4-bit prefix that indicates if 1x60 bits, 2x30 bits, 4x15 bits, or 5x12 bits instructions are packed in the remaining 60 bits. For 12-bit instructions, you probably have the first instruction only able to store into r1 or r2, the second instruction only able to store into r3 or r4, etc. so that you have a 6-bit opcode, 1 bit for destination register and 5 bits for operand register. Plenty of old processors had an accumulator that was the implicit destination for most instructions.) Some of the other 4-bit prefixes would be used to indicate a combination of instruction widths and instruction parallelism to allow lower-powered in-order superscalar implementations. Fixed alignment of 64-bit bundles of variable-width instructions makes parallel instruction decoding cheaper (and makes static analysis easier/finding ROP exploit targets very slightly harder). Your high-performance cores are probably still going to have dynamic out-of-order scheduling in hardware. However, your power-efficient in-order cores might use parallelism data embedded in that 4-bit prefix. Ideally, the hardware would have reservoir sampling for which branches are mispredicted and which instructions stall the pipeline. This information could be aggregated for a background process to re-optimize the binary, similar to what the current Android Runtime does. Efficient hardware branch tracing (perhaps a second stack where, when the tracing machine state bit is set, the destination of each conditional/indirect branch target is pushed, along with an interrupt when the trace gets full) might allow for efficient runtime re-optimization of binaries, including inlining of dynamic library code into the code hot spots. On a side note, I know RSIC-V at least at one point had a proposal for an extension to use the integer registers for floating-point. Does anyone have a feel for how costly it would be for register renaming logic to handle separate integer and floating point register files so that low-power implementations could use a single unified register file (perhaps without register renaming) and higher performance implementations (which would presumably have register renaming anyway) could use separate integer and floating-point register files? In general, I hope we can find ISA designs that leave room for both very-low power implementations that can push some of the work into software, and high-performance implementations that aren't hindered by the features that allow lower-power/simpler implementations.
- dragontamer 3y ago
  Xilinx Versal AI cores suggest that VLIW + SIMD is a winner. Speaking of which: so was AMD Terrascale: https://en.wikipedia.org/wiki/TeraScale_(microarchitecture) https://en.wikipedia.org/wiki/TeraScale_(microarchitecture) ----------- It just needs to be in the right situation. GPUs almost entirely compute inside of register space, while FPGAs have carefully laid out memory exactly perfectly for their compute, so once again VLIW can fly. SIMD is an important technique to scale vs GPUs, because... well... SIMD is basically free scaling so might as well get it. So that's the combo IMO. Remove the random memory delays associated with general purpose compute, have a crap-ton of register space to never touch memory under normal circumstances, have huge kernels for DSP / GPU like tasks, and let the machine fly. ---------- I think it so happens that SIMD-parallelism was easier than expected, while VLIW-parallelism is harder than expected. So SIMD takes priority, but both should happen in these use cases. If you consider that the 2000s period of AMD64 vs IA-64 going on, the AMD systems focused on SSE / SIMD parallelism to provide us with the high-performance multimedia DivX decoders and whatnot (and other multimedia problems that consumers needed SIMD compute for back in the day). That parallelism was enough to be competitive vs IA-64. "General purpose CPUs" are so RAM-latency limited in practice. So out-of-order machines that can find work while waiting for RAM (again: AMD64) get a benefit.
  - touisteur 3y ago
    Carefully waiting to see whether Versal actually ends up making money for Xilinx and whether they stabilize on something after the shotgun approach of the first Versal iteration.
- chasil 3y ago
  The first iteration of Itanium, known as Merced, was truly awful, and this was Intel's fault. The second iteration, McKinley, was designed by HP and had more reasonable performance, but not enough to stop AMD64. A big problem was that a quality compiler was hard to find, and without it everything just ran very slowly.
- jcranmer 3y ago
  VLIW is one of those things that looks good on paper but just doesn't work that well in practice, and there's two main reasons for this. The first issue is the static versus dynamic scheduling problem. Static scheduling first requires a sufficiently smart compiler to output the correct static schedule. But you're also limited in your ability to parallelize based on what you can divine statically. Some operations have fundamentally dynamic execution times (memory operations, branches, and division operations are the most common of these), and if you guess wrong as to how long they're going to take, you're going to force the computer to do nothing where a dynamic scheduler could have slotted other work in instead Additionally, you can only schedule work to be done in parallel within a relatively small scope, largely a basic block, and definitely on function boundaries. Functions like sin and cos boil down to polynomial evaluation, a chain of ~5 FMAs that have to be executed sequentially. Even if you have two FMA execution units, a static scheduler is forced to execute sin(x) * cos(y) serially, whereas a dynamic scheduler can usually get some overlap between the FMA chains. A sufficiently smart compiler that can create the optimal static schedule for a VLIW chip can also emit the code so that the superscalar chip will dynamically choose the optimal schedule, so a VLIW chip ends up getting no better throughput than a regular superscalar chip, and if the stars align less well, will likely get worse throughput. The other fundamental issue with VLIW is that it forces you to expose more of your microarchitectural details at the ISA level, which makes it harder to adjust those details (e.g., add more execution units) should you desire to in later chip iterations. Now Itanium does have a design which ameliorates this to a degree, but the cost of such a design is largely bringing back the transistor- and power-hungry aspects of the superscalar chip into your VLIW design, at which point you start asking how much you're really saving.
- pjmlp 3y ago
  I am convinced they only failed because AMD was allowed to come up with AMD64. Had it not been for them, Intel and HP would have managed to push Itanium no matter what.
sneed_chucker 3y ago
From the thread: > I'm a little bias because my company is a re-sellers of the HP Itanium ia64 hardware (RX & ZX boxes), as well as PA-RISC. For that reason, I would hate to see it fade away in any sector. The ia64 platform is still widely used with HP-UX Unix and Open VMS users worldwide. This hardware is embedded in most every data center and large and medium companies that have been around since the 80s/90s, its probably the oldest box they have in there but its the one thats in the corner running for 20 years, long before most people started working there. Can anyone speak to how true this is? Are there still a bunch of companies that run some critical service using machines/OSs like this? I have a morbid curiosity about "obsolete" tech still being in everyday use.
- jamesfmilne 3y ago
  It may be true, but anyone who's still running missing-critical applications on those machines won't migrate to Linux. If they moved to Linux they'd surely just dump the hardware too.
- ch_123 3y ago
  Yes, there are companies who are still using HP-UX and VMS in production on IA64. The current maintainers of VMS have created an x86 port so I would expect VMS sites will probably move over to it in the coming years. HP-UX is stuck on IA64, meaning that anyone using that platform has a more risky port to Linux if they want to move off IA64.
- chasil 3y ago
  VMS has a reputation for clustering. At one point, [I heard that] Oracle bought clustering technology out of VMS that was used in making RAC (Real Application Cluster), and ditching Parallel Server. VMS for x86-64 has just emerged for production deployment in the past year. I don't know if it has the OS-level emulation of previous architectures (macro32 and vest, if I am correct in the VAX components, and their Alpha and Itanium equivalents), but the most modern VMS installations are on Itanium. Intel was a big user of VMS in the past, although I don't know if that remains true.
  - 3y ago
jauntywundrkind 3y ago
Is there an ok emulator anywhere? Does qemu have support?
- basementcat 3y ago
  There is ski but it may be tricky to build on newer kernels because of changes in syscalls. https://github.com/trofi/ski https://github.com/trofi/ski
  - jauntywundrkind 3y ago
    Nice, thanks. It's a pointless optimization problem but it'd be interesting to futz around & see what either compiler techniques or weird things we could get AI to generate for this architecture. I always wonder if there's some kind of strange explicit smt or on-thr-fly instruction building or swapping... dynamically melding coroutines into ones main code.
- ChickeNES 3y ago
  There's a partial usermode emulation fork of qemu out there, but it is very old. I've meant to look into seeing what is needed to flesh out the softmmu emulation, but it would be rough going as I don't have Itanium hardware to test against. Simics also used to support Itanium years ago, but unfortunately that support doesn't exist in the version that Intel released to the public a few years back
brycewray 3y ago
Might want to change “Arch” to “Architecture” in the headline, since this is about Linux, after all. :-)
- redundantly 3y ago
  I was confused, wondering why Arch Linux was dropping support for it.
  - alwayslikethis 3y ago
    I thought it was Arch, but then I remembered Arch didn't support it at least officially in the first place.
trebligdivad 3y ago
I have fond memories of ia64 code optimisation at a startup in the mid 2000's; it was a fun [but slow] architecture. I have less fond memories of lifting the Intel Tiger systems, Intel would happily send out to devs in the desperate attempts to get them to port anything to Itanium.
- touisteur 3y ago
  Well at least they tried that (sending dev kits with HW) for Itanium. Wish they'd done some of that on their last (PVC) GPU run.
extraduder_ire 3y ago
I guess that's it for rearranging the deckchairs on the Itanic.

69 points by ndesaulniers 3y ago

Imagine HP & Intel execute on Itanium to perfection: Were they still doomed to failure, or is VLIW/EPIC actually a viable competitive approach for high-performance general purpose CPUs? The market evidence suggests it’s not, but there haven’t been that many well-funded, competent attempts (Itanium, Transmeta, ???) so the sample size is small.

KMag 3y ago
Code density is an issue, requiring huge caches for Itanium to remain competitive. These large caches consume lots of die area, making the chips both expensive and power-hungry. As I and others have pointed out, instead of a constant 3 instructions in 128 bits, a variable number of instructions in 64 bits should get you competitive code densities. (Say, have a 4-bit prefix that indicates if 1x60 bits, 2x30 bits, 4x15 bits, or 5x12 bits instructions are packed in the remaining 60 bits. For 12-bit instructions, you probably have the first instruction only able to store into r1 or r2, the second instruction only able to store into r3 or r4, etc. so that you have a 6-bit opcode, 1 bit for destination register and 5 bits for operand register. Plenty of old processors had an accumulator that was the implicit destination for most instructions.) Some of the other 4-bit prefixes would be used to indicate a combination of instruction widths and instruction parallelism to allow lower-powered in-order superscalar implementations. Fixed alignment of 64-bit bundles of variable-width instructions makes parallel instruction decoding cheaper (and makes static analysis easier/finding ROP exploit targets very slightly harder). Your high-performance cores are probably still going to have dynamic out-of-order scheduling in hardware. However, your power-efficient in-order cores might use parallelism data embedded in that 4-bit prefix. Ideally, the hardware would have reservoir sampling for which branches are mispredicted and which instructions stall the pipeline. This information could be aggregated for a background process to re-optimize the binary, similar to what the current Android Runtime does. Efficient hardware branch tracing (perhaps a second stack where, when the tracing machine state bit is set, the destination of each conditional/indirect branch target is pushed, along with an interrupt when the trace gets full) might allow for efficient runtime re-optimization of binaries, including inlining of dynamic library code into the code hot spots. On a side note, I know RSIC-V at least at one point had a proposal for an extension to use the integer registers for floating-point. Does anyone have a feel for how costly it would be for register renaming logic to handle separate integer and floating point register files so that low-power implementations could use a single unified register file (perhaps without register renaming) and higher performance implementations (which would presumably have register renaming anyway) could use separate integer and floating-point register files? In general, I hope we can find ISA designs that leave room for both very-low power implementations that can push some of the work into software, and high-performance implementations that aren't hindered by the features that allow lower-power/simpler implementations.
dragontamer 3y ago
Xilinx Versal AI cores suggest that VLIW + SIMD is a winner. Speaking of which: so was AMD Terrascale: https://en.wikipedia.org/wiki/TeraScale_(microarchitecture) https://en.wikipedia.org/wiki/TeraScale_(microarchitecture) ----------- It just needs to be in the right situation. GPUs almost entirely compute inside of register space, while FPGAs have carefully laid out memory exactly perfectly for their compute, so once again VLIW can fly. SIMD is an important technique to scale vs GPUs, because... well... SIMD is basically free scaling so might as well get it. So that's the combo IMO. Remove the random memory delays associated with general purpose compute, have a crap-ton of register space to never touch memory under normal circumstances, have huge kernels for DSP / GPU like tasks, and let the machine fly. ---------- I think it so happens that SIMD-parallelism was easier than expected, while VLIW-parallelism is harder than expected. So SIMD takes priority, but both should happen in these use cases. If you consider that the 2000s period of AMD64 vs IA-64 going on, the AMD systems focused on SSE / SIMD parallelism to provide us with the high-performance multimedia DivX decoders and whatnot (and other multimedia problems that consumers needed SIMD compute for back in the day). That parallelism was enough to be competitive vs IA-64. "General purpose CPUs" are so RAM-latency limited in practice. So out-of-order machines that can find work while waiting for RAM (again: AMD64) get a benefit.
- touisteur 3y ago
  Carefully waiting to see whether Versal actually ends up making money for Xilinx and whether they stabilize on something after the shotgun approach of the first Versal iteration.
chasil 3y ago
The first iteration of Itanium, known as Merced, was truly awful, and this was Intel's fault. The second iteration, McKinley, was designed by HP and had more reasonable performance, but not enough to stop AMD64. A big problem was that a quality compiler was hard to find, and without it everything just ran very slowly.
jcranmer 3y ago
VLIW is one of those things that looks good on paper but just doesn't work that well in practice, and there's two main reasons for this. The first issue is the static versus dynamic scheduling problem. Static scheduling first requires a sufficiently smart compiler to output the correct static schedule. But you're also limited in your ability to parallelize based on what you can divine statically. Some operations have fundamentally dynamic execution times (memory operations, branches, and division operations are the most common of these), and if you guess wrong as to how long they're going to take, you're going to force the computer to do nothing where a dynamic scheduler could have slotted other work in instead Additionally, you can only schedule work to be done in parallel within a relatively small scope, largely a basic block, and definitely on function boundaries. Functions like sin and cos boil down to polynomial evaluation, a chain of ~5 FMAs that have to be executed sequentially. Even if you have two FMA execution units, a static scheduler is forced to execute sin(x) * cos(y) serially, whereas a dynamic scheduler can usually get some overlap between the FMA chains. A sufficiently smart compiler that can create the optimal static schedule for a VLIW chip can also emit the code so that the superscalar chip will dynamically choose the optimal schedule, so a VLIW chip ends up getting no better throughput than a regular superscalar chip, and if the stars align less well, will likely get worse throughput. The other fundamental issue with VLIW is that it forces you to expose more of your microarchitectural details at the ISA level, which makes it harder to adjust those details (e.g., add more execution units) should you desire to in later chip iterations. Now Itanium does have a design which ameliorates this to a degree, but the cost of such a design is largely bringing back the transistor- and power-hungry aspects of the superscalar chip into your VLIW design, at which point you start asking how much you're really saving.
pjmlp 3y ago
I am convinced they only failed because AMD was allowed to come up with AMD64. Had it not been for them, Intel and HP would have managed to push Itanium no matter what.

sneed_chucker 3y ago

From the thread: > I'm a little bias because my company is a re-sellers of the HP Itanium ia64 hardware (RX & ZX boxes), as well as PA-RISC. For that reason, I would hate to see it fade away in any sector. The ia64 platform is still widely used with HP-UX Unix and Open VMS users worldwide. This hardware is embedded in most every data center and large and medium companies that have been around since the 80s/90s, its probably the oldest box they have in there but its the one thats in the corner running for 20 years, long before most people started working there. Can anyone speak to how true this is? Are there still a bunch of companies that run some critical service using machines/OSs like this? I have a morbid curiosity about "obsolete" tech still being in everyday use.

jamesfmilne 3y ago
It may be true, but anyone who's still running missing-critical applications on those machines won't migrate to Linux. If they moved to Linux they'd surely just dump the hardware too.
ch_123 3y ago
Yes, there are companies who are still using HP-UX and VMS in production on IA64. The current maintainers of VMS have created an x86 port so I would expect VMS sites will probably move over to it in the coming years. HP-UX is stuck on IA64, meaning that anyone using that platform has a more risky port to Linux if they want to move off IA64.
chasil 3y ago
VMS has a reputation for clustering. At one point, [I heard that] Oracle bought clustering technology out of VMS that was used in making RAC (Real Application Cluster), and ditching Parallel Server. VMS for x86-64 has just emerged for production deployment in the past year. I don't know if it has the OS-level emulation of previous architectures (macro32 and vest, if I am correct in the VAX components, and their Alpha and Itanium equivalents), but the most modern VMS installations are on Itanium. Intel was a big user of VMS in the past, although I don't know if that remains true.
- 3y ago

jauntywundrkind 3y ago

Is there an ok emulator anywhere? Does qemu have support?

basementcat 3y ago
There is ski but it may be tricky to build on newer kernels because of changes in syscalls. https://github.com/trofi/ski https://github.com/trofi/ski
- jauntywundrkind 3y ago
  Nice, thanks. It's a pointless optimization problem but it'd be interesting to futz around & see what either compiler techniques or weird things we could get AI to generate for this architecture. I always wonder if there's some kind of strange explicit smt or on-thr-fly instruction building or swapping... dynamically melding coroutines into ones main code.
ChickeNES 3y ago
There's a partial usermode emulation fork of qemu out there, but it is very old. I've meant to look into seeing what is needed to flesh out the softmmu emulation, but it would be rough going as I don't have Itanium hardware to test against. Simics also used to support Itanium years ago, but unfortunately that support doesn't exist in the version that Intel released to the public a few years back

brycewray 3y ago

Might want to change “Arch” to “Architecture” in the headline, since this is about Linux, after all. :-)

redundantly 3y ago
I was confused, wondering why Arch Linux was dropping support for it.
- alwayslikethis 3y ago
  I thought it was Arch, but then I remembered Arch didn't support it at least officially in the first place.

trebligdivad 3y ago

I have fond memories of ia64 code optimisation at a startup in the mid 2000's; it was a fun [but slow] architecture. I have less fond memories of lifting the Intel Tiger systems, Intel would happily send out to devs in the desperate attempts to get them to port anything to Itanium.

touisteur 3y ago
Well at least they tried that (sending dev kits with HW) for Itanium. Wish they'd done some of that on their last (PVC) GPU run.

extraduder_ire 3y ago

I guess that's it for rearranging the deckchairs on the Itanic.