7 ms·

IMO VLIW is an absurdly bad choice for a general purpose processor. It requires baking in a huge amount of low level micro-architectural details into the compil

IMO VLIW is an absurdly bad choice for a general purpose processor. It requires baking in a huge amount of low level micro-architectural details into the compiler / generated code. Which obviously leads to problems with choosing what hardware generation to optimize for / not being able to generate good code for future architectures. And the compiler doesn't even come close to having as much information as the CPU has. Which basically means that most of the VLIW stuff just ends up needing to be broken up inside the CPU for good performance.

darksaints 6y ago
Traditional compiler techniques may have struggled with maintaining code for different architectures, but a lot has changed in the last 15 years. The rise of widely used IR languages has led to compilers that support dozens of architectures and hundreds of instruction sets. And they are getting better all the time. The compiler has nearly all of the information that the CPU has, and it has orders of magnitude more. At best, your CPU can think a couple dozen cycles ahead of what it is currently executing. The compiler can see the whole program, can analyze it using dozens of methodologies and models, and can optimize accordingly. Something like Link Time Optimization can be done trivially with a compiler, but it would take an army of engineers decades of work to be able to implement in hardware.
- dragontamer 6y ago
  > At best, your CPU can think a couple dozen cycles ahead of what it is currently executing. The 200-sized reorder buffer says otherwise. Loads/stores can be reordered for 200+ different concurrent objects on modern Intel skylake (2015 through 2020) CPUs. And its about to get a bump to 300+ sized reorder buffers in Icelake. Modern CPUs are designed to "think ahead" almost the entirety of DDR4 RAM Latency, allowing reordering of instructions to keep the CPU pipes as full as possible (at least, if the underlying assembly code has enough ILP to fill the pipelines while waiting for RAM). > Something like Link Time Optimization can be done trivially with a compiler, but it would take an army of engineers decades of work to be able to implement in hardware. You might be surprised at what the modern Branch predictor is doing. If your "call rax" indirect call constantly calls the same location, the branch predictor will remember that location these days.
  - KMag 6y ago
    With proper profiling (say, reservoir sampling of instructions causing pipeline stalls), and dynamic recompilation/reoptimization like IBM's project DAISY / HP's Dynamo, you may get performance near a modern out-of-order desktop processor at the power budget of a modern in-order low-power chip. You get instructions scheduled based on actual dynamically measured usage patterns, but you don't pay for dedicated circuits to do it, and you don't re-do those calculations in hardware for every single instruction executed. It's not a guaranteed win, but I think it's worth exploring.
    - dragontamer 6y ago
      But once you do that, then you hardware optimize the interpreter, and then its no longer called a "dynamic recompiler", but instead a "frontend to the microcode". :-)
      - KMag 6y ago
        No doubt there is still room for a power-hungry out-of-order speed demon of an implementation, but you need to leave the door open for something with approximately the TDP of a very-low-power in-order-processor with performance closer to an out-of-order machine.
  - branko_d 6y ago
    Neo: What are you trying to tell me? That I can dodge "call rax"? Morpheus: No, Neo. I'm trying to tell you that when you're ready, you won't need "call rax". --- Compiler has access to optimizations that are at the higher level of abstraction than what CPU can do. For example, the compiler can eliminate the call completely (i.e. inline the function), or convert a dynamic dispatch into static (if it can prove that an object will always have a specific type at the call site), or decide where to favor small code over fast code (via profile-guided optimization), or even switch from non-optimized code (but with short start-up time) to optimized code mid-execution (tiered compilation in JITs), move computation outside loops (if it can prove that the result is the same in all iterations), and many other things...
- formerly_proven 6y ago
  > The compiler has nearly all of the information that the CPU has, and it has orders of magnitude more. The CPU has something the compiler can never have. Runtime information. That's why VLIW works great for DSP which is 99.9 % fixed access patterns, while being bad for general purpose code.
dragontamer 6y ago
VLIW was the best implementation (20 years ago) of instruction level parallelism. But what have we learned in these past 20 years? * Computers will continue to become more parallel -- AMD Zen2 has 10 execution pipelines, supporting 4-way decode and 6-uop / clock tick dispatch per core, with somewhere close to 200 registers for renaming / reordering instructions. Future processors will be bigger and more parallel, Ice Lake is rumored to have over 300-renaming registers. * We need assembly code that scales to all different processors of different sizes. Traditional assembly code is surprisingly good (!!!) at scaling, thanks to "dependency cutting" with instructions like "xor eax, eax". * Compilers can understand dependency chains, "cut them up" and allow code to scale. The same code optimized for Intel Sandy Bridge (2011-era chips) will continue to be well-optimized for Intel Icelake (2021 era) ten years later, thanks to these dependency-cutting compilers. I think a future VLIW chip can be made that takes advantage of these facts. But it wouldn't look like Itanium. ---------- EDIT: I feel like "xor eax, eax" and other such instructions for "dependency cutting" are wasting bits. There might be a better way for encoding the dependency graph rather than entire instructions. Itanium's VLIW "packages" is too static. I've discussed NVidia's Volta elsewhere, which has 6-bit dependency bitmasks on every instruction. That's the kind of "dependency graph" information that a compiler can provide very easily, and probably save a ton on power / decoding.
- anarazel 6y ago
  I don't understand how an increase, including the implied variability, of CPU internal parallelism and VLIW benefits go together?
  - dragontamer 6y ago
    I'm talking about a mythical / mystical VLIW architecture. Obviously, older VLIW designs have failed in this regards... but I don't necessarily see "future" VLIW processors making the same mistake. Perhaps from your perspective, a VLIW architecture that fixes these problems wouldn't necessarily be VLIW anymore. Which... could be true.
- moonchild 6y ago
  Have you seen the mill cpu?
KMag 6y ago
> And the compiler doesn't even come close to having as much information as the CPU has. Unless your CPU has a means for profiling where your pipeline stalls are coming from, combined with dynamic recompilation/reoptimization similar to IBM's project DAISY or HP's Dynamo. It's not going to do well as out-of-order CPUs that make instruction re-optimization decisions for every instruction, but I wouldn't rule out software-controlled dynamic re-optimization getting most of the performance benefits of out-of-order execution with a much smaller power budget, due to not re-doing those optimization calculations for every instruction. There are reasons most low-power implementations are in-order chips.
- csharptwdec19 6y ago
  I feel like what you describe is possible. When I think of what Transmeta was able to accomplish in the early 2000s just with CMS, certainly so.

darksaints 6y ago

Traditional compiler techniques may have struggled with maintaining code for different architectures, but a lot has changed in the last 15 years. The rise of widely used IR languages has led to compilers that support dozens of architectures and hundreds of instruction sets. And they are getting better all the time. The compiler has nearly all of the information that the CPU has, and it has orders of magnitude more. At best, your CPU can think a couple dozen cycles ahead of what it is currently executing. The compiler can see the whole program, can analyze it using dozens of methodologies and models, and can optimize accordingly. Something like Link Time Optimization can be done trivially with a compiler, but it would take an army of engineers decades of work to be able to implement in hardware.

dragontamer 6y ago
> At best, your CPU can think a couple dozen cycles ahead of what it is currently executing. The 200-sized reorder buffer says otherwise. Loads/stores can be reordered for 200+ different concurrent objects on modern Intel skylake (2015 through 2020) CPUs. And its about to get a bump to 300+ sized reorder buffers in Icelake. Modern CPUs are designed to "think ahead" almost the entirety of DDR4 RAM Latency, allowing reordering of instructions to keep the CPU pipes as full as possible (at least, if the underlying assembly code has enough ILP to fill the pipelines while waiting for RAM). > Something like Link Time Optimization can be done trivially with a compiler, but it would take an army of engineers decades of work to be able to implement in hardware. You might be surprised at what the modern Branch predictor is doing. If your "call rax" indirect call constantly calls the same location, the branch predictor will remember that location these days.
- KMag 6y ago
  With proper profiling (say, reservoir sampling of instructions causing pipeline stalls), and dynamic recompilation/reoptimization like IBM's project DAISY / HP's Dynamo, you may get performance near a modern out-of-order desktop processor at the power budget of a modern in-order low-power chip. You get instructions scheduled based on actual dynamically measured usage patterns, but you don't pay for dedicated circuits to do it, and you don't re-do those calculations in hardware for every single instruction executed. It's not a guaranteed win, but I think it's worth exploring.
  - dragontamer 6y ago
    But once you do that, then you hardware optimize the interpreter, and then its no longer called a "dynamic recompiler", but instead a "frontend to the microcode". :-)
    - KMag 6y ago
      No doubt there is still room for a power-hungry out-of-order speed demon of an implementation, but you need to leave the door open for something with approximately the TDP of a very-low-power in-order-processor with performance closer to an out-of-order machine.
- branko_d 6y ago
  Neo: What are you trying to tell me? That I can dodge "call rax"? Morpheus: No, Neo. I'm trying to tell you that when you're ready, you won't need "call rax". --- Compiler has access to optimizations that are at the higher level of abstraction than what CPU can do. For example, the compiler can eliminate the call completely (i.e. inline the function), or convert a dynamic dispatch into static (if it can prove that an object will always have a specific type at the call site), or decide where to favor small code over fast code (via profile-guided optimization), or even switch from non-optimized code (but with short start-up time) to optimized code mid-execution (tiered compilation in JITs), move computation outside loops (if it can prove that the result is the same in all iterations), and many other things...
formerly_proven 6y ago
> The compiler has nearly all of the information that the CPU has, and it has orders of magnitude more. The CPU has something the compiler can never have. Runtime information. That's why VLIW works great for DSP which is 99.9 % fixed access patterns, while being bad for general purpose code.

dragontamer 6y ago

VLIW was the best implementation (20 years ago) of instruction level parallelism. But what have we learned in these past 20 years? * Computers will continue to become more parallel -- AMD Zen2 has 10 execution pipelines, supporting 4-way decode and 6-uop / clock tick dispatch per core, with somewhere close to 200 registers for renaming / reordering instructions. Future processors will be bigger and more parallel, Ice Lake is rumored to have over 300-renaming registers. * We need assembly code that scales to all different processors of different sizes. Traditional assembly code is surprisingly good (!!!) at scaling, thanks to "dependency cutting" with instructions like "xor eax, eax". * Compilers can understand dependency chains, "cut them up" and allow code to scale. The same code optimized for Intel Sandy Bridge (2011-era chips) will continue to be well-optimized for Intel Icelake (2021 era) ten years later, thanks to these dependency-cutting compilers. I think a future VLIW chip can be made that takes advantage of these facts. But it wouldn't look like Itanium. ---------- EDIT: I feel like "xor eax, eax" and other such instructions for "dependency cutting" are wasting bits. There might be a better way for encoding the dependency graph rather than entire instructions. Itanium's VLIW "packages" is too static. I've discussed NVidia's Volta elsewhere, which has 6-bit dependency bitmasks on every instruction. That's the kind of "dependency graph" information that a compiler can provide very easily, and probably save a ton on power / decoding.

anarazel 6y ago
I don't understand how an increase, including the implied variability, of CPU internal parallelism and VLIW benefits go together?
- dragontamer 6y ago
  I'm talking about a mythical / mystical VLIW architecture. Obviously, older VLIW designs have failed in this regards... but I don't necessarily see "future" VLIW processors making the same mistake. Perhaps from your perspective, a VLIW architecture that fixes these problems wouldn't necessarily be VLIW anymore. Which... could be true.
moonchild 6y ago
Have you seen the mill cpu?

KMag 6y ago

> And the compiler doesn't even come close to having as much information as the CPU has. Unless your CPU has a means for profiling where your pipeline stalls are coming from, combined with dynamic recompilation/reoptimization similar to IBM's project DAISY or HP's Dynamo. It's not going to do well as out-of-order CPUs that make instruction re-optimization decisions for every instruction, but I wouldn't rule out software-controlled dynamic re-optimization getting most of the performance benefits of out-of-order execution with a much smaller power budget, due to not re-doing those optimization calculations for every instruction. There are reasons most low-power implementations are in-order chips.

csharptwdec19 6y ago
I feel like what you describe is possible. When I think of what Transmeta was able to accomplish in the early 2000s just with CMS, certainly so.