5 ms·

> I suppose a similar analogy would be doing everything in x86 with SIMD instructions and not using the scalar set at all. With ever wider SIMD units this is a

> I suppose a similar analogy would be doing everything in x86 with SIMD instructions and not using the scalar set at all. With ever wider SIMD units this is actually happening to some extent. With AVX512 (e.g. on KNL) you can do 64 operations (single precision FLOPs) in vector units in the same amount of time as you can do 1 scalar operation. Combined with the low clock speed of the KNL, you really don't want to be doing scalar operations if you care at all about performance. Of course SIMD isn't the same as VLIW, since the operation you're doing has to be the same in all vector lanes, but with masking support in AVX512, it is getting a little closer to that. Also compilers have gotten quite a bit smarter and with hardware die shrinks taking longer, I wouldn't be at all surprised if VLIW ISAs were to make a comeback. I guess the Itanium was a bit ahead of its time.

deepnotderp 9y ago
This, recent compiler advances in the past two decades have actually made VLIW somewhat tractable, but since the memory hierarchy and data movement wall consumes most of the power nowadays, the benefits of VLIW are mitigated.
- dom0 9y ago
  Even if that were so, things are moving away from VLIW. E.g. AMD switched from VLIW to a more standard load-store SIMT approach a couple years ago, and even though it's more complex for the GPU, it increased system efficiency (esp. for workloads that their compiler was not well tuned for).
- CalChris 9y ago
  Trace scheduling started with Fisher 81 [1] and Ellis 85 [2]. Trace scheduling is what made VLIW even possible if not exactly tractable. That was 30+ years ago. What recent advances have made VLIW any more tractable than trace scheduling already did? BTW, trace scheduling works for scheduling superscalar processors as well. VLIW never went away. It's used in embedded, the TriMedia processors. It's used in the REX Computing NEO chip. The Mill CPU is a VLIW of sorts. I'm not anti-VLIW but I don't know of any recent breakthroughs that make it any more tractable now for non-embedded, non-HPC general purpose computing. [1] Trace Scheduling: A Technique for Global Microcode Compaction https://pdfs.semanticscholar.org/5698/09af0fcbe5a42371cea8d30932d7d8f4a933.pdf https://pdfs.semanticscholar.org/5698/09af0fcbe5a42371cea8d3... [2] Bulldog: a compiler for VLIW architectures http://dl.acm.org/citation.cfm?id=912347 http://dl.acm.org/citation.cfm?id=912347
  - trsohmers 9y ago
    Designer of the Neo here, and owner of the California "VLIW" license plate. As you can guess, I am a die hard VLIW advocate, and a strong believer that the original promises of VLIW (Drastically simpler decode logic, implicit instruction level parallelism, virtually no control/data hazards on chip). VLIW has gotten an extremely bad rap outside of the embedded space due to Itanium, which I strongly contend was not a VLIW in spirit (which you cans see me talk in more depth on here: https://youtu.be/ki6jVXZM2XU?t=441 https://youtu.be/ki6jVXZM2XU?t=441). Itanium introduced a ton of indeterminism by having static scheduling unfriendly things like branch prediction, variable latency caches, and still trying to have some level of support for x86. The core problem for Itanium is that it was impossible for the compiler to make good decisions in the vast majority of applications since the compiler could not know exactly where and when things in memory would be. I believe the underlying failure point has been computer architects for failed VLIW architectures not realizing that fancy bells and whistles and "cleverly complex" hardware designs is the exact opposite of what you want when the compiler needs to be able to do purely static allocation. Adding fancy dynamic pieces in hardware to account from this (seen in a lot of places with Itanium) just makes the problem worse. The main improvement we have made with the Neo architecture is we have hard (exact cycle count) guarantees on all memory movements on and off the chip, which actually gives the compiler the information it needs to make good decisions and very condensed code. I think the jury is out on the applicability for "general purpose" code on VLIWs... it is not something we at REX really care about at the moment, and I think RISC-V is a great solution with improvements over x86 and ARM.
    - CyberDildonics 9y ago
      What are some examples of VLIW succeeding? (I haven't watched your video yet)
      - trsohmers 9y ago
        The two best application cases of instruction level parallelism that we have at the moment are our FFT and DGEMM, where our 64 bit instruction word contains the operations for doing a double precision fused multiply add, two 128 bit load/stores, and an ALU op for address calculation. The inner loops of these programs are able to get up to 4 instructions per cycle (5 if you count the FMA as 2 ops, which is what everyone else does but I think is cheating), and compared to a superscalar architecture, uses very little space in local memory (fitting 4 operations in only 64 bits is awesome!). Compared to the latest 14/16nm offerings from Intel and NVIDIA, we have a 25x energy efficiency advantage for our FFT while our test chip is still on 28nm. These are the same reasons why DSPs have been VLIW since the 90s, and our initial applications overlap a lot. It has been sad to see the trend of TI and other DSP makers going down the path of adding hardware caching (that reduces determinism and energy efficiency) to try to "make it easier for programmers" when they already have a decent enough architecture, but a lack of good compiler tools to take advantage of it.
- thesz 9y ago
  Make VLIW running at <just one digit>0MHz. First, your silicon will be smaller - no need for pipelines, even double precision floating point multiply can be done in one clock cycle. Less registers, less silicon area. You can internally do that operation at higher frequencies than main core, timesharing die area for SIMD operations - saving silicon in other way. Second, your compiler will be simpler - no need for NP-hard algorithms to schedule operations in the basic block, no need for fancy software pipelining for loop code to be warmed up and/or cooled down. Third, your CPU won't be faster than memory. Your code won't get big hit from missed cache access. And cache will be used to save energy, not the time. That's exactly the reasons behind one of exascale CPU architectures. It was long ago - circa 2012. I remember they managed to get FLOP/W ratios much higher than Intel's offering at the time.
userbinator 9y ago
With ever wider SIMD units this is actually happening to some extent. With AVX512 (e.g. on KNL) you can do 64 operations (single precision FLOPs) in vector units in the same amount of time as you can do 1 scalar operation I was talking about code which simply can't be parallelised because it is branchy or has long and unavoidable dependency chains. In those cases using SIMD instructions will only make the code larger (thus more cache-consuming) and arguably more power-consuming too, since 63/64 of those operations aren't contributing anything to the computation.
- KenoFischer 9y ago
  Yes, I understand. What I meant was that you don't want to be running those kinds of codes on a KNL, because the performance is absolutely terrible. Nevertheless, Intel is producing the KNL, so there's now a market for this kind of thing, which may not have been there 15 years ago.
CyberDildonics 9y ago
Actually it is 16 floating point operations in one AVX512 instruction and each core has two units, making 32 flops if you are running two or more threads (each core can do 4 way SMT but can also only decode two instructions per cycle).
- KenoFischer 9y ago
  I was counting FMA as 2 operations ;).
  - CyberDildonics 9y ago
    That's Intel marketing and is somewhere between deceptive and wrong. Intel uses that to claim inflated maximum flops, with the huge aside that you would have to be doing nothing but fused multiply add, which is absurd.
    - jabl 9y ago
      AFAICS everybody has been doing that long before Intel, so in their defense they're just following precedent. But yes, outside of carefully tuned kernels (e.g. GEMM and FFT), you're not going to see a 2x speedup. But that's no different from many other ISA extensions.

deepnotderp 9y ago

This, recent compiler advances in the past two decades have actually made VLIW somewhat tractable, but since the memory hierarchy and data movement wall consumes most of the power nowadays, the benefits of VLIW are mitigated.

dom0 9y ago
Even if that were so, things are moving away from VLIW. E.g. AMD switched from VLIW to a more standard load-store SIMT approach a couple years ago, and even though it's more complex for the GPU, it increased system efficiency (esp. for workloads that their compiler was not well tuned for).
CalChris 9y ago
Trace scheduling started with Fisher 81 [1] and Ellis 85 [2]. Trace scheduling is what made VLIW even possible if not exactly tractable. That was 30+ years ago. What recent advances have made VLIW any more tractable than trace scheduling already did? BTW, trace scheduling works for scheduling superscalar processors as well. VLIW never went away. It's used in embedded, the TriMedia processors. It's used in the REX Computing NEO chip. The Mill CPU is a VLIW of sorts. I'm not anti-VLIW but I don't know of any recent breakthroughs that make it any more tractable now for non-embedded, non-HPC general purpose computing. [1] Trace Scheduling: A Technique for Global Microcode Compaction https://pdfs.semanticscholar.org/5698/09af0fcbe5a42371cea8d30932d7d8f4a933.pdf https://pdfs.semanticscholar.org/5698/09af0fcbe5a42371cea8d3... [2] Bulldog: a compiler for VLIW architectures http://dl.acm.org/citation.cfm?id=912347 http://dl.acm.org/citation.cfm?id=912347
- trsohmers 9y ago
  Designer of the Neo here, and owner of the California "VLIW" license plate. As you can guess, I am a die hard VLIW advocate, and a strong believer that the original promises of VLIW (Drastically simpler decode logic, implicit instruction level parallelism, virtually no control/data hazards on chip). VLIW has gotten an extremely bad rap outside of the embedded space due to Itanium, which I strongly contend was not a VLIW in spirit (which you cans see me talk in more depth on here: https://youtu.be/ki6jVXZM2XU?t=441 https://youtu.be/ki6jVXZM2XU?t=441). Itanium introduced a ton of indeterminism by having static scheduling unfriendly things like branch prediction, variable latency caches, and still trying to have some level of support for x86. The core problem for Itanium is that it was impossible for the compiler to make good decisions in the vast majority of applications since the compiler could not know exactly where and when things in memory would be. I believe the underlying failure point has been computer architects for failed VLIW architectures not realizing that fancy bells and whistles and "cleverly complex" hardware designs is the exact opposite of what you want when the compiler needs to be able to do purely static allocation. Adding fancy dynamic pieces in hardware to account from this (seen in a lot of places with Itanium) just makes the problem worse. The main improvement we have made with the Neo architecture is we have hard (exact cycle count) guarantees on all memory movements on and off the chip, which actually gives the compiler the information it needs to make good decisions and very condensed code. I think the jury is out on the applicability for "general purpose" code on VLIWs... it is not something we at REX really care about at the moment, and I think RISC-V is a great solution with improvements over x86 and ARM.
  - CyberDildonics 9y ago
    What are some examples of VLIW succeeding? (I haven't watched your video yet)
    - trsohmers 9y ago
      The two best application cases of instruction level parallelism that we have at the moment are our FFT and DGEMM, where our 64 bit instruction word contains the operations for doing a double precision fused multiply add, two 128 bit load/stores, and an ALU op for address calculation. The inner loops of these programs are able to get up to 4 instructions per cycle (5 if you count the FMA as 2 ops, which is what everyone else does but I think is cheating), and compared to a superscalar architecture, uses very little space in local memory (fitting 4 operations in only 64 bits is awesome!). Compared to the latest 14/16nm offerings from Intel and NVIDIA, we have a 25x energy efficiency advantage for our FFT while our test chip is still on 28nm. These are the same reasons why DSPs have been VLIW since the 90s, and our initial applications overlap a lot. It has been sad to see the trend of TI and other DSP makers going down the path of adding hardware caching (that reduces determinism and energy efficiency) to try to "make it easier for programmers" when they already have a decent enough architecture, but a lack of good compiler tools to take advantage of it.
thesz 9y ago
Make VLIW running at <just one digit>0MHz. First, your silicon will be smaller - no need for pipelines, even double precision floating point multiply can be done in one clock cycle. Less registers, less silicon area. You can internally do that operation at higher frequencies than main core, timesharing die area for SIMD operations - saving silicon in other way. Second, your compiler will be simpler - no need for NP-hard algorithms to schedule operations in the basic block, no need for fancy software pipelining for loop code to be warmed up and/or cooled down. Third, your CPU won't be faster than memory. Your code won't get big hit from missed cache access. And cache will be used to save energy, not the time. That's exactly the reasons behind one of exascale CPU architectures. It was long ago - circa 2012. I remember they managed to get FLOP/W ratios much higher than Intel's offering at the time.

userbinator 9y ago

With ever wider SIMD units this is actually happening to some extent. With AVX512 (e.g. on KNL) you can do 64 operations (single precision FLOPs) in vector units in the same amount of time as you can do 1 scalar operation I was talking about code which simply can't be parallelised because it is branchy or has long and unavoidable dependency chains. In those cases using SIMD instructions will only make the code larger (thus more cache-consuming) and arguably more power-consuming too, since 63/64 of those operations aren't contributing anything to the computation.

KenoFischer 9y ago
Yes, I understand. What I meant was that you don't want to be running those kinds of codes on a KNL, because the performance is absolutely terrible. Nevertheless, Intel is producing the KNL, so there's now a market for this kind of thing, which may not have been there 15 years ago.

CyberDildonics 9y ago

Actually it is 16 floating point operations in one AVX512 instruction and each core has two units, making 32 flops if you are running two or more threads (each core can do 4 way SMT but can also only decode two instructions per cycle).

KenoFischer 9y ago
I was counting FMA as 2 operations ;).
- CyberDildonics 9y ago
  That's Intel marketing and is somewhere between deceptive and wrong. Intel uses that to claim inflated maximum flops, with the huge aside that you would have to be doing nothing but fused multiply add, which is absurd.
  - jabl 9y ago
    AFAICS everybody has been doing that long before Intel, so in their defense they're just following precedent. But yes, outside of carefully tuned kernels (e.g. GEMM and FFT), you're not going to see a 2x speedup. But that's no different from many other ISA extensions.