5 ms·

Something I've thought about a few times is how a fair number of smart folks who are in the know view VLIW as something that was tried and failed, so they're di

Something I've thought about a few times is how a fair number of smart folks who are in the know view VLIW as something that was tried and failed, so they're dismissive of new designs which use VLIW. For example Linus Torvalds has made this assertion a few times that I've seen. However, it seems to me (at least by their own telling) that their focus was almost entirely on trying to do x86 better than x86 and doesn't really seem like a straight forward or easy task. What I wonder about is VLIW doing its own thing (e.g. not trying to run Windows games) but running applications intentionally developed for it. Perhaps there are use cases where VLIW works very well... or at least well enough to a reasonable alternative value proposition.

throwaway_pdp09 6y ago
Saying a whole industry maybe did stuff wrong and they should try again is daft. Perhaps do some reading and save yourself going down the same path? VLIW blew up for obvious[0] reasons - memory access latency is a dynamic thing and can't be anticipated statically very well. Therefore an in-order VLIW chip with its basket of instructions (one of which being a mem access) will stall, the whole basket freezes, the subsequent baskets are held up in the queue, etc. That's why OOO processors won, a memory miss can to a degree be worked around by executing instructions ahead in the queue, if they're not dependent on the stalled instruction. [0] Blindingly obvious to me when it was pointed out, but it did need pointing out I have to admit.
- ghusbands 6y ago
  The folks working on the Mill seem to disagree. They have a VLIW-style architecture (up to 33 sub-instructions in an up-to-512-bit instruction) that they claim does not have the same drawbacks. That at least means that someone is still chasing the VLIW general purpose CPU dream.
  - throwaway_pdp09 6y ago
    I'm aware of the mill and that it's VLIW-ish. You're right, I will do some (re)-reading on it to remember how it works around that (although in a video ivan goddard does say if you hit main mem randomly, no architecture can save you, mill included). DSPs, and of course GPUs, do exist and again you're right, for special purpose but important niches of course, but I was referring to general purpose CPUs but didn't make that clear. Thanks.
  - drivebycomment 6y ago
    Mill has nothing to show for it. Mill has no answer for the unpredictable cache misses. Their belt idea is cute and interesting but inconsequential and irrelevant. The main challenge of the modern microprocessors is not in how many ALUs you run in parallel nor how you model and implement your register file, but how to keep the ALUs fed with data from the memory hierarchy. VLIW doesn't help you with that at all, which is why it is largely irrelevant and a tiny niche. For applications like GPU, you address the feeding problem with multiple threads through SMT/SIMT and multiple cores, and SIMD. For applications where a single thread performance is important, you use OOO and/or speculative execution to maximize the feeding of the data. If you only care about running a particular loop and you only need to meet a fixed cycle budget like DSP application, VLIW may make sense when you want to save some engineering time and you have relatively low volume. Once volume is high enough, you will likely choose an in-order superscalar instead, for better performance or even simpler implementation, ease of programming, better backward compatibility and higher instruction density. VLIW has become more of a niche even in DSP. E.g. see https://www.eetimes.com/a-case-for-superscalar-dsps/ https://www.eetimes.com/a-case-for-superscalar-dsps/ which is now almost 20 years old article. The basic idea behind VLIW of moving as much work as you can to the compiler is not wrong, but VLIW turned out to be a bad way to divide work between the pipeline and the compiler.
    - zozbot234 6y ago
      > For applications where a single thread performance is important, you use OOO and/or speculative execution to maximize the feeding of the data. OOO and speculative execution are very energy-intensive per unit of compute, and hard to implement securely. Even if a VLIW-ish cpu like the Mill is still ultimately bottlenecked by memory bandwidth, avoiding pretty much every known drawback of OOO is a huge deal.
      - drivebycomment 6y ago
        For the power-efficiency, in-order superscalars proved to be efficient enough - see various in-order superscalar ARMs. Even for OOO, various ARMs and Apple showed that they can get quite power efficient. Also, more importantly, VLIW is not a power efficient approach. It fundamentally wastes instruction density. It wastes the die space - superscalar + SMT are inherently more efficient when you have more than 1 thread, and VLIW is suboptimal for SMT. It also turns out in-order superscalars can match VLIW in ALU power efficiency by powering off unused ALUs just like VLIW can, and for applications where the power efficiency is more important, the control logic can be simplified enough (at the price of performance obviously) to add negligible power overhead. As for the security aspect of the speculative execution, OOO outperforms VLIW dramatically for a single thread performance even without any speculative execution. Moreover, since the discovery of Spectre, there's been plenty of research into how to do speculative execution without leaking: https://ieeexplore.ieee.org/abstract/document/8806916 https://ieeexplore.ieee.org/abstract/document/8806916 https://ieeexplore.ieee.org/abstract/document/8574559 https://ieeexplore.ieee.org/abstract/document/8574559 https://ieeexplore.ieee.org/abstract/document/9138957 https://ieeexplore.ieee.org/abstract/document/9138957 https://dl.acm.org/doi/abs/10.1145/3316781.3317914 https://dl.acm.org/doi/abs/10.1145/3316781.3317914 It's not yet clear what approach will actually win out and will be adopted, but it's clear there are multiple viable solutions such that this is a solvable problem. The whackamole of fixing / addressing individual spectre-like vulnerability likely will continue for a while - after all, processor folks didn't have that much time given the development cycle for new pipeline/architecture is measured in years - but it's unlikely that side-channel issue will make speculative execution not viable. Edit: s/SIMT/SMT/
trm42 6y ago
Please do remember that Torvalds worked for Transmeta which created and designed vliw CPUs from scratch. He's also really practical guy if you happen to remember about Minix micro-kernel VS monolithic Linux-kernel ;)
- Quequau 6y ago
  I remember all those things, it's why I mentioned Linus as someone worth listening to but who nevertheless might too quick to dismiss VLIW now. Anyway, there are a couple of efforts which include VLIW tech which are in currently in the development pipeline now. So maybe in the next few years we will see some interesting things with it.
fulafel 6y ago
Most applications of vliw are in dsp, gpu, video codec coprocessor type things. It's mature technology and successful outside general purpose CPUs. Transmeta was 25 years ago and trying to apply vliw in a novel way to run x86 code, a daring idea that didn't pay off very well.
- CalChris 6y ago
  It paid off just fine except in a monetary sense. They got to market with a low power fast x86 clone, arguably creating that low power market. However, for that thankless and difficult task of creating an X86 clone (x87 was a nightmare) they received the great prize of competing against Intel in both manufacturing and marketing. Moreover, they were a 32b machine in 2000 and 64b was right around the corner. Indeed when AMD64 came along, announced in 2000, it was simpler and forced Intel+HP to give up on Itanium. (Yes, Itanium had other problems but engineering problems can, to a degree, be surmounted. Market problems are harder.)
  - fulafel 6y ago
    It's always possible that they had just bad luck / timing, but why didn't anyone try/succeed with it since?
    - CalChris 6y ago
      Well a bunch of the Crusoe people did. They eventually got acquired by Nvidia and that’s what Denver is. Not VLIW though.
      - monocasa 6y ago
        I was pretty sure that both the Transmeta cores, as well as the Denver cores were VLIW internally.
      - fulafel 6y ago
        Yep, The OOO nature of the cpu might make the static scheduling less attractive for Denver. But I think the public info doesn't say either way vliw or not. Also it's now mainstream (Intel does it) to have an intermediate code storage format with the introduction of trace caches. I wonder what the usual format of those is.
celrod 6y ago
I think this was a great comment and discussion thread on OOO vs VLIW: https://news.ycombinator.com/item?id=24466593 https://news.ycombinator.com/item?id=24466593 The biggest takeaway for me was the importance of scalability. As OOO CPUs get better, they can become increasingly parallel when running the exact same assembly. While VLIW like Itanium was limited to 3 ops/instruction, which is already behind today's OOO CPUs. (Although Mill's 33 ops/instruction sounds like it'd take much longer to beat.) My takeaway was that NVidea's Volta looked on the right track, where it added 6-bit dependency bitmasks to instructions to make building the dependency graphs (primary cost to OOO) more efficient.
- microtherion 6y ago
  It seems to me that earlier RISC designs learned the same lesson with branch and load delay slots. They performed fine when introduced, but did not adapt well. All of this was tried to improve the speed of in-order CPUs. Once OOO became widespread, such hardware assists became pointless (And it would appear to me that the dependency assists you describe will similarly not stay relevant all that long).
- xscott 6y ago
  > While VLIW like Itanium was limited to 3 ops/instruction, which is already behind today's OOO CPUs Itanium could group 3 instructions per 128 bit "bundle", but multiple bundles could be run in parallel, and the compiler had to insert explicit "stops" when that wasn't allowed. The architecture was designed specifically so that future processors could run old code with more parallelism. https://en.wikipedia.org/wiki/Explicitly_parallel_instruction_computing#Moving_beyond_VLIW https://en.wikipedia.org/wiki/Explicitly_parallel_instructio...
- Quequau 6y ago
  Thanks for the link. I actually saw that when it was posted but there's new comments since then.
- marcosdumay 6y ago
  > As OOO CPUs get better, they can become increasingly parallel when running the exact same assembly. While VLIW like Itanium was limited to 3 ops/instruction, which is already behind today's OOO CPUs. There's no reason for the VLIW machine not to get wider with time. The fact that Itanium didn't speaks more of its economic unhealthiness than of anything else.
wyldfire 6y ago
> fair number of smart folks who are in the know view VLIW as something that was tried and failed, so they're dismissive of new designs which use VLIW. Snapdragon based phones have multiple Hexagon DSPs, they're VLIW that are also a commercial reality. It's fair to say that there's a lot fewer folks who know how to write hexagon assembly than other architectures but you could play this optimization game w/it too.
hedgehog 6y ago
Summarizing what I learned when I looked into this: VLIW directly exposes hardware parallelism to the programmer, making more throughput available without requiring complex scheduling machinery (save die area + power). This works well for code targeting a single architecture (like embedded) where timing is predictable (like DSP code). Essentially VLIW machine code embeds a bunch of very specific assumptions about processor's internal design. Here VLIW is alive and well for targets like a cell modem baseband or set top box video decoder chip that needs maximum efficiency and only runs the firmware from the manufacturer. General purpose CPUs don't fit this model. They need to support running the same code across chip variants and generations, each with varying amounts of internal parallelism, cache, etc, and instruction timing is not predictable or deterministic (memory and IO latency, cache residency, etc etc). Modern high performance CPUs have extraordinarily complex and clever machinery to process what user space programmers think of as "machine code" to find and use as much parallelism as possible. This includes speculative execution based on behavior observed at run time, something not possible even in a perfect VLIW compiler. Given this you're not likely to see VLIW instruction sets exposed to user space. This is consistent with Linus's view I think. Notably, as 3D accelerators became user-programmable VLIW stopped working well (see ATI/AMD GCN architecture rationale). What Transmeta did is try to combine the advantages of VLIW hardware and stability of existing serial x86 code by filling the gap with a JIT living below the kernel ("Code Morphing Software"). There are examples like HP Dynamo that suggest this is a workable idea and in practice the 1st gen Crusoe product was decent. There are some different ideas why they failed but at least part of the problem is that they got squeezed by Intel's manufacturing advantage: 20 years ago being a year or two ahead in fab process was more effective than having a better design. I haven't seen any strong case that the technical approach wouldn't work at least as well today with TSMC and Samsung now leading in fab. Interestingly the Denver architecture NVIDIA uses in their embedded products follows the Transmeta approach with an ARM front end and embedded JIT. Reportedly they were originally going to do x86 as well. I'm curious what's going to happen there now that they own Arm.
- Quequau 6y ago
  Hey thanks for this response. It's broadly consistent with my understanding, though obviously better articulated, and you did have some points I hadn't heard before. Last year's HotChips (or maybe even the year before) had a talk from a group that was working on a new design which is arguably VLIW (or has some elements of VLIW) and recently-ish they've been back in the news for having actual silicon in test. It's my hope that they are successful in their efforts, at least in some technical way. Lastly somewhere in the back of my head is the vague idea that Apple doesn't actually use ARM cores either but instead uses a continuation the technology they bought with PA Semi only with an "ARM front end". I've long forgotten where I heard that from though.
guenthert 6y ago
Isn't an interesting aspect of VLIW CPUs, that they don't use branch prediction and hence are not susceptible to SPECTRE class of security vulnerabilities? I'm actually surprised that not more is made out of that. Afaics, only Secure64 is tooting that horn: https://secure64.com/2018/01/08/secure64-dns-family-not-subject-to-spectre-and-meltdown/ https://secure64.com/2018/01/08/secure64-dns-family-not-subj...

throwaway_pdp09 6y ago

Saying a whole industry maybe did stuff wrong and they should try again is daft. Perhaps do some reading and save yourself going down the same path? VLIW blew up for obvious[0] reasons - memory access latency is a dynamic thing and can't be anticipated statically very well. Therefore an in-order VLIW chip with its basket of instructions (one of which being a mem access) will stall, the whole basket freezes, the subsequent baskets are held up in the queue, etc. That's why OOO processors won, a memory miss can to a degree be worked around by executing instructions ahead in the queue, if they're not dependent on the stalled instruction. [0] Blindingly obvious to me when it was pointed out, but it did need pointing out I have to admit.

ghusbands 6y ago
The folks working on the Mill seem to disagree. They have a VLIW-style architecture (up to 33 sub-instructions in an up-to-512-bit instruction) that they claim does not have the same drawbacks. That at least means that someone is still chasing the VLIW general purpose CPU dream.
- throwaway_pdp09 6y ago
  I'm aware of the mill and that it's VLIW-ish. You're right, I will do some (re)-reading on it to remember how it works around that (although in a video ivan goddard does say if you hit main mem randomly, no architecture can save you, mill included). DSPs, and of course GPUs, do exist and again you're right, for special purpose but important niches of course, but I was referring to general purpose CPUs but didn't make that clear. Thanks.
- drivebycomment 6y ago
  Mill has nothing to show for it. Mill has no answer for the unpredictable cache misses. Their belt idea is cute and interesting but inconsequential and irrelevant. The main challenge of the modern microprocessors is not in how many ALUs you run in parallel nor how you model and implement your register file, but how to keep the ALUs fed with data from the memory hierarchy. VLIW doesn't help you with that at all, which is why it is largely irrelevant and a tiny niche. For applications like GPU, you address the feeding problem with multiple threads through SMT/SIMT and multiple cores, and SIMD. For applications where a single thread performance is important, you use OOO and/or speculative execution to maximize the feeding of the data. If you only care about running a particular loop and you only need to meet a fixed cycle budget like DSP application, VLIW may make sense when you want to save some engineering time and you have relatively low volume. Once volume is high enough, you will likely choose an in-order superscalar instead, for better performance or even simpler implementation, ease of programming, better backward compatibility and higher instruction density. VLIW has become more of a niche even in DSP. E.g. see https://www.eetimes.com/a-case-for-superscalar-dsps/ https://www.eetimes.com/a-case-for-superscalar-dsps/ which is now almost 20 years old article. The basic idea behind VLIW of moving as much work as you can to the compiler is not wrong, but VLIW turned out to be a bad way to divide work between the pipeline and the compiler.
  - zozbot234 6y ago
    > For applications where a single thread performance is important, you use OOO and/or speculative execution to maximize the feeding of the data. OOO and speculative execution are very energy-intensive per unit of compute, and hard to implement securely. Even if a VLIW-ish cpu like the Mill is still ultimately bottlenecked by memory bandwidth, avoiding pretty much every known drawback of OOO is a huge deal.
    - drivebycomment 6y ago
      For the power-efficiency, in-order superscalars proved to be efficient enough - see various in-order superscalar ARMs. Even for OOO, various ARMs and Apple showed that they can get quite power efficient. Also, more importantly, VLIW is not a power efficient approach. It fundamentally wastes instruction density. It wastes the die space - superscalar + SMT are inherently more efficient when you have more than 1 thread, and VLIW is suboptimal for SMT. It also turns out in-order superscalars can match VLIW in ALU power efficiency by powering off unused ALUs just like VLIW can, and for applications where the power efficiency is more important, the control logic can be simplified enough (at the price of performance obviously) to add negligible power overhead. As for the security aspect of the speculative execution, OOO outperforms VLIW dramatically for a single thread performance even without any speculative execution. Moreover, since the discovery of Spectre, there's been plenty of research into how to do speculative execution without leaking: https://ieeexplore.ieee.org/abstract/document/8806916 https://ieeexplore.ieee.org/abstract/document/8806916 https://ieeexplore.ieee.org/abstract/document/8574559 https://ieeexplore.ieee.org/abstract/document/8574559 https://ieeexplore.ieee.org/abstract/document/9138957 https://ieeexplore.ieee.org/abstract/document/9138957 https://dl.acm.org/doi/abs/10.1145/3316781.3317914 https://dl.acm.org/doi/abs/10.1145/3316781.3317914 It's not yet clear what approach will actually win out and will be adopted, but it's clear there are multiple viable solutions such that this is a solvable problem. The whackamole of fixing / addressing individual spectre-like vulnerability likely will continue for a while - after all, processor folks didn't have that much time given the development cycle for new pipeline/architecture is measured in years - but it's unlikely that side-channel issue will make speculative execution not viable. Edit: s/SIMT/SMT/

trm42 6y ago

Please do remember that Torvalds worked for Transmeta which created and designed vliw CPUs from scratch. He's also really practical guy if you happen to remember about Minix micro-kernel VS monolithic Linux-kernel ;)

Quequau 6y ago
I remember all those things, it's why I mentioned Linus as someone worth listening to but who nevertheless might too quick to dismiss VLIW now. Anyway, there are a couple of efforts which include VLIW tech which are in currently in the development pipeline now. So maybe in the next few years we will see some interesting things with it.

fulafel 6y ago

Most applications of vliw are in dsp, gpu, video codec coprocessor type things. It's mature technology and successful outside general purpose CPUs. Transmeta was 25 years ago and trying to apply vliw in a novel way to run x86 code, a daring idea that didn't pay off very well.

CalChris 6y ago
It paid off just fine except in a monetary sense. They got to market with a low power fast x86 clone, arguably creating that low power market. However, for that thankless and difficult task of creating an X86 clone (x87 was a nightmare) they received the great prize of competing against Intel in both manufacturing and marketing. Moreover, they were a 32b machine in 2000 and 64b was right around the corner. Indeed when AMD64 came along, announced in 2000, it was simpler and forced Intel+HP to give up on Itanium. (Yes, Itanium had other problems but engineering problems can, to a degree, be surmounted. Market problems are harder.)
- fulafel 6y ago
  It's always possible that they had just bad luck / timing, but why didn't anyone try/succeed with it since?
  - CalChris 6y ago
    Well a bunch of the Crusoe people did. They eventually got acquired by Nvidia and that’s what Denver is. Not VLIW though.
    - monocasa 6y ago
      I was pretty sure that both the Transmeta cores, as well as the Denver cores were VLIW internally.
    - fulafel 6y ago
      Yep, The OOO nature of the cpu might make the static scheduling less attractive for Denver. But I think the public info doesn't say either way vliw or not. Also it's now mainstream (Intel does it) to have an intermediate code storage format with the introduction of trace caches. I wonder what the usual format of those is.

celrod 6y ago

I think this was a great comment and discussion thread on OOO vs VLIW: https://news.ycombinator.com/item?id=24466593 https://news.ycombinator.com/item?id=24466593 The biggest takeaway for me was the importance of scalability. As OOO CPUs get better, they can become increasingly parallel when running the exact same assembly. While VLIW like Itanium was limited to 3 ops/instruction, which is already behind today's OOO CPUs. (Although Mill's 33 ops/instruction sounds like it'd take much longer to beat.) My takeaway was that NVidea's Volta looked on the right track, where it added 6-bit dependency bitmasks to instructions to make building the dependency graphs (primary cost to OOO) more efficient.

microtherion 6y ago
It seems to me that earlier RISC designs learned the same lesson with branch and load delay slots. They performed fine when introduced, but did not adapt well. All of this was tried to improve the speed of in-order CPUs. Once OOO became widespread, such hardware assists became pointless (And it would appear to me that the dependency assists you describe will similarly not stay relevant all that long).
xscott 6y ago
> While VLIW like Itanium was limited to 3 ops/instruction, which is already behind today's OOO CPUs Itanium could group 3 instructions per 128 bit "bundle", but multiple bundles could be run in parallel, and the compiler had to insert explicit "stops" when that wasn't allowed. The architecture was designed specifically so that future processors could run old code with more parallelism. https://en.wikipedia.org/wiki/Explicitly_parallel_instruction_computing#Moving_beyond_VLIW https://en.wikipedia.org/wiki/Explicitly_parallel_instructio...
Quequau 6y ago
Thanks for the link. I actually saw that when it was posted but there's new comments since then.
marcosdumay 6y ago
> As OOO CPUs get better, they can become increasingly parallel when running the exact same assembly. While VLIW like Itanium was limited to 3 ops/instruction, which is already behind today's OOO CPUs. There's no reason for the VLIW machine not to get wider with time. The fact that Itanium didn't speaks more of its economic unhealthiness than of anything else.

wyldfire 6y ago

> fair number of smart folks who are in the know view VLIW as something that was tried and failed, so they're dismissive of new designs which use VLIW. Snapdragon based phones have multiple Hexagon DSPs, they're VLIW that are also a commercial reality. It's fair to say that there's a lot fewer folks who know how to write hexagon assembly than other architectures but you could play this optimization game w/it too.

hedgehog 6y ago

Summarizing what I learned when I looked into this: VLIW directly exposes hardware parallelism to the programmer, making more throughput available without requiring complex scheduling machinery (save die area + power). This works well for code targeting a single architecture (like embedded) where timing is predictable (like DSP code). Essentially VLIW machine code embeds a bunch of very specific assumptions about processor's internal design. Here VLIW is alive and well for targets like a cell modem baseband or set top box video decoder chip that needs maximum efficiency and only runs the firmware from the manufacturer. General purpose CPUs don't fit this model. They need to support running the same code across chip variants and generations, each with varying amounts of internal parallelism, cache, etc, and instruction timing is not predictable or deterministic (memory and IO latency, cache residency, etc etc). Modern high performance CPUs have extraordinarily complex and clever machinery to process what user space programmers think of as "machine code" to find and use as much parallelism as possible. This includes speculative execution based on behavior observed at run time, something not possible even in a perfect VLIW compiler. Given this you're not likely to see VLIW instruction sets exposed to user space. This is consistent with Linus's view I think. Notably, as 3D accelerators became user-programmable VLIW stopped working well (see ATI/AMD GCN architecture rationale). What Transmeta did is try to combine the advantages of VLIW hardware and stability of existing serial x86 code by filling the gap with a JIT living below the kernel ("Code Morphing Software"). There are examples like HP Dynamo that suggest this is a workable idea and in practice the 1st gen Crusoe product was decent. There are some different ideas why they failed but at least part of the problem is that they got squeezed by Intel's manufacturing advantage: 20 years ago being a year or two ahead in fab process was more effective than having a better design. I haven't seen any strong case that the technical approach wouldn't work at least as well today with TSMC and Samsung now leading in fab. Interestingly the Denver architecture NVIDIA uses in their embedded products follows the Transmeta approach with an ARM front end and embedded JIT. Reportedly they were originally going to do x86 as well. I'm curious what's going to happen there now that they own Arm.

Quequau 6y ago
Hey thanks for this response. It's broadly consistent with my understanding, though obviously better articulated, and you did have some points I hadn't heard before. Last year's HotChips (or maybe even the year before) had a talk from a group that was working on a new design which is arguably VLIW (or has some elements of VLIW) and recently-ish they've been back in the news for having actual silicon in test. It's my hope that they are successful in their efforts, at least in some technical way. Lastly somewhere in the back of my head is the vague idea that Apple doesn't actually use ARM cores either but instead uses a continuation the technology they bought with PA Semi only with an "ARM front end". I've long forgotten where I heard that from though.

guenthert 6y ago

Isn't an interesting aspect of VLIW CPUs, that they don't use branch prediction and hence are not susceptible to SPECTRE class of security vulnerabilities? I'm actually surprised that not more is made out of that. Afaics, only Secure64 is tooting that horn: https://secure64.com/2018/01/08/secure64-dns-family-not-subject-to-spectre-and-meltdown/ https://secure64.com/2018/01/08/secure64-dns-family-not-subj...