7 ms·

Why do and then un-do? The reason is that we've not yet found a better way of making fast general purpose processors. The OOO (= out-of-order) approach with h

Why do and then un-do? The reason is that we've not yet found a better way of making fast general purpose processors. The OOO (= out-of-order) approach with high-quality prediction (e.g. branch, value) to processor micro-architecture makes sense if you have to mask a lot of memory-access latency, which comes from data-dependent (= unpredictable) memory access. General purpose workloads have a lot of that. (If memory access patterns are more predictable, you'd probably run your workload on a GPU or TPU or DSP, or some other accelerator.) Compilers, whether ahead-of-time, or JIT, have not got enough information to schedule commands in a way that can mask memory latency the way an OOO scheduler inside a processor can. Stalling the pipeline because you are waiting for data to arrive from memory is disastrous for performance and to be avoided at all cost. Intel's Itanium was based on the premise that it is possible statically to schedule well enough, but that has been considered a failure for general workloads. Moreover, if you let a compiler schedule you need to have ISA extensions that allow you to communicate scheduling information to the processor which is not cost free (for example you may spend precious bits to encode scheduling order that you then cannot use for other things). I suspect that the very last word has not been spoken in this discussion, but everything obvious to make dataflow competitive (and quite a lot more) has been tried, and failed.

Veedrac 6y ago
IMO a lot of the problem is simply that powerful players (eg. Intel) had a vested interest in tying things to the platform, and the megaliths we've built on top all assume this construction, so even if there are clear ways in which things can be done better (there are), you'd have to reinvent the world to do them. Fundamental innovation is really difficult, and OoO CPUs avoid the need for it.
- gpderetta 6y ago
  GPUs have been extremely successful, so have DSPs, each on their niche. They just don't work well on branchy pointer chasey code. Intel itself tried to move beyond OoO and utterly failed. There is no conspiracy here.
  - Veedrac 6y ago
    GPUs were only able to come about into the mainstream because they were brought about and developed alongside their use-case. Much the same can be said for camera ISPs or neural accelerators. GPUs only started tackling more general workloads long after they were a standard, and even today GPGPU is a bit of a mess. If GPUs didn't come around when they did, and games were all CPU-based today, you wouldn't have an easy time inventing and selling a GPU. > Intel itself tried to move beyond OoO and utterly failed. One attempt in 2001 (and a few before then when the market was more competitive but before people had the context we have today) by the company with market power doesn't invalidate my point as much as reinforce it. People have tried approximately nothing in the last 20 years, all pointing back to this one bad design Intel did 20 years ago as if that proves the taken path is the only valid path.
    - MaxBarraclough 6y ago
      > If GPUs didn't come around when they did, and games were all CPU-based today, you wouldn't have an easy time inventing and selling a GPU. I'm not convinced. You'd just need to show that hardware acceleration can produce worthwhile gains for computer graphics, which doesn't strike me as insurmountable. Perhaps start with an FPGA. > People have tried approximately nothing in the last 20 years Do you follow the 'Mill' architecture? https://en.wikipedia.org/wiki/Mill_architecture https://en.wikipedia.org/wiki/Mill_architecture
      - Veedrac 6y ago
        Good luck showing that when all graphics code is designed to be run on CPUs, nobody has spent more than trivial effort making GPU-compatible graphics algorithms, you can't afford new nodes, your GPU is first-generation (and so far from optimal), you can only use CPU memories (who would invent HBM if GPUs didn't exist?), you can only do fixed-function logic efficiently, you have no demos that don't look like programmer art, and you have no way of getting past the adoption hurdle. Even realtime RT, which is mostly just a compatible add-on to raster, only managed to get off the ground when NVIDIA decided to go all-in and tank the criticism for a generation by effectively forcing it on the customer. No other company could have done that, even though Imagination was pushing for RT hardware for ages before NVIDIA. > Do you follow the 'Mill' architecture? Yes. https://medium.com/@veedrac/to-reinvent-the-processor-671139a4a034 https://medium.com/@veedrac/to-reinvent-the-processor-671139...
- 1996 6y ago
  > you'd have to reinvent the world to do them It reminds me of how MIPS branching is wasted on nops: https://electronics.stackexchange.com/questions/28444/mips-pic32-branch-vs-branch-likely https://electronics.stackexchange.com/questions/28444/mips-p... "The trick in writing efficient code is to put in an instruction that will be useful as part of the loop that is being taken executed, but do no harm if the branch is not taken. The MIPS designers were counting on compiler writers to write clever enough code generators to handle this efficiently. However many do not (including Microchips C32 compiler, based on GCC), and just put NOP's after every branch, wasting both code space and cycles"
  - ekiwi 6y ago
    The problem with delay slots is that their optimal size is somewhat tied to the particular micro-architecture. I.e., different MIPS CPU implementations would have different optimal branch delay slot sizes. This doesn't matter too much if your compiler is allowed to optimize for your particular implementation, but we are still mostly shipping binaries and thus you would have to make trade-offs for portability reasons.
    - Veedrac 6y ago
      They're also pointless once you do even a small amount of dynamic local reordering, which every fast CPU does.
- rsynnott 6y ago
  Intel _tried_ a major departure: https://en.wikipedia.org/wiki/Itanium https://en.wikipedia.org/wiki/Itanium Spoiler: It didn't end well.
  - pjmlp 6y ago
    Only because AMD exists and had access to the patents that allowed them to come up with an alternative. Had it not been the case and Itanium would have been pushed until it eventually replaced x86.
    - rsynnott 6y ago
      I think that's very unlikely, really; Itanium just really wasn't great. To a large extent, if AMD64 hadn't existed it would have been necessary to create it (either that or Intel could've gone off in a RISC-y direction and made something along the lines of a DEC Alpha, I suppose). But in any case, it's hard to see an alternate universe where Itanium was actually successful.
      - pjmlp 6y ago
        Windows and HP-UX were already available on Itanium before amd64 came into scene. Without access to Intel patents who would come with enough money and ecosystem support to actually be a challenge to Itanium?
- cpgxiii 6y ago
  It's not at all clear that better ways exist for general purpose code. A lot of time and effort has been spent by major players, with very little to success to show for it. Intel, quite infamously, tried to make "something better" work with several generations of Itanium and failed impressively against the reality that a sufficiently smart compiler cannot be made to work. You can't blame any of the big players for holding back - Intel, MS, and HP were "all in" on Itanium as the 64-bit future - if it could have been made to work, it would have been widely adopted. At the same time, Transmeta was building a X86-on-VLIW, which, guess what, also failed to be a competitive product. Nvidia, of course, decided to pick up from where Transmeta left off to build what ended up being an aarch64-on-VLIW design. In both cases, the hope was that the low-level translation would allow the benefits of a VLIW architecture while avoiding the "sufficiently smart compiler is impossible" problem, and in both cases performance just wasn't good enough to be competitive.
  - Veedrac 6y ago
    > A lot of time and effort has been spent by major players This really isn't the case IMO. Itanium is one attempt 20 years ago, it didn't try to rewrite the world (compatibility with standard C code is a huge crutch; GPUs couldn't exist in that environment for example), and the design sucked. And for 20 years since it has been sufficient to invoke the name to scare away any attempt of actually trying something new, even when they don't make the same mistakes, or even work anything like Itanium. > Transmeta It's in the name: the most significant part of the design was all the effort spent on compatibility with the dominant paradigm, running stock x86 code! That hugely limits what you can do.
    - cpgxiii 6y ago
      Itanium was not one attempt, it was several generations of trying to make the concept work. And it didn't. > And for 20 years since it has been sufficient to invoke the name to scare away any attempt of actually trying something new, even when they don't make the same mistakes, or even work anything like Itanium. People have been invoking Itanium for the last 20 years because that's what you do with expensive, painful lessons. Just like "branch delay slots are bad" (MIPS et al) or "no sub-word-size-access is bad" (Alpha) or "there's such a thing as too-relaxed memory ordering" (Alpha) or "putting too much magic in the ISA is bad" (iAPX 432), some lessons don't need to be repeatedly revisited and repeated. I get that the Mill team wants to prove everyone wrong, but the only way to show that is working hardware and that doesn't exist (and given the time they've been working, probably won't ever). > compatibility with standard C code is a huge crutch; GPUs couldn't exist in that environment for example Except, guess what, Cuda and OpenCL look exactly like C in many ways. The major features they don't support (well) are dynamic memory allocations and branchy code - good luck trying to make an architecture work that can't efficiently handle malloc. People have tried a lot of different language+hardware architectures over the years, and all of them have been outperformed by OoO machines. > compatibility with the dominant paradigm, running stock x86 code If either Transmeta or Nvidia could have made a standout VLIW architecture, they would have had plenty of business. Defense and HPC customers have absolutely no problem with esoteric architectures so long as the performance is good enough. You could build the craziest architecture, and if you can show it'll be a winner, the DoE will buy it.
fulafel 6y ago
What we call general purpouse processors are just a model that has coevolved with software. If we got better at making software for parallel processors that are different from current GP CPUs, we could call them those processors purpouse too.

Veedrac 6y ago

IMO a lot of the problem is simply that powerful players (eg. Intel) had a vested interest in tying things to the platform, and the megaliths we've built on top all assume this construction, so even if there are clear ways in which things can be done better (there are), you'd have to reinvent the world to do them. Fundamental innovation is really difficult, and OoO CPUs avoid the need for it.

gpderetta 6y ago
GPUs have been extremely successful, so have DSPs, each on their niche. They just don't work well on branchy pointer chasey code. Intel itself tried to move beyond OoO and utterly failed. There is no conspiracy here.
- Veedrac 6y ago
  GPUs were only able to come about into the mainstream because they were brought about and developed alongside their use-case. Much the same can be said for camera ISPs or neural accelerators. GPUs only started tackling more general workloads long after they were a standard, and even today GPGPU is a bit of a mess. If GPUs didn't come around when they did, and games were all CPU-based today, you wouldn't have an easy time inventing and selling a GPU. > Intel itself tried to move beyond OoO and utterly failed. One attempt in 2001 (and a few before then when the market was more competitive but before people had the context we have today) by the company with market power doesn't invalidate my point as much as reinforce it. People have tried approximately nothing in the last 20 years, all pointing back to this one bad design Intel did 20 years ago as if that proves the taken path is the only valid path.
  - MaxBarraclough 6y ago
    > If GPUs didn't come around when they did, and games were all CPU-based today, you wouldn't have an easy time inventing and selling a GPU. I'm not convinced. You'd just need to show that hardware acceleration can produce worthwhile gains for computer graphics, which doesn't strike me as insurmountable. Perhaps start with an FPGA. > People have tried approximately nothing in the last 20 years Do you follow the 'Mill' architecture? https://en.wikipedia.org/wiki/Mill_architecture https://en.wikipedia.org/wiki/Mill_architecture
    - Veedrac 6y ago
      Good luck showing that when all graphics code is designed to be run on CPUs, nobody has spent more than trivial effort making GPU-compatible graphics algorithms, you can't afford new nodes, your GPU is first-generation (and so far from optimal), you can only use CPU memories (who would invent HBM if GPUs didn't exist?), you can only do fixed-function logic efficiently, you have no demos that don't look like programmer art, and you have no way of getting past the adoption hurdle. Even realtime RT, which is mostly just a compatible add-on to raster, only managed to get off the ground when NVIDIA decided to go all-in and tank the criticism for a generation by effectively forcing it on the customer. No other company could have done that, even though Imagination was pushing for RT hardware for ages before NVIDIA. > Do you follow the 'Mill' architecture? Yes. https://medium.com/@veedrac/to-reinvent-the-processor-671139a4a034 https://medium.com/@veedrac/to-reinvent-the-processor-671139...
1996 6y ago
> you'd have to reinvent the world to do them It reminds me of how MIPS branching is wasted on nops: https://electronics.stackexchange.com/questions/28444/mips-pic32-branch-vs-branch-likely https://electronics.stackexchange.com/questions/28444/mips-p... "The trick in writing efficient code is to put in an instruction that will be useful as part of the loop that is being taken executed, but do no harm if the branch is not taken. The MIPS designers were counting on compiler writers to write clever enough code generators to handle this efficiently. However many do not (including Microchips C32 compiler, based on GCC), and just put NOP's after every branch, wasting both code space and cycles"
- ekiwi 6y ago
  The problem with delay slots is that their optimal size is somewhat tied to the particular micro-architecture. I.e., different MIPS CPU implementations would have different optimal branch delay slot sizes. This doesn't matter too much if your compiler is allowed to optimize for your particular implementation, but we are still mostly shipping binaries and thus you would have to make trade-offs for portability reasons.
  - Veedrac 6y ago
    They're also pointless once you do even a small amount of dynamic local reordering, which every fast CPU does.
rsynnott 6y ago
Intel _tried_ a major departure: https://en.wikipedia.org/wiki/Itanium https://en.wikipedia.org/wiki/Itanium Spoiler: It didn't end well.
- pjmlp 6y ago
  Only because AMD exists and had access to the patents that allowed them to come up with an alternative. Had it not been the case and Itanium would have been pushed until it eventually replaced x86.
  - rsynnott 6y ago
    I think that's very unlikely, really; Itanium just really wasn't great. To a large extent, if AMD64 hadn't existed it would have been necessary to create it (either that or Intel could've gone off in a RISC-y direction and made something along the lines of a DEC Alpha, I suppose). But in any case, it's hard to see an alternate universe where Itanium was actually successful.
    - pjmlp 6y ago
      Windows and HP-UX were already available on Itanium before amd64 came into scene. Without access to Intel patents who would come with enough money and ecosystem support to actually be a challenge to Itanium?
cpgxiii 6y ago
It's not at all clear that better ways exist for general purpose code. A lot of time and effort has been spent by major players, with very little to success to show for it. Intel, quite infamously, tried to make "something better" work with several generations of Itanium and failed impressively against the reality that a sufficiently smart compiler cannot be made to work. You can't blame any of the big players for holding back - Intel, MS, and HP were "all in" on Itanium as the 64-bit future - if it could have been made to work, it would have been widely adopted. At the same time, Transmeta was building a X86-on-VLIW, which, guess what, also failed to be a competitive product. Nvidia, of course, decided to pick up from where Transmeta left off to build what ended up being an aarch64-on-VLIW design. In both cases, the hope was that the low-level translation would allow the benefits of a VLIW architecture while avoiding the "sufficiently smart compiler is impossible" problem, and in both cases performance just wasn't good enough to be competitive.
- Veedrac 6y ago
  > A lot of time and effort has been spent by major players This really isn't the case IMO. Itanium is one attempt 20 years ago, it didn't try to rewrite the world (compatibility with standard C code is a huge crutch; GPUs couldn't exist in that environment for example), and the design sucked. And for 20 years since it has been sufficient to invoke the name to scare away any attempt of actually trying something new, even when they don't make the same mistakes, or even work anything like Itanium. > Transmeta It's in the name: the most significant part of the design was all the effort spent on compatibility with the dominant paradigm, running stock x86 code! That hugely limits what you can do.
  - cpgxiii 6y ago
    Itanium was not one attempt, it was several generations of trying to make the concept work. And it didn't. > And for 20 years since it has been sufficient to invoke the name to scare away any attempt of actually trying something new, even when they don't make the same mistakes, or even work anything like Itanium. People have been invoking Itanium for the last 20 years because that's what you do with expensive, painful lessons. Just like "branch delay slots are bad" (MIPS et al) or "no sub-word-size-access is bad" (Alpha) or "there's such a thing as too-relaxed memory ordering" (Alpha) or "putting too much magic in the ISA is bad" (iAPX 432), some lessons don't need to be repeatedly revisited and repeated. I get that the Mill team wants to prove everyone wrong, but the only way to show that is working hardware and that doesn't exist (and given the time they've been working, probably won't ever). > compatibility with standard C code is a huge crutch; GPUs couldn't exist in that environment for example Except, guess what, Cuda and OpenCL look exactly like C in many ways. The major features they don't support (well) are dynamic memory allocations and branchy code - good luck trying to make an architecture work that can't efficiently handle malloc. People have tried a lot of different language+hardware architectures over the years, and all of them have been outperformed by OoO machines. > compatibility with the dominant paradigm, running stock x86 code If either Transmeta or Nvidia could have made a standout VLIW architecture, they would have had plenty of business. Defense and HPC customers have absolutely no problem with esoteric architectures so long as the performance is good enough. You could build the craziest architecture, and if you can show it'll be a winner, the DoE will buy it.

fulafel 6y ago

What we call general purpouse processors are just a model that has coevolved with software. If we got better at making software for parallel processors that are different from current GP CPUs, we could call them those processors purpouse too.