4 ms·

> fundamentally flawed assumption (that you don't need dynamic scheduling, AKA OoO processing) I'm still unconvinced this is fundamental. It certainly was fla

> fundamentally flawed assumption (that you don't need dynamic scheduling, AKA OoO processing) I'm still unconvinced this is fundamental. It certainly was flawed back then, but compiler theory has improved a LOT since then, we have polyhedral optimization, e.g. that we didn't have access to... You could probably optimize delay line technology that way.

JonChesterfield 4y ago
If you know how long data will take to go to/from memory then you can schedule pretty well. If you don't know whether some value will hit in the L1 or the L3 cache there's wild variance on how long it'll take so you have to do something else in the meantime. On x64, that's the pipeline and speculation. On a GPU, you swap to another fibre/warp until the memory op finished. Fundamentally the hardware knows how long the memory access took, the software can only guess how long it will take. That kills effective ahead of time scheduling on most architectures.
- throwawaymaths 4y ago
  Iirc caches mostly exist to help keep pipelines full. You could maybe imagine an architecture with deterministic memory access times from specified regions.
  - cwzwarich 4y ago
    Such architectures exist (e.g. for signal processing). They just aren't good for running general-purpose software.
    - Kon-Peki 4y ago
      Itanium sold, barely, and still has people using it. But, as you can imagine, not with general-purpose software. By the 2nd or 3rd generation, Intel had made some changes that really improved performance a lot. Was the final generation of Itanium the 4th generation?
  - nordsieck 4y ago
    > You could maybe imagine an architecture with deterministic memory access times from specified regions. Sure. But it's very easy to level down. And almost impossible to level up.
  - anonymoushn 4y ago
    > Iirc caches mostly exist to help keep pipelines full You can find out by disabling the stride prefetcher and observing the 90+% performance regression in all your software!
- nullc 4y ago
  I don't think building a threaded cpu that switches threads to hide memory latency would be philosophically incompatible with VLIW. I think the later generation itaniums were threaded too. VLIW might make it hard to share functional units, but don't-- the relatively simplicity of the core should let you have more copies of the whole shabang. The SMT would just be for hiding memory accesses not for higher functional unit utilization. On x86 SMT gets a bad rap because its never been tremendously effective. It's much more effective in IBM's implementation on Power. It wouldn't be hard to imagine some Itanium SMT monster.
  - yosefk 4y ago
    How many threads would it take to hide 100ns of DRAM access latency and what serial performance would we get?
    - nullc 4y ago
      It doesn't have to be hidden completely, there are still caches. With 2 unstalled threads you effectively halve it (in terms of throughput), 4 unstalled threads you effectively quarter it, etc.
  - Kon-Peki 4y ago
    > It wouldn't be hard to imagine some Itanium SMT monster. Itanium did have SMT
- deleted 4y ago
  [deleted]
moonchild 4y ago
Applications tend to exhibit a lot of dynamic behaviour. Consider any graph munger. I do expect there is an interesting subset of applications, particularly those that have fairly narrow scope and that have been highly optimised, which could be effectively statically scheduled. But for general-purpose computation, I don't buy it. Perhaps as part of the trend towards increasingly heterogeneous architectures, we'll see big VLIW coprocessors for power efficiency in certain serial workloads (GPUs are massively parallel but little VLIW coprocessors; however, they do have dynamic scheduling).
- killerstorm 4y ago
  Itanium also does dynamic scheduling. It's not really VLIW - it is just optimized to decode and schedule a lot of instructions at once.
  - FullyFunctional 4y ago
    That’s not what dynamic scheduling means. What you need is to dynamically extract ILP so you can get good performance on single thread applications in light of branch mispredictions and cache misses. We are talking SPECint, not SPECfp nor throughput (threading). x86 chips and Apple Silicon show what you can do here and to do that with IA-64 would have been harder, not easier.
throw0101c 4y ago
> It certainly was flawed back then, but compiler theory has improved a LOT since then Which would be fine if there was a time machine available to send back what we know now to the past. But at the time having sucky compilers for what you want to (try to) accomplish was a bad idea. The same decision now may be good, but then it was a mistake. It'd be like trying to have a Moon landing project in the 1920s: we got there eventually, but certain things are only possible when the time 'is right'.
- FullyFunctional 4y ago
  compiler technology doesn’t help when cache misses and branches aren’t predictable. That’s the fatal flaw.
- kjs3 4y ago
  Spot on. I think it's also important to point out that "the compiler will fix it" line wasn't new with Itanic. The Multiflow guys (for one) said the same thing for their Trace line of VLIW machines 15 years or so before HP & Intel, and that didn't work so well. So now, 40 years down the line, can the compiler really fix it, or do we acknowledge that maybe some things are theoretically nifty but don't work out in practice and move on? I don't know the answer to that, but I have my suspicions.
ilyt 4y ago
Because even if you get compilers to optimize for your current VLIW, that makes it harder to make improvements down the line. Current way, while arguably pretty wasteful on all the micro-optimizing CPU does on incoming bytecode, allows designers to nearly freely expand hardware to meet the needs without having compilers to produce different code.
hajile 4y ago
The halting problem means that for typical programs, you can't prove control flow. Proving optimization means not only proving control flow, but then knowing which branches get taken most often so you can optimize. There may be a small subset of programs for which this is true, but the rest leave the compiler completely blinded. OoO execution does an end run around this by examining the program as it runs and adapting to the current reality rather than a simulated reality. This is the same reason a JIT can do optimizations that a compiler cannot do. The ability to look 500-700 instruction into the future to bundle them together into a kind of VLIW dynamically is a very powerful feature. As to compiler theory, it really isn't that advanced. Our "cutting edge" compilers are doing glorified find-and-replace for their final optimizations (peephole optimization). Look at SIMD and auto-vectorization. There are so many potential questions the compiler can't prove the answer to that even trivial vectorization that any programmer would identify can't be used by the compiler to the point where the entire area of research has resulted in basically zero improvements in real-world code.
therealcamino 4y ago
This is the same argument that was being made in the 90s, ironically, that compilers were now good enough that it would work. The thing is, on regular, array-based codes, it's great. And it was then, too -- the polyhedral approach was all being developed at exactly that time, but maybe it's not clear in hindsight because the terminology hadn't settled yet. Ancourt and Irigoin's "Scanning Polyhedra With Do Loops" was published in 1991. Lots of the unification of loop optimization was labeled affine optimization or described as based on Presburger arithmetic. But that is the technology that they were depending on to make it work. But most code we run is not Fortran in another guise. The dependences aren't known at compile time. That issue hasn't changed much. The one change now is that workloads that were called "scientific computing" then are now being run for machine learning. But now it doesn't make sense to run regular, FP-intensive codes on a CPU at all, because of GPUs and ML accelerators. So what's left for CPUs that excel on that workload? I'm not sure there is a niche there.

JonChesterfield 4y ago

If you know how long data will take to go to/from memory then you can schedule pretty well. If you don't know whether some value will hit in the L1 or the L3 cache there's wild variance on how long it'll take so you have to do something else in the meantime. On x64, that's the pipeline and speculation. On a GPU, you swap to another fibre/warp until the memory op finished. Fundamentally the hardware knows how long the memory access took, the software can only guess how long it will take. That kills effective ahead of time scheduling on most architectures.

throwawaymaths 4y ago
Iirc caches mostly exist to help keep pipelines full. You could maybe imagine an architecture with deterministic memory access times from specified regions.
- cwzwarich 4y ago
  Such architectures exist (e.g. for signal processing). They just aren't good for running general-purpose software.
  - Kon-Peki 4y ago
    Itanium sold, barely, and still has people using it. But, as you can imagine, not with general-purpose software. By the 2nd or 3rd generation, Intel had made some changes that really improved performance a lot. Was the final generation of Itanium the 4th generation?
- nordsieck 4y ago
  > You could maybe imagine an architecture with deterministic memory access times from specified regions. Sure. But it's very easy to level down. And almost impossible to level up.
- anonymoushn 4y ago
  > Iirc caches mostly exist to help keep pipelines full You can find out by disabling the stride prefetcher and observing the 90+% performance regression in all your software!
nullc 4y ago
I don't think building a threaded cpu that switches threads to hide memory latency would be philosophically incompatible with VLIW. I think the later generation itaniums were threaded too. VLIW might make it hard to share functional units, but don't-- the relatively simplicity of the core should let you have more copies of the whole shabang. The SMT would just be for hiding memory accesses not for higher functional unit utilization. On x86 SMT gets a bad rap because its never been tremendously effective. It's much more effective in IBM's implementation on Power. It wouldn't be hard to imagine some Itanium SMT monster.
- yosefk 4y ago
  How many threads would it take to hide 100ns of DRAM access latency and what serial performance would we get?
  - nullc 4y ago
    It doesn't have to be hidden completely, there are still caches. With 2 unstalled threads you effectively halve it (in terms of throughput), 4 unstalled threads you effectively quarter it, etc.
- Kon-Peki 4y ago
  > It wouldn't be hard to imagine some Itanium SMT monster. Itanium did have SMT
deleted 4y ago
[deleted]

moonchild 4y ago

Applications tend to exhibit a lot of dynamic behaviour. Consider any graph munger. I do expect there is an interesting subset of applications, particularly those that have fairly narrow scope and that have been highly optimised, which could be effectively statically scheduled. But for general-purpose computation, I don't buy it. Perhaps as part of the trend towards increasingly heterogeneous architectures, we'll see big VLIW coprocessors for power efficiency in certain serial workloads (GPUs are massively parallel but little VLIW coprocessors; however, they do have dynamic scheduling).

killerstorm 4y ago
Itanium also does dynamic scheduling. It's not really VLIW - it is just optimized to decode and schedule a lot of instructions at once.
- FullyFunctional 4y ago
  That’s not what dynamic scheduling means. What you need is to dynamically extract ILP so you can get good performance on single thread applications in light of branch mispredictions and cache misses. We are talking SPECint, not SPECfp nor throughput (threading). x86 chips and Apple Silicon show what you can do here and to do that with IA-64 would have been harder, not easier.

throw0101c 4y ago

> It certainly was flawed back then, but compiler theory has improved a LOT since then Which would be fine if there was a time machine available to send back what we know now to the past. But at the time having sucky compilers for what you want to (try to) accomplish was a bad idea. The same decision now may be good, but then it was a mistake. It'd be like trying to have a Moon landing project in the 1920s: we got there eventually, but certain things are only possible when the time 'is right'.

FullyFunctional 4y ago
compiler technology doesn’t help when cache misses and branches aren’t predictable. That’s the fatal flaw.
kjs3 4y ago
Spot on. I think it's also important to point out that "the compiler will fix it" line wasn't new with Itanic. The Multiflow guys (for one) said the same thing for their Trace line of VLIW machines 15 years or so before HP & Intel, and that didn't work so well. So now, 40 years down the line, can the compiler really fix it, or do we acknowledge that maybe some things are theoretically nifty but don't work out in practice and move on? I don't know the answer to that, but I have my suspicions.

ilyt 4y ago

Because even if you get compilers to optimize for your current VLIW, that makes it harder to make improvements down the line. Current way, while arguably pretty wasteful on all the micro-optimizing CPU does on incoming bytecode, allows designers to nearly freely expand hardware to meet the needs without having compilers to produce different code.

hajile 4y ago

The halting problem means that for typical programs, you can't prove control flow. Proving optimization means not only proving control flow, but then knowing which branches get taken most often so you can optimize. There may be a small subset of programs for which this is true, but the rest leave the compiler completely blinded. OoO execution does an end run around this by examining the program as it runs and adapting to the current reality rather than a simulated reality. This is the same reason a JIT can do optimizations that a compiler cannot do. The ability to look 500-700 instruction into the future to bundle them together into a kind of VLIW dynamically is a very powerful feature. As to compiler theory, it really isn't that advanced. Our "cutting edge" compilers are doing glorified find-and-replace for their final optimizations (peephole optimization). Look at SIMD and auto-vectorization. There are so many potential questions the compiler can't prove the answer to that even trivial vectorization that any programmer would identify can't be used by the compiler to the point where the entire area of research has resulted in basically zero improvements in real-world code.

therealcamino 4y ago

This is the same argument that was being made in the 90s, ironically, that compilers were now good enough that it would work. The thing is, on regular, array-based codes, it's great. And it was then, too -- the polyhedral approach was all being developed at exactly that time, but maybe it's not clear in hindsight because the terminology hadn't settled yet. Ancourt and Irigoin's "Scanning Polyhedra With Do Loops" was published in 1991. Lots of the unification of loop optimization was labeled affine optimization or described as based on Presburger arithmetic. But that is the technology that they were depending on to make it work. But most code we run is not Fortran in another guise. The dependences aren't known at compile time. That issue hasn't changed much. The one change now is that workloads that were called "scientific computing" then are now being run for machine learning. But now it doesn't make sense to run regular, FP-intensive codes on a CPU at all, because of GPUs and ML accelerators. So what's left for CPUs that excel on that workload? I'm not sure there is a niche there.