3 ms·

The ground is littered with the dead bodies of CPU designers who said “we’ll just make a smart compiler”. The CPU knows what the software is actually doing and

The ground is littered with the dead bodies of CPU designers who said “we’ll just make a smart compiler”. The CPU knows what the software is actually doing and has done in the past. The compiler does not. The CPU can change implementations; the compiler must bake any assumptions in at build time. Speculative execution is required so long as memory latencies are so large. That trend isn’t reversing itself anytime soon.

gruez 8y ago
>The CPU knows what the software is actually doing and has done in the past. The compiler does not. profile guided optimizations?
- gmueckl 8y ago
  Profiles capture only program brhabiour under test Data and sssumed that that pattern is prevalent. If the actual runtime behaviour is bimodal or worse, then only one mode will be optimized at most. A CPU can keep statiatics at runtime (the branch predictor does) and thus adapt ever so slightly to the actual current program behaviour.
  - flukus 8y ago
    Runtime optimizations have there own problems. Sometimes the rare case will also be the most sensitive, sometimes the statistics will take too long to turn around, etc. At least profile guided optimization gives you a degree of control.
waterhouse 8y ago
Not that people making the smart-compiler assumption in the days before JITs were justified... but that sounds like an argument for using JITs.
mokus 8y ago
It seems like the main problem is that the compiler doesn’t know the specific target it will run on. We actually do have our smart-enough compilers. They just come in the form of microcode that we can’t audit or even opt out of because the true instruction set of the machine is a trade secret and we’ve been locked out anyway by code signing.
- angry_octet 8y ago
  Well you can target a specific CPU model and its specific cache sizes and functional units, even the microcode revision it uses. Some cryptographic code certainly does that. But it is a major overhead to comprehend all that detail for even a small section of code. There are undocumented micro ops that are present, but they are not really relevant to this, they are not the cause of speculative execution attacks. The compiler is poor at anticipating the many types of dynamic program behaviour. But the CPU is also bad at dealing with it. The very simple statistical model for branch prediction is used because space on the die is precious. Cycle accurate simulation is done routinely, if anyone had come up with a much better (and equally general) predictor that could fit on die it would already be in use. People have tried many variants of dynamically recompiling machine code. For example, early MIPS processors (R3000) recompiled everything to be slightly closer to the specific CPU, and exposed pipeline stages (like delayed branch, delayed load), software page fault handling etc. In some ISAs you can even encode the branch prediction hints the compiler makes (from analysis or profiling) into the opcode. But the gains are limited.

gruez 8y ago

>The CPU knows what the software is actually doing and has done in the past. The compiler does not. profile guided optimizations?

gmueckl 8y ago
Profiles capture only program brhabiour under test Data and sssumed that that pattern is prevalent. If the actual runtime behaviour is bimodal or worse, then only one mode will be optimized at most. A CPU can keep statiatics at runtime (the branch predictor does) and thus adapt ever so slightly to the actual current program behaviour.
- flukus 8y ago
  Runtime optimizations have there own problems. Sometimes the rare case will also be the most sensitive, sometimes the statistics will take too long to turn around, etc. At least profile guided optimization gives you a degree of control.

waterhouse 8y ago

Not that people making the smart-compiler assumption in the days before JITs were justified... but that sounds like an argument for using JITs.

mokus 8y ago

It seems like the main problem is that the compiler doesn’t know the specific target it will run on. We actually do have our smart-enough compilers. They just come in the form of microcode that we can’t audit or even opt out of because the true instruction set of the machine is a trade secret and we’ve been locked out anyway by code signing.

angry_octet 8y ago
Well you can target a specific CPU model and its specific cache sizes and functional units, even the microcode revision it uses. Some cryptographic code certainly does that. But it is a major overhead to comprehend all that detail for even a small section of code. There are undocumented micro ops that are present, but they are not really relevant to this, they are not the cause of speculative execution attacks. The compiler is poor at anticipating the many types of dynamic program behaviour. But the CPU is also bad at dealing with it. The very simple statistical model for branch prediction is used because space on the die is precious. Cycle accurate simulation is done routinely, if anyone had come up with a much better (and equally general) predictor that could fit on die it would already be in use. People have tried many variants of dynamically recompiling machine code. For example, early MIPS processors (R3000) recompiled everything to be slightly closer to the specific CPU, and exposed pipeline stages (like delayed branch, delayed load), software page fault handling etc. In some ISAs you can even encode the branch prediction hints the compiler makes (from analysis or profiling) into the opcode. But the gains are limited.