3 ms·

Matt Godbolt: Five things you didn't realise your CPU did for you

nuc1e0n 4y ago
The question to ask at this point is why compilers still target the emulated x86 opcodes and not the microcode directly? Also, speculative execution has proven to be an unsound approach in light of the spectre vulnerability right?
- Yujf 4y ago
  Even of you can detect effects of speculative execution it does not necessarily mean that it is unsound, because the program still functions as expected. It is however not secure in many cases, but I also think that newer implementations do better in this regard so it is probably not a fundamental issue. And about targeting microcode directly, that would be a new ISA which is not desireable and also decoding instructions is not that big of a deal compared to the rest of the core so it is not worth the hassle of switching ISAs.
  - nuc1e0n 4y ago
    It's not my field of expertise of course, but if the speculative execution microcode knows what has and needs to occur, what's to stop a game of speculative execution vulnerability whack-a-mole occurring? I would think from a security standpoint it would be better to not do perform any operation that isn't necessary. That way there would be no speculative data available that could be snooped upon I would've thought. If compilers had more direct control of the microcode opcodes, speculative execution wouldn't even be so necessary. LLVM is quite good at doing SSA for itself without support for it in the processor as well. We're now emulating a 40 year old architecture even for non legacy software, for what? Heating up silicon? That doesn't seem very energy efficient.
    - tambourine_man 4y ago
      One positive and unplanned side effect is code density. Besides taking less memory, if you can “decompress” the ISA faster than memory bandwidth would otherwise allow, it’s a net win. There are many other downsides to x86, like variable length instructions, of course, but it’s something to consider.
afandian 4y ago
The crazy thing to me is that the overhead of all this bookkeeping and extra hardware complexity adds up to less than the performance improvement.
- tambourine_man 4y ago
  Where did you get that from?
  - afandian 4y ago
    Just implicature. I assume that there was a net benefit to adding all of these features.
    - tambourine_man 4y ago
      I don’t get what your are saying. OoO and speculative execution can increase speed dramatically. It’s a lot more complex and potentially dangerous, but all high performance cores do this so it’s obviously worth it.
      - afandian 4y ago
        What PoignardAzur said. The machinery to instrument and calculate probabilities on branches. Maintaining state enough that speculatively executed branches can be rolled back. That sounds non-trivial and I imagine it would take a lot of real estate, energy, etc.
        tambourine_man 4y ago
        Yeah, I got it on a third read :) Agreed.
  - PoignardAzur 4y ago
    To be clear, I think GP is saying "that stuff manages to be cheap enough that the performance improvements more than make up for the overhead".
PoignardAzur 4y ago
Summary: - Prefetches predictable load patterns. - Converts to micro-code. - Does OoO execution. - Renames registers to SSA-like form. - Predicts branches. (Man, seeing how much work the processor goes through to hammer x86 into something vaguely optimized, I really wish something like the Mill architecture had taken over so we'd get an instructions set made for compilers, not one that requires its own invisible compiler in the chip.)
- IshKebab 4y ago
  The downside of that approach is that the compiler doesn't have access to runtime information, or implementation details of the CPU (e.g. how many internal registers it has).
  - PoignardAzur 4y ago
    The solution to that is to have two stages of compilation: the first produces a binary with a general instruction set, and the second produces a binary with an instruction set specific to the machine it's installed on. (Though even with that approach you probably still need microcode) The Mill architecture supposedly does this; the ISA has three variants for different product lines.
    - mattgodbolt 4y ago
      I think the point that was being made is that the _dynamic_ performance of the system is important too: branch prediction, cache access patterns etc, can't always be determined ahead of time, or even at the point the program is being installed or first run. This is why JIT compilers exist that tune the program as it runs, and this is what effectively is also happening at the micro-op level in your CPU as it runs.

The question to ask at this point is why compilers still target the emulated x86 opcodes and not the microcode directly? Also, speculative execution has proven to be an unsound approach in light of the spectre vulnerability right?

Yujf 4y ago
Even of you can detect effects of speculative execution it does not necessarily mean that it is unsound, because the program still functions as expected. It is however not secure in many cases, but I also think that newer implementations do better in this regard so it is probably not a fundamental issue. And about targeting microcode directly, that would be a new ISA which is not desireable and also decoding instructions is not that big of a deal compared to the rest of the core so it is not worth the hassle of switching ISAs.
- nuc1e0n 4y ago
  It's not my field of expertise of course, but if the speculative execution microcode knows what has and needs to occur, what's to stop a game of speculative execution vulnerability whack-a-mole occurring? I would think from a security standpoint it would be better to not do perform any operation that isn't necessary. That way there would be no speculative data available that could be snooped upon I would've thought. If compilers had more direct control of the microcode opcodes, speculative execution wouldn't even be so necessary. LLVM is quite good at doing SSA for itself without support for it in the processor as well. We're now emulating a 40 year old architecture even for non legacy software, for what? Heating up silicon? That doesn't seem very energy efficient.
  - tambourine_man 4y ago
    One positive and unplanned side effect is code density. Besides taking less memory, if you can “decompress” the ISA faster than memory bandwidth would otherwise allow, it’s a net win. There are many other downsides to x86, like variable length instructions, of course, but it’s something to consider.

afandian 4y ago

The crazy thing to me is that the overhead of all this bookkeeping and extra hardware complexity adds up to less than the performance improvement.

tambourine_man 4y ago
Where did you get that from?
- afandian 4y ago
  Just implicature. I assume that there was a net benefit to adding all of these features.
  - tambourine_man 4y ago
    I don’t get what your are saying. OoO and speculative execution can increase speed dramatically. It’s a lot more complex and potentially dangerous, but all high performance cores do this so it’s obviously worth it.
    - afandian 4y ago
      What PoignardAzur said. The machinery to instrument and calculate probabilities on branches. Maintaining state enough that speculatively executed branches can be rolled back. That sounds non-trivial and I imagine it would take a lot of real estate, energy, etc.
      - tambourine_man 4y ago
        Yeah, I got it on a third read :) Agreed.
- PoignardAzur 4y ago
  To be clear, I think GP is saying "that stuff manages to be cheap enough that the performance improvements more than make up for the overhead".

PoignardAzur 4y ago

Summary: - Prefetches predictable load patterns. - Converts to micro-code. - Does OoO execution. - Renames registers to SSA-like form. - Predicts branches. (Man, seeing how much work the processor goes through to hammer x86 into something vaguely optimized, I really wish something like the Mill architecture had taken over so we'd get an instructions set made for compilers, not one that requires its own invisible compiler in the chip.)

IshKebab 4y ago
The downside of that approach is that the compiler doesn't have access to runtime information, or implementation details of the CPU (e.g. how many internal registers it has).
- PoignardAzur 4y ago
  The solution to that is to have two stages of compilation: the first produces a binary with a general instruction set, and the second produces a binary with an instruction set specific to the machine it's installed on. (Though even with that approach you probably still need microcode) The Mill architecture supposedly does this; the ISA has three variants for different product lines.
  - mattgodbolt 4y ago
    I think the point that was being made is that the _dynamic_ performance of the system is important too: branch prediction, cache access patterns etc, can't always be determined ahead of time, or even at the point the program is being installed or first run. This is why JIT compilers exist that tune the program as it runs, and this is what effectively is also happening at the micro-op level in your CPU as it runs.