5 ms·

How NOP Nearly Became a Non-NOP on AMD64 (2007)

72 points by mtviewdave 11y ago

userbinator 11y ago
I find myself nodding in assent at the last comment... I also believe that the amount of code out there which would benefit from having an extra 24 32-bit registers is far more than that which would benefit from having 16 64/32-bit ones instead.
- mgraczyk 11y ago
  Not likely. Modern out-of-order microarchitectures like Bulldozer and Broadwell have far more physical registers than the ISA specifies. Haswell has 168 physical integer register, for example. http://www.realworldtech.com/haswell-cpu/3/ http://www.realworldtech.com/haswell-cpu/3/ The lack of registers in the ISA moves the burden of data dependency checking from compiler to core, but it doesn't increase the number of stalls.
  - devit 11y ago
    Well, it means that if code cannot fit everything into the ISA registers, it has to spill them to the stack, which AFAIK is not renamed to the physical registers on current x86, so you still need to pay an extra penalty to access the stack data in the L1 cache.
    - Tuna-Fish 11y ago
      But stack accesses are generally perfectly predicted, meaning that your loads from stack get executed and the data loaded into those extra registers long before your code needs to use those values.
      - tux3 11y ago
        The prefetcher does a great job, but it's not remotely enough to make stack access penalties disappear. It's trivial to write microbenchmarks where spilling hot read/write variables to the stack destroys performance, much harder to find special cases where the difference isn't noticeable.
        stephencanon 11y ago
        The hot region of the stack is, for practical purposes, always resident in D$, so there's no prefetching to be done. I think that you're really talking about out-of-order and speculative loads. You're absolutely correct that spill/fill can be catastrophic for performance, however.
      - stephencanon 11y ago
        There are two common scenarios where spill/fill has catastrophic performance implications: - Workloads that are LSU-bound, like simple per-pixel image operations and level 1 and level 2 BLAS[1]. Here every spill is taking up two LSU dispatch slots that would otherwise be used for "real work". - Spilled loop-carried dependencies in tight loops; here you're simply adding latency to the critical path. Both are fairly rare in "generic" code, but very real concerns for compute kernels. [1] The BLAS operations have few enough buffers that spill/fill never actually happens in practice, but more complex operations on multiple buffers do run into this.
- Tuna-Fish 11y ago
  The last comment is very ignorant of CPU design. Clearing the highest bits of a register is not a stupid thing to do, it's what you have to do if you don't want to have false dependencies on the previous contents of the register. Partially updating registers in an OoO system is much more expensive operation than what it seems to be for a software guy, and the naive way to use them that the comment suggested would have killed any attempts to extract workable ILP from the system. At the time, they evaluated how many registers they needed and ran simulations for 16, 32 and 64. Assuming a modern implementation with register renaming and good OoO, they found only a few % gain from 32 and practically none with 64. They decided to go with 16 to save encoding bits. If they found that x86 needed more registers, they would have added more registers, not made partial accesses more common. Remember that spilling to stack in x86 does not have a latency hit in normal conditions because all modern x86 cpus have a separate stack engine that allows the loads to retire immediately after they enter the instruction window, typically tens of cycles before any instructions immediately following them.
- rayiner 11y ago
  Rare to come across a pro-segmentation position.
Asbostos 11y ago
It seems more a matter of wording than anything in the CPU " An assembler would translate the mnemonic “nop” into “xchg ax, ax” (opcode 0×90)" But 0x90 also means nop, so it's not really translating anything. And now it still means nop in AMD64.
- 0x0 11y ago
  It makes more sense if you look at the neighbors on the opcode map. It seems like the range 0x90-0x97 are all variants of xchg, just that the first happens to have a destination same as source: http://sparksandflames.com/files/x86InstructionChart.html http://sparksandflames.com/files/x86InstructionChart.html
- creshal 11y ago
  > But 0x90 also means nop As… the article even explained further down, sometimes. That's where the whole discussion comes from. • In 32 bit mode, it doesn't matter. Some (dumb) CPUs treated it as xchg ax,ax; pipelined CPUs optimized it away. • In 64 bit mode, it matters: Is "xchg eax, eax" a valid way to clear the upper 32 bits of eax? Or will it always be optimized away as legacy NOP? AMD decided for the latter. They could also have introduced a new opcode for it instead (there are already multiple nop instructions, like nopl/nopw, so it wouldn't have been too far off) – as this only affects 64 bit mode, backwards compatibility didn't really matter, both would have been possible.
  - Asbostos 11y ago
    Opcode 0x90 only means xchg eax,eax on paper. If no documentation ever called it that then it would be a non-issue. It would always have been nop and still be nop. Somebody could also have called it xchg ebx,ebx as well and it would have been just the same in 32-bit mode.
    - creshal 11y ago
      > If no documentation ever called it that then it would be a non-issue. XCHG EAX,target is defined as opcode (0x90 + offset of target register), with EAX having offset 0. So, it was xchg eax,eax originally, and documented as such, before it was turned into NOP because it happened to be safe for it. It's still documented as "alias for the XCHG (E)AX, (E)AX instruction" in Intel's instruction set reference, and pre-486 embedded x86s still treat it as xchg.

72 points by mtviewdave 11y ago

I find myself nodding in assent at the last comment... I also believe that the amount of code out there which would benefit from having an extra 24 32-bit registers is far more than that which would benefit from having 16 64/32-bit ones instead.

mgraczyk 11y ago
Not likely. Modern out-of-order microarchitectures like Bulldozer and Broadwell have far more physical registers than the ISA specifies. Haswell has 168 physical integer register, for example. http://www.realworldtech.com/haswell-cpu/3/ http://www.realworldtech.com/haswell-cpu/3/ The lack of registers in the ISA moves the burden of data dependency checking from compiler to core, but it doesn't increase the number of stalls.
- devit 11y ago
  Well, it means that if code cannot fit everything into the ISA registers, it has to spill them to the stack, which AFAIK is not renamed to the physical registers on current x86, so you still need to pay an extra penalty to access the stack data in the L1 cache.
  - Tuna-Fish 11y ago
    But stack accesses are generally perfectly predicted, meaning that your loads from stack get executed and the data loaded into those extra registers long before your code needs to use those values.
    - tux3 11y ago
      The prefetcher does a great job, but it's not remotely enough to make stack access penalties disappear. It's trivial to write microbenchmarks where spilling hot read/write variables to the stack destroys performance, much harder to find special cases where the difference isn't noticeable.
      - stephencanon 11y ago
        The hot region of the stack is, for practical purposes, always resident in D$, so there's no prefetching to be done. I think that you're really talking about out-of-order and speculative loads. You're absolutely correct that spill/fill can be catastrophic for performance, however.
    - stephencanon 11y ago
      There are two common scenarios where spill/fill has catastrophic performance implications: - Workloads that are LSU-bound, like simple per-pixel image operations and level 1 and level 2 BLAS[1]. Here every spill is taking up two LSU dispatch slots that would otherwise be used for "real work". - Spilled loop-carried dependencies in tight loops; here you're simply adding latency to the critical path. Both are fairly rare in "generic" code, but very real concerns for compute kernels. [1] The BLAS operations have few enough buffers that spill/fill never actually happens in practice, but more complex operations on multiple buffers do run into this.
Tuna-Fish 11y ago
The last comment is very ignorant of CPU design. Clearing the highest bits of a register is not a stupid thing to do, it's what you have to do if you don't want to have false dependencies on the previous contents of the register. Partially updating registers in an OoO system is much more expensive operation than what it seems to be for a software guy, and the naive way to use them that the comment suggested would have killed any attempts to extract workable ILP from the system. At the time, they evaluated how many registers they needed and ran simulations for 16, 32 and 64. Assuming a modern implementation with register renaming and good OoO, they found only a few % gain from 32 and practically none with 64. They decided to go with 16 to save encoding bits. If they found that x86 needed more registers, they would have added more registers, not made partial accesses more common. Remember that spilling to stack in x86 does not have a latency hit in normal conditions because all modern x86 cpus have a separate stack engine that allows the loads to retire immediately after they enter the instruction window, typically tens of cycles before any instructions immediately following them.
rayiner 11y ago
Rare to come across a pro-segmentation position.

Asbostos 11y ago

It seems more a matter of wording than anything in the CPU " An assembler would translate the mnemonic “nop” into “xchg ax, ax” (opcode 0×90)" But 0x90 also means nop, so it's not really translating anything. And now it still means nop in AMD64.

0x0 11y ago
It makes more sense if you look at the neighbors on the opcode map. It seems like the range 0x90-0x97 are all variants of xchg, just that the first happens to have a destination same as source: http://sparksandflames.com/files/x86InstructionChart.html http://sparksandflames.com/files/x86InstructionChart.html
creshal 11y ago
> But 0x90 also means nop As… the article even explained further down, sometimes. That's where the whole discussion comes from. • In 32 bit mode, it doesn't matter. Some (dumb) CPUs treated it as xchg ax,ax; pipelined CPUs optimized it away. • In 64 bit mode, it matters: Is "xchg eax, eax" a valid way to clear the upper 32 bits of eax? Or will it always be optimized away as legacy NOP? AMD decided for the latter. They could also have introduced a new opcode for it instead (there are already multiple nop instructions, like nopl/nopw, so it wouldn't have been too far off) – as this only affects 64 bit mode, backwards compatibility didn't really matter, both would have been possible.
- Asbostos 11y ago
  Opcode 0x90 only means xchg eax,eax on paper. If no documentation ever called it that then it would be a non-issue. It would always have been nop and still be nop. Somebody could also have called it xchg ebx,ebx as well and it would have been just the same in 32-bit mode.
  - creshal 11y ago
    > If no documentation ever called it that then it would be a non-issue. XCHG EAX,target is defined as opcode (0x90 + offset of target register), with EAX having offset 0. So, it was xchg eax,eax originally, and documented as such, before it was turned into NOP because it happened to be safe for it. It's still documented as "alias for the XCHG (E)AX, (E)AX instruction" in Intel's instruction set reference, and pre-486 embedded x86s still treat it as xchg.