5 ms·

Instruction decoding is a bottleneck for x86 these days: Apple M1 can do 8-wide decode, Intel just managed to reach 6-wide in Alder Lake, and AMD Zen 3 only has

by unnah 4y ago

Instruction decoding is a bottleneck for x86 these days: Apple M1 can do 8-wide decode, Intel just managed to reach 6-wide in Alder Lake, and AMD Zen 3 only has a 4-wide decoder. One would think that dropping legacy 16-bit and 32-bit instructions would enable simpler and more efficient instruction decoders in future x86 versions.

amluto 4y ago
Sadly not. x86_64’s encoding is extremely similar to the legacy encodings. AIUI the fundamental problem is that x86 is a variable-length encoding, so a fully parallel decoder needs to decode at guessed offsets, many of which will be wrong. ARM64 instructions are aligned. Dumping legacy features would be great for all kinds of reasons, but not this particular reason.
- hyperman1 4y ago
  This suggests another way forward: Re-encode the existing opcodes with new, more regular byte sequences. E.g. 32 bits / instruction, with some escape for e.g. 64bit constants. You'll have to redo the backend of the assembler, but most of the compiler and optimization wisdom can be reused as-is. Of course, this breaks backward compatibility completely so the high performance mode can only be unlocked for recompiles.
  - colejohnson66 4y ago
    That was Itanium, and it failed for a variety of reasons; one of which was a compatibility layer that sucked. You can't get rid of x86's backwards compatibility. Intel and AMD have done their best by using vector prefixes (like VEX and EVEX)[a] that massively simplify decoding, but there's only so much that can be done. People get caught up in the variable length issue that x86 has, and then claim that M1 beats x86 because of that. Sure, decoding ARM instructions is easier than x86, but the variable length aspect is handled in the predecode/cache stage, not the actual decoder. The decoder, when it reaches an instruction, already knows various bits of info are. The RISC vs CISC debate is useless today. M1's big advantage comes from the memory ordering model (and other things)[0], not the instruction format. Apple actually had to create a special mode for the M1 (for Rosetta 2) that enforces the x86 ordering model (TSO with load forwarding), and native performance is slightly worse when doing so. [0]: https://twitter.com/ErrataRob/status/1331735383193903104 https://twitter.com/ErrataRob/status/1331735383193903104 [a]: There's also others that predate AVX (VEX) such as the 0F38 prefix group consisting only of opcodes that have a ModR/M byte and no immediate, and the 0F3A prefix being the same, but with an 8 bit immediate.
    - kmeisthax 4y ago
      M1 doesn't have a special mode for Rosetta. All code is executed with x86 TSO on M1's application processors. How do I know this? Well, did you know Apple ported Rosetta 2 to Linux? You can get it by running a Linux VM on macOS. It does not require any kernel changes to support in VMs, and if you extract the binary to run it on Asahi Linux, it works just fine too. None of the Asahi team did anything to support x86 TSO. Rosetta also works just fine in m1n1's hypervisor mode, which exists specifically to log all hardware access to detect these sorts of things. If there is a hardware toggle for TSO, it's either part of the chicken bits (and thus enabled all the time anyway) or turned on by iBoot (and thus enabled before any user code runs). Related point: Hector Martin just upstreamed a patch to Linux that fixes a memory ordering bug in workqueues that's been around since before Linux had Git history. He also found a bug in some ARM litmus tests that he was using to validate whether or not they were implemented correctly. Both of those happened purely because M1 and M2 are so hilariously wide and speculative that they trigger memory reorders no other CPU would.
      - anyfoo 4y ago
        > M1 doesn't have a special mode for Rosetta. All code is executed with x86 TSO on M1's application processors. That's not true. (And doesn't your last paragraph contradict it already?) You might just have figured out that most stuff will run fine (or appear to run fine for a long time) when TSO isn't enabled.
        messe 4y ago
        I have no idea why you are being downvoted. You are entirely correct.
        anyfoo 4y ago
        Thanks, I was puzzled as well. The downvotes seem to have stopped, though.
      - messe 4y ago
        I'm sorry, but please cite some sources, because this contradicts everything that's been said about M1's x86 emulation that I've read so far. > Well, did you know Apple ported Rosetta 2 to Linux? You can get it by running a Linux VM on macOS. It does not require any kernel changes to support in VMs, and if you extract the binary to run it on Asahi Linux, it works just fine too. None of the Asahi team did anything to support x86 TSO. Rosetta also works just fine in m1n1's hypervisor mode, which exists specifically to log all hardware access to detect these sorts of things. If there is a hardware toggle for TSO, it's either part of the chicken bits (and thus enabled all the time anyway) or turned on by iBoot (and thus enabled before any user code runs). Apple tells you to attach a special volume/FS to your linux VM in order for Rosetta to work. When such a volume is attached, it runs the VM in TSO mode. As simple as that. The rosetta binary itself doesn't know whether or not TSO is enabled, so its not surprising that it runs fine under Asahi. As marcan42 himself said on twitter[1], most x86 applications will run fine even without TSO enabled. You're liable to run into edge cases in heavily multithreaded code though. [1]: https://twitter.com/marcan42/status/1534054757421432833 https://twitter.com/marcan42/status/1534054757421432833 > Both of those happened purely because M1 and M2 are so hilariously wide and speculative that they trigger memory reorders no other CPU would. In other words, they're not constantly running in TSO mode? Because if they were, why would they trigger such re-orders? EDIT: I've just run a modified version of the following test program[2] (removing the references to the tso_enable sysctl which requires an extension), both native and under Rosetta. Running natively, it fails after ~3500 iterations. Under Rosetta, it completes the entire test successfully. [2] https://github.com/losfair/tsotest/ https://github.com/losfair/tsotest/
  - School-Cotton 4y ago
    If you're inventing a completely incompatible ISA, why not just use ARM64 at that point?
    - anamax 4y ago
      Perhaps because you don't want to commit to ARM compatiblity AND licensing fees.
      - yjftsjthsd-h 4y ago
        Alright, then RISC-V?
  - hakfoo 4y ago
    I always wondered if there was a case to be made for a JIT that recompiled "normal" x86-64 into a restricted subset designed to be more optimizable. This could be paired with a sort of two-tiered processor design-- really fast on the restricted subset, but mediocre-to-bad performance on anything else. The code only has to activate the "mediocre to bad" paths long enough to build a JIT cache though,
    - deleted 4y ago
      [deleted]
    - therealcamino 4y ago
      That's sort of similar to Transmeta's approach, except the underlying machine was pretty different from x86, not a fast subset.
- orionblastar 4y ago
  Legacy features are still needed for 16 bit DOS and Windows programs. You can cut out 16 bit legacy mode and use DOSBOX: https://www.dosbox.com/ https://www.dosbox.com/ for your 16 bit programs via emulation. There are those users who still want to use DOS programs: https://www.freedos.org/ https://www.freedos.org/
  - tinus_hn 4y ago
    Considering this ‘needed’ legacy application is running code made for a 8MHz processor, you can trivially get away with emulating that processor on the multi GHz CPU in a recent computer. By the way, 64 bit Windows doesn’t run 16 bits applications and never has. The only relevant 16 bit code is InstallShield and that is emulated using trickery. For the rest you need to emulate a whole computer. https://docs.microsoft.com/en-us/troubleshoot/windows-client/application-management/x64-windows-not-support-16-bit-programs https://docs.microsoft.com/en-us/troubleshoot/windows-client...
    - amluto 4y ago
      But Linux does run 16-bit code natively, even on x86_64, via modify_ldt(). The kernel support is straightforward, and the user side is actively supported by DOSEMU2. Sadly, there was a serious design error in the original x86_64 support for i387 floating point, and it was never possible for a 64-bit kernel to correctly context switch some of the more obscure legacy floating point exception state. Rather than fixing it, both AMD and Intel nerfed the hardware support, and new CPUs no longer fully support this state. This makes some older 16-bit software work poorly.
      - tinus_hn 4y ago
        Then again there is no 16 bit Linux software as there has never been 16 bit Linux, and 16-bit software for other operating systems is trivially emulated without requiring hardware support. It might all be somewhat of an interesting exercise but it doesn’t have real world value.
gabereiser 4y ago
To someone who is interested in bare metal, can you explain the significance of this? Is this how much data a CPU can handle simultaneously? Via instructions from the kernel?
- anyfoo 4y ago
  It means how many instructions the CPU can decode at the same time, roughly to "figure out what they mean and dispatch what they actually have to do to the functional units of the CPU which will perform the work of the instruction". It is not directly how much data a superscalar CPU can handle in parallel, but still plays a role, in the sense that there is a number of functional units available in the CPU, and if you cannot keep those busy with decoded instructions, then they lay around unused. So a too narrow decoder can be one of the bottlenecks in optimal CPU usage (but note how as a sibling commenter mentioned, the complexity of the instructions/architecture is also important, e.g. a single CISC instruction may keep things pretty busy by itself). Whether the instructions come from the kernel or from userspace does not matter at all, they all go through the same decoder and functional units. The kernel/userspace differentiation is a higher level concept.
monocasa 4y ago
It's more complex than that if you'll excuse the pun. Instructions on CISC cores aren't 1 to 1 with RISC instructions, and tend to encode quite a bit more micro ops. Something like inc dword [rbp+16] is one instruction, but would be a minimum of three micro ops (and would be three RISC instructions as well). Long story short, this isn't really the bottle neck, or we'd see more simple decoders on the tail end of the decode window.
mhh__ 4y ago
Decode bound performance issues are actually pretty rare. X86 is quite dense.

by unnah 4y ago

amluto 4y ago

Sadly not. x86_64’s encoding is extremely similar to the legacy encodings. AIUI the fundamental problem is that x86 is a variable-length encoding, so a fully parallel decoder needs to decode at guessed offsets, many of which will be wrong. ARM64 instructions are aligned. Dumping legacy features would be great for all kinds of reasons, but not this particular reason.

hyperman1 4y ago
This suggests another way forward: Re-encode the existing opcodes with new, more regular byte sequences. E.g. 32 bits / instruction, with some escape for e.g. 64bit constants. You'll have to redo the backend of the assembler, but most of the compiler and optimization wisdom can be reused as-is. Of course, this breaks backward compatibility completely so the high performance mode can only be unlocked for recompiles.
- colejohnson66 4y ago
  That was Itanium, and it failed for a variety of reasons; one of which was a compatibility layer that sucked. You can't get rid of x86's backwards compatibility. Intel and AMD have done their best by using vector prefixes (like VEX and EVEX)[a] that massively simplify decoding, but there's only so much that can be done. People get caught up in the variable length issue that x86 has, and then claim that M1 beats x86 because of that. Sure, decoding ARM instructions is easier than x86, but the variable length aspect is handled in the predecode/cache stage, not the actual decoder. The decoder, when it reaches an instruction, already knows various bits of info are. The RISC vs CISC debate is useless today. M1's big advantage comes from the memory ordering model (and other things)[0], not the instruction format. Apple actually had to create a special mode for the M1 (for Rosetta 2) that enforces the x86 ordering model (TSO with load forwarding), and native performance is slightly worse when doing so. [0]: https://twitter.com/ErrataRob/status/1331735383193903104 https://twitter.com/ErrataRob/status/1331735383193903104 [a]: There's also others that predate AVX (VEX) such as the 0F38 prefix group consisting only of opcodes that have a ModR/M byte and no immediate, and the 0F3A prefix being the same, but with an 8 bit immediate.
  - kmeisthax 4y ago
    M1 doesn't have a special mode for Rosetta. All code is executed with x86 TSO on M1's application processors. How do I know this? Well, did you know Apple ported Rosetta 2 to Linux? You can get it by running a Linux VM on macOS. It does not require any kernel changes to support in VMs, and if you extract the binary to run it on Asahi Linux, it works just fine too. None of the Asahi team did anything to support x86 TSO. Rosetta also works just fine in m1n1's hypervisor mode, which exists specifically to log all hardware access to detect these sorts of things. If there is a hardware toggle for TSO, it's either part of the chicken bits (and thus enabled all the time anyway) or turned on by iBoot (and thus enabled before any user code runs). Related point: Hector Martin just upstreamed a patch to Linux that fixes a memory ordering bug in workqueues that's been around since before Linux had Git history. He also found a bug in some ARM litmus tests that he was using to validate whether or not they were implemented correctly. Both of those happened purely because M1 and M2 are so hilariously wide and speculative that they trigger memory reorders no other CPU would.
    - anyfoo 4y ago
      > M1 doesn't have a special mode for Rosetta. All code is executed with x86 TSO on M1's application processors. That's not true. (And doesn't your last paragraph contradict it already?) You might just have figured out that most stuff will run fine (or appear to run fine for a long time) when TSO isn't enabled.
      - messe 4y ago
        I have no idea why you are being downvoted. You are entirely correct.
        anyfoo 4y ago
        Thanks, I was puzzled as well. The downvotes seem to have stopped, though.
    - messe 4y ago
      I'm sorry, but please cite some sources, because this contradicts everything that's been said about M1's x86 emulation that I've read so far. > Well, did you know Apple ported Rosetta 2 to Linux? You can get it by running a Linux VM on macOS. It does not require any kernel changes to support in VMs, and if you extract the binary to run it on Asahi Linux, it works just fine too. None of the Asahi team did anything to support x86 TSO. Rosetta also works just fine in m1n1's hypervisor mode, which exists specifically to log all hardware access to detect these sorts of things. If there is a hardware toggle for TSO, it's either part of the chicken bits (and thus enabled all the time anyway) or turned on by iBoot (and thus enabled before any user code runs). Apple tells you to attach a special volume/FS to your linux VM in order for Rosetta to work. When such a volume is attached, it runs the VM in TSO mode. As simple as that. The rosetta binary itself doesn't know whether or not TSO is enabled, so its not surprising that it runs fine under Asahi. As marcan42 himself said on twitter[1], most x86 applications will run fine even without TSO enabled. You're liable to run into edge cases in heavily multithreaded code though. [1]: https://twitter.com/marcan42/status/1534054757421432833 https://twitter.com/marcan42/status/1534054757421432833 > Both of those happened purely because M1 and M2 are so hilariously wide and speculative that they trigger memory reorders no other CPU would. In other words, they're not constantly running in TSO mode? Because if they were, why would they trigger such re-orders? EDIT: I've just run a modified version of the following test program[2] (removing the references to the tso_enable sysctl which requires an extension), both native and under Rosetta. Running natively, it fails after ~3500 iterations. Under Rosetta, it completes the entire test successfully. [2] https://github.com/losfair/tsotest/ https://github.com/losfair/tsotest/
- School-Cotton 4y ago
  If you're inventing a completely incompatible ISA, why not just use ARM64 at that point?
  - anamax 4y ago
    Perhaps because you don't want to commit to ARM compatiblity AND licensing fees.
    - yjftsjthsd-h 4y ago
      Alright, then RISC-V?
- hakfoo 4y ago
  I always wondered if there was a case to be made for a JIT that recompiled "normal" x86-64 into a restricted subset designed to be more optimizable. This could be paired with a sort of two-tiered processor design-- really fast on the restricted subset, but mediocre-to-bad performance on anything else. The code only has to activate the "mediocre to bad" paths long enough to build a JIT cache though,
  - deleted 4y ago
    [deleted]
  - therealcamino 4y ago
    That's sort of similar to Transmeta's approach, except the underlying machine was pretty different from x86, not a fast subset.
orionblastar 4y ago
Legacy features are still needed for 16 bit DOS and Windows programs. You can cut out 16 bit legacy mode and use DOSBOX: https://www.dosbox.com/ https://www.dosbox.com/ for your 16 bit programs via emulation. There are those users who still want to use DOS programs: https://www.freedos.org/ https://www.freedos.org/
- tinus_hn 4y ago
  Considering this ‘needed’ legacy application is running code made for a 8MHz processor, you can trivially get away with emulating that processor on the multi GHz CPU in a recent computer. By the way, 64 bit Windows doesn’t run 16 bits applications and never has. The only relevant 16 bit code is InstallShield and that is emulated using trickery. For the rest you need to emulate a whole computer. https://docs.microsoft.com/en-us/troubleshoot/windows-client/application-management/x64-windows-not-support-16-bit-programs https://docs.microsoft.com/en-us/troubleshoot/windows-client...
  - amluto 4y ago
    But Linux does run 16-bit code natively, even on x86_64, via modify_ldt(). The kernel support is straightforward, and the user side is actively supported by DOSEMU2. Sadly, there was a serious design error in the original x86_64 support for i387 floating point, and it was never possible for a 64-bit kernel to correctly context switch some of the more obscure legacy floating point exception state. Rather than fixing it, both AMD and Intel nerfed the hardware support, and new CPUs no longer fully support this state. This makes some older 16-bit software work poorly.
    - tinus_hn 4y ago
      Then again there is no 16 bit Linux software as there has never been 16 bit Linux, and 16-bit software for other operating systems is trivially emulated without requiring hardware support. It might all be somewhat of an interesting exercise but it doesn’t have real world value.

gabereiser 4y ago

To someone who is interested in bare metal, can you explain the significance of this? Is this how much data a CPU can handle simultaneously? Via instructions from the kernel?

anyfoo 4y ago
It means how many instructions the CPU can decode at the same time, roughly to "figure out what they mean and dispatch what they actually have to do to the functional units of the CPU which will perform the work of the instruction". It is not directly how much data a superscalar CPU can handle in parallel, but still plays a role, in the sense that there is a number of functional units available in the CPU, and if you cannot keep those busy with decoded instructions, then they lay around unused. So a too narrow decoder can be one of the bottlenecks in optimal CPU usage (but note how as a sibling commenter mentioned, the complexity of the instructions/architecture is also important, e.g. a single CISC instruction may keep things pretty busy by itself). Whether the instructions come from the kernel or from userspace does not matter at all, they all go through the same decoder and functional units. The kernel/userspace differentiation is a higher level concept.

monocasa 4y ago

It's more complex than that if you'll excuse the pun. Instructions on CISC cores aren't 1 to 1 with RISC instructions, and tend to encode quite a bit more micro ops. Something like inc dword [rbp+16] is one instruction, but would be a minimum of three micro ops (and would be three RISC instructions as well). Long story short, this isn't really the bottle neck, or we'd see more simple decoders on the tail end of the decode window.

mhh__ 4y ago

Decode bound performance issues are actually pretty rare. X86 is quite dense.