6 ms·

Bonus quirk: there's BSF/BSR, for which the Intel SDM states that on zero input, the destination has an undefined value. (AMD documents that the destination is

Bonus quirk: there's BSF/BSR, for which the Intel SDM states that on zero input, the destination has an undefined value. (AMD documents that the destination is not modified in that case.) And then there's glibc, which happily uses the undocumented fact that the destination is also unmodified on Intel [1]. It took me quite some time to track down the issue in my binary translator. (There's also TZCNT/LZCNT, which is BSF/BSR encoded with F3-prefix -- which is silently ignored on older processors not supporting the extension. So the same code will behave differently on different CPUs. At least, that's documented.) Encoding: People often complain about prefixes, but IMHO, that's by far not the worst thing. It is well known and somewhat well documented. There are worse quirks: For example, REX/VEX/EVEX.RXB extension bits are ignored when they do not apply (e.g., MMX registers); except for mask registers (k0-k7), where they trigger #UD -- also fine -- except if the register is encoded in ModRM.rm, in which case the extension bit is ignored again. APX takes the number of quirks to a different level: the REX2 prefix can encode general-purpose registers r16-r31, but not xmm16-xmm31; the EVEX prefix has several opcode-dependent layouts; and the extension bits for a register used depend on the register type (XMM registers use X3:B3:rm and V4:X3:idx; GP registers use B4:B3:rm, X4:X3:idx). I can't give a complete list yet, I still haven't finished my APX decoder after a year... [1]: https://sourceware.org/bugzilla/show_bug.cgi?id=31748 https://sourceware.org/bugzilla/show_bug.cgi?id=31748

__turbobrew__ 2y ago
I know nothing about this space, but it would be interesting to hook up a jtag interface to a x86 CPU and them step instruction by instruction and record all the register values. You could then use this data to test whether or not your emulator perfectly emulates the hardware by running the same program through the emulated CPU and validate the state is the same at every instruction.
- saagarjha 2y ago
  No need to JTAG; you can test most of the processor in virtualization. The only part you can’t check is the interface for that itself if you’re aiming to emulate it (most don’t). (Also I’m pretty sure the debugging interface is either fused off or locked out on modern Intel processors.)
tyfighter 2y ago
The BSF/BSR quirk is annoying, but I think there is a reason for it is that they were only thinking about it being used in a loop (or maybe an if) with something like: int mask = something; ... for (int index; _bit_scan_forward(&index, mask); mask ^= 1<<index) { ... } Since it sets the ZF on a zero input, they thought that must be all you need. But there are many other uses for (trailing|leading) zero count operations, and it would have been much better for them to just write the register anyway.
CoastalCoder 2y ago
Can you imagine having to make all this logic work faithfully, let alone fast, in silicon? X86 used to be Intel's moat, but what a nightmarish burden to carry.
- turndown 2y ago
  Intel is coming out with an improved x86 instruction set that removes a lot of the cruft, called ‘APX’ for advanced performance extensions.
  - astrange 2y ago
    Amusingly it looks very much like arm64. It's hard to call it an improvement when the old instructions are still around though.
- dx4100 2y ago
  Did people just... do this by hand (in software), transistor by transistor, or was it laid out programmatically in some sense? As in, were segments created algorithmically, then repeated to obtain the desired outcome? CPU design baffles me, especially considering there are 134 BILLION transistors or so in the latest i7 CPU. How does the team even keep track of, work on, or even load the files to WORK on the CPUs?
  - tristor 2y ago
    They use EDA (Electronic Design Automation) software, there are only a handful of vendors, the largest probably being Mentor Graphics, now owned by Siemens. So, yes, they use automation to algorithmically build and track/resolve refactors as they design CPUs. CPUs are /generally/ block-type designs these days, so particular functions get repeated identically in different places and can be somewhat abstracted away in your EDA. It's still enormously complex, and way more complex than the last time I touched this stuff more than 15 years ago.
    - rikthevik 2y ago
bonzini 2y ago
On and off over the last year I have been rewriting QEMU's x86 decoder. It started as a necessary task to incorporate AVX support, but I am now at a point where only a handful of opcodes are left to rewrite, after which it should not be too hard to add APX support. For EVEX my plan is to keep the raw bits until after the opcode has been read (i.e. before immediates and possibly before modrm) and the EVEX class identified. My decoder is mostly based on the tables in the manual, and the code is mostly okay—not too much indentation and phases mostly easy to separate/identify. Because the output is JITted code, it's ok to not be super efficient and keep the code readable; it's not where most of the time is spent. Nevertheless there are several cases in which the manual is wrong or doesn't say the whole story. And the tables haven't been updated for several years (no K register instructions, for example), so going forward there will be more manual work to do. :( The top comment explains a bit what's going on: https://github.com/qemu/qemu/blob/59084feb256c617063e0dbe7e64821ae8852d7cf/target/i386/tcg/decode-new.c.inc#L22 https://github.com/qemu/qemu/blob/59084feb256c617063e0dbe7e6... (As I said above, there are still a few instructions handled by the old code predating the rewrite, notably BT/BTS/BTR/BTC. I have written the code but not merged it yet).
- aengelke 2y ago
  Thanks for the pointer to QEMU's decoder! I actually never looked at it before. So you coded all the tables manually in C -- interesting, that's quite some effort. I opted to autogenerate the tables (and keep them as data only => smaller memory footprint) [1,2]. That's doable, because x86 encodings are mostly fairly consistent. I can also generate an encoder from it (ok, you don't need that). Re 'custom size "xh"': AVX-512 also has fourth and eighth. Also interesting that you have a separate row for "66+F2". I special case these two (CRC32, MOVBE) instructions with a flag. I think the prefix decoding is not quite right for x86-64: 26/2e/36/3e are ignored in 64-bit mode, except for 2e/3e as branch-not-taken/taken hints and 3e as notrack. (See SDM Vol. 1 3.3.7.1 "Other segment override prefixes (CS, DS, ES, and SS) are ignored.") Also, REX prefixes that don't immediately preceed the opcode (or VEX/EVEX prefix) are ignored. Anyhow, I need to take a closer look at the decoder with more time. :-) > For EVEX my plan is to keep the raw bits until after the opcode has been read I came to the same conclusion that this is necessary with APX. The map+prefix+opcode combination identifies how the other fields are to be interpreted. For AVX-512, storing the last byte was sufficient, but with APX, vvvv got a second meaning. > Nevertheless there are several cases in which the manual is wrong or doesn't say the whole story. Yes... especially for corner cases, getting real hardware is the only reliable way to find out, how the CPU behaves. [1]: https://github.com/aengelke/fadec/blob/master/instrs.txt https://github.com/aengelke/fadec/blob/master/instrs.txt [2]: https://github.com/aengelke/fadec/blob/master/decode.c https://github.com/aengelke/fadec/blob/master/decode.c
mananaysiempre 2y ago
The semantics of LZCNT combined with its encoding feels like an own goal: it’s encoded as a BSR instruction with a legacy-ignored prefix, but for nonzero inputs its return value is the operand size minus the return value of the legacy version. Yes, clz() is a function that exists, but the extra subtraction in its implementation feels like a small cost to pay for extra compatibility when LZCNT could’ve just been BSR with different zero-input semantics.
- bonzini 2y ago
  Yes, it's like someone looked at TZCNT and thought "let's encode LZCNT the same way", but it makes no sense.
  - adrian_b 2y ago
    LZCNT and TZCNT are corrections (originally introduced by AMD) for the serious mistake done by the designers of Intel 80386 when they have defined BSF and BSR. Because on the very slow 80386 the wrong definition for the null input did not matter much, they have failed to foresee how bad it will become for the future pipelined and superscalar CPUs, where having to insert a test for null input can slow down a program many times. Nevertheless, they should have paid more attention to the earlier use of such instructions. For instance Cray-1 had defined LZCNT in the right way almost ten years earlier.
    - bonzini 2y ago
      LZCNT was introduced by Intel in BMI1, while TZCNT was introduced by AMD.
- BeeOnRope 2y ago
  I'm not following: as long as you are introducing a new, incompatible instruction for leading zero counting, you'd definitely choose LZCNT over BSR as LZCNT has definitely won in retrospect over BSR as the primitive for this use case. BSR is just a historical anomaly which has a zero-input problem for no benefit. What would be the point of offering a new variation BSR with different input semantics?
torusle 2y ago
Another bonus quirk, from the 486 and Pentium area.. BSWAP EAX converts from little endian to big endian and vice versa. It was a 32 bit instruction to begin with. However, we have the 0x66 prefix that switches between 16 and 32 bit mode. If you apply that to BSWAP EAX undefined funky things happen. On some CPU architectures (Intel vs. AMD) the prefix was just ignored. On others it did something that I call an "inner swap". E.g. in your four bytes that are stored in EAX byte 1 and 2 are swapped. 0x11223344 became 0x11332244.
- userbinator 2y ago
  Also known as "bswap ax", and research shows that it does something surprising but consistent on almost all hardware: It zeros the register. https://www.ragestorm.net/blogs/?p=141 https://www.ragestorm.net/blogs/?p=141 https://gynvael.coldwind.pl/?id=268 https://gynvael.coldwind.pl/?id=268 However, this page, now gone, suggests that some CPUs (early 486s?) did something different: http://web.archive.org/web/20071231192014/http://www.df.lth.se/~john_e/gems/gem000c.html http://web.archive.org/web/20071231192014/http://www.df.lth.... Unfortunately I have not found any evidence nor reason to believe that this "inner swap" behaviour you mention exists in some CPU --- except perhaps some flawed emulators?
  - deleted 2y ago
    [deleted]

__turbobrew__ 2y ago

I know nothing about this space, but it would be interesting to hook up a jtag interface to a x86 CPU and them step instruction by instruction and record all the register values. You could then use this data to test whether or not your emulator perfectly emulates the hardware by running the same program through the emulated CPU and validate the state is the same at every instruction.

saagarjha 2y ago
No need to JTAG; you can test most of the processor in virtualization. The only part you can’t check is the interface for that itself if you’re aiming to emulate it (most don’t). (Also I’m pretty sure the debugging interface is either fused off or locked out on modern Intel processors.)

tyfighter 2y ago

The BSF/BSR quirk is annoying, but I think there is a reason for it is that they were only thinking about it being used in a loop (or maybe an if) with something like: int mask = something; ... for (int index; _bit_scan_forward(&index, mask); mask ^= 1<<index) { ... } Since it sets the ZF on a zero input, they thought that must be all you need. But there are many other uses for (trailing|leading) zero count operations, and it would have been much better for them to just write the register anyway.

CoastalCoder 2y ago

Can you imagine having to make all this logic work faithfully, let alone fast, in silicon? X86 used to be Intel's moat, but what a nightmarish burden to carry.

turndown 2y ago
Intel is coming out with an improved x86 instruction set that removes a lot of the cruft, called ‘APX’ for advanced performance extensions.
- astrange 2y ago
  Amusingly it looks very much like arm64. It's hard to call it an improvement when the old instructions are still around though.
dx4100 2y ago
Did people just... do this by hand (in software), transistor by transistor, or was it laid out programmatically in some sense? As in, were segments created algorithmically, then repeated to obtain the desired outcome? CPU design baffles me, especially considering there are 134 BILLION transistors or so in the latest i7 CPU. How does the team even keep track of, work on, or even load the files to WORK on the CPUs?
- tristor 2y ago
  They use EDA (Electronic Design Automation) software, there are only a handful of vendors, the largest probably being Mentor Graphics, now owned by Siemens. So, yes, they use automation to algorithmically build and track/resolve refactors as they design CPUs. CPUs are /generally/ block-type designs these days, so particular functions get repeated identically in different places and can be somewhat abstracted away in your EDA. It's still enormously complex, and way more complex than the last time I touched this stuff more than 15 years ago.
  - rikthevik 2y ago

bonzini 2y ago

On and off over the last year I have been rewriting QEMU's x86 decoder. It started as a necessary task to incorporate AVX support, but I am now at a point where only a handful of opcodes are left to rewrite, after which it should not be too hard to add APX support. For EVEX my plan is to keep the raw bits until after the opcode has been read (i.e. before immediates and possibly before modrm) and the EVEX class identified. My decoder is mostly based on the tables in the manual, and the code is mostly okay—not too much indentation and phases mostly easy to separate/identify. Because the output is JITted code, it's ok to not be super efficient and keep the code readable; it's not where most of the time is spent. Nevertheless there are several cases in which the manual is wrong or doesn't say the whole story. And the tables haven't been updated for several years (no K register instructions, for example), so going forward there will be more manual work to do. :( The top comment explains a bit what's going on: https://github.com/qemu/qemu/blob/59084feb256c617063e0dbe7e64821ae8852d7cf/target/i386/tcg/decode-new.c.inc#L22 https://github.com/qemu/qemu/blob/59084feb256c617063e0dbe7e6... (As I said above, there are still a few instructions handled by the old code predating the rewrite, notably BT/BTS/BTR/BTC. I have written the code but not merged it yet).

aengelke 2y ago
Thanks for the pointer to QEMU's decoder! I actually never looked at it before. So you coded all the tables manually in C -- interesting, that's quite some effort. I opted to autogenerate the tables (and keep them as data only => smaller memory footprint) [1,2]. That's doable, because x86 encodings are mostly fairly consistent. I can also generate an encoder from it (ok, you don't need that). Re 'custom size "xh"': AVX-512 also has fourth and eighth. Also interesting that you have a separate row for "66+F2". I special case these two (CRC32, MOVBE) instructions with a flag. I think the prefix decoding is not quite right for x86-64: 26/2e/36/3e are ignored in 64-bit mode, except for 2e/3e as branch-not-taken/taken hints and 3e as notrack. (See SDM Vol. 1 3.3.7.1 "Other segment override prefixes (CS, DS, ES, and SS) are ignored.") Also, REX prefixes that don't immediately preceed the opcode (or VEX/EVEX prefix) are ignored. Anyhow, I need to take a closer look at the decoder with more time. :-) > For EVEX my plan is to keep the raw bits until after the opcode has been read I came to the same conclusion that this is necessary with APX. The map+prefix+opcode combination identifies how the other fields are to be interpreted. For AVX-512, storing the last byte was sufficient, but with APX, vvvv got a second meaning. > Nevertheless there are several cases in which the manual is wrong or doesn't say the whole story. Yes... especially for corner cases, getting real hardware is the only reliable way to find out, how the CPU behaves. [1]: https://github.com/aengelke/fadec/blob/master/instrs.txt https://github.com/aengelke/fadec/blob/master/instrs.txt [2]: https://github.com/aengelke/fadec/blob/master/decode.c https://github.com/aengelke/fadec/blob/master/decode.c

mananaysiempre 2y ago

The semantics of LZCNT combined with its encoding feels like an own goal: it’s encoded as a BSR instruction with a legacy-ignored prefix, but for nonzero inputs its return value is the operand size minus the return value of the legacy version. Yes, clz() is a function that exists, but the extra subtraction in its implementation feels like a small cost to pay for extra compatibility when LZCNT could’ve just been BSR with different zero-input semantics.

bonzini 2y ago
Yes, it's like someone looked at TZCNT and thought "let's encode LZCNT the same way", but it makes no sense.
- adrian_b 2y ago
  LZCNT and TZCNT are corrections (originally introduced by AMD) for the serious mistake done by the designers of Intel 80386 when they have defined BSF and BSR. Because on the very slow 80386 the wrong definition for the null input did not matter much, they have failed to foresee how bad it will become for the future pipelined and superscalar CPUs, where having to insert a test for null input can slow down a program many times. Nevertheless, they should have paid more attention to the earlier use of such instructions. For instance Cray-1 had defined LZCNT in the right way almost ten years earlier.
  - bonzini 2y ago
    LZCNT was introduced by Intel in BMI1, while TZCNT was introduced by AMD.
BeeOnRope 2y ago
I'm not following: as long as you are introducing a new, incompatible instruction for leading zero counting, you'd definitely choose LZCNT over BSR as LZCNT has definitely won in retrospect over BSR as the primitive for this use case. BSR is just a historical anomaly which has a zero-input problem for no benefit. What would be the point of offering a new variation BSR with different input semantics?

torusle 2y ago

Another bonus quirk, from the 486 and Pentium area.. BSWAP EAX converts from little endian to big endian and vice versa. It was a 32 bit instruction to begin with. However, we have the 0x66 prefix that switches between 16 and 32 bit mode. If you apply that to BSWAP EAX undefined funky things happen. On some CPU architectures (Intel vs. AMD) the prefix was just ignored. On others it did something that I call an "inner swap". E.g. in your four bytes that are stored in EAX byte 1 and 2 are swapped. 0x11223344 became 0x11332244.

userbinator 2y ago
Also known as "bswap ax", and research shows that it does something surprising but consistent on almost all hardware: It zeros the register. https://www.ragestorm.net/blogs/?p=141 https://www.ragestorm.net/blogs/?p=141 https://gynvael.coldwind.pl/?id=268 https://gynvael.coldwind.pl/?id=268 However, this page, now gone, suggests that some CPUs (early 486s?) did something different: http://web.archive.org/web/20071231192014/http://www.df.lth.se/~john_e/gems/gem000c.html http://web.archive.org/web/20071231192014/http://www.df.lth.... Unfortunately I have not found any evidence nor reason to believe that this "inner swap" behaviour you mention exists in some CPU --- except perhaps some flawed emulators?
- deleted 2y ago
  [deleted]