5 ms·

Reading this makes me realize how little I know of the hardware that runs my software > Prefixes allow you to change how instructions behave by enabling or dis

by xyst 3y ago

Reading this makes me realize how little I know of the hardware that runs my software > Prefixes allow you to change how instructions behave by enabling or disabling features Why do we need “prefixes” to disable or enable features? Is this for dynamically toggling feature so you don’t have to go into BIOS?

jeffbee 3y ago
It's just because x86 as an ISA has accreted over the course of 40+ years, and has variable-length instructions. Every time they extend the ISA they carve out part of the opcode space to squeeze in a new prefix. This will only continue, considering that Intel has proposed another new scheme this year.
shenberg 3y ago
Prefixes are modifiers to specific instructions executed by the processor, e.g. to control the size of the operands or enable locking for concurrency.
Tuna-Fish 3y ago
x86 was designed in 78, basically for the purpose of running a primitive laser printer (or other similar workloads). The big problem with this is that the encoding space for instructions was "efficiently utilized". When new instructions, or worse, additional registers were later added, you had to fit the new instruction variants in somehow, and you did this by tacking on prefixes.
- mschuster91 3y ago
  Nah, x86 goes even earlier in its heritage - it was, effectively, a bolt-on on Intel's way older designs, as a huge part of the 8086 was being ASM source-compatible with the older 8xxx chips, even as the instruction set itself changed [1]. What utterly amazes me is that the original 8086 was mostly designed by hand by a team of not even two dozen people - and today, we got hundreds if not thousands of people working on designing ASICs... [1] https://en.wikipedia.org/wiki/Intel_8086#The_first_x86_design https://en.wikipedia.org/wiki/Intel_8086#The_first_x86_desig...
  - hulitu 3y ago
    It is because testing plays a bigger part today than back then. The complexity has also increased (people do not design at transistor level anymore).
  - irdc 3y ago
    Acckkghtually, if you go back far enough you end up at the Datapoint 2200. If you want to understand where some of the crazier parts of the 8086 originate from, Ken Shirriff has a nice read: http://www.righto.com/2023/08/datapoint-to-8086.html http://www.righto.com/2023/08/datapoint-to-8086.html
- thaumasiotes 3y ago
  > x86 was designed in 78, basically for the purpose of running a primitive laser printer It's interesting that ASCII is transparently just a bunch of control codes for a physical printer/typewriter, combining things like "advance the paper one line", "advance the paper one inch", "reset the carriage position", and "strike an F at the carriage position", all of which are different mechanical actions that you might want a typewriter to do. But now we have Unicode, which is dedicated to the purpose of assigning ID numbers to visual glyphs, and ASCII has been interpreted as a bunch of glyph references instead of a bunch of machine instructions, and there are the control codes with no visual representation, sitting in Unicode, being inappropriate in every possible way. It's kind of like if Unicode were to incorporate "start microwave" as part of a set with "1", "2", "3", etc.
  - rswail 3y ago
    ASCII was used by teletypes, not typewriters. They were "cylinder" heads, as compared to IBM's golfball typewriters. The endless CR/LF/CRLF line ending problem would have been solved if the RS (Record Separator) ASCII code was used instead of the physical CR = carriage return, ie move print head back to start of line, and LF = line feed, ie rotate paper up one line. But Unix decided on LF, Apple used CR, Windows used CRLF, and even today, I had to get a guy to stop setting his system to "Windows" because he was screwing up a git repo with extraneous CRs.
    - deleted 3y ago
      [deleted]
db48x 3y ago
Read https://wiki.osdev.org/X86-64_Instruction_Encoding#Legacy_Prefixes https://wiki.osdev.org/X86-64_Instruction_Encoding#Legacy_Pr... The REP prefixes are the most common; they just let you perform the same instruction a variable number of times. It looks in the CX register for the count. This makes many common loops really, really short, especially for moving objects around in memory. The memcpy function is often inlined as a single REP MOVS instruction, possibly with an instruction to copy the count into CX if it isn’t already there. I suppose the REX (operand size) prefix is pretty common too, since 64–bit programs will want to operate on 64–bit values and addresses pretty frequently. None of the prefixes toggle things that can be set globally, by the BIOS or otherwise. They all just specify things that the next instruction needs to do.
- pclmulqdq 3y ago
  The ModR/M and SIB prefixes are probably the most common prefixes in instructions. They are so common that assemblers elide their existence when you read code. REX is in the same boat: so common that it's usually elided. The VEX prefix is also really common (all of the V* AVX instructions, like VMOVDQ), and then the LOCK prefix (all atomics). After all of those, REP is not that uncommon of a prefix to run into, although many people prefer SIMD memcpy/memset to REP MOVSB/REP STOSB. It is slightly unusual.
  - bonzini 3y ago
    ModRM and SIB are not a prefix, they're part of the opcode (second and third byte after all the prefixes and the 0Fh/0F38h/0F3Ah opcode map selectors)
    - EarlKing 3y ago
      More specifically, they're affixed to certain opcodes that require them. There are a number of byte-sized opcodes that do not require a ModRM or SIB byte (although a number of those got gobbled up to make the REX prefix, but that's another story). TL;DR Weeee! Intel machine language is crazy!
  - 3y ago
epcoa 3y ago
That's a very poor summary of what prefixes are. My advice, just skip the original article which isn't very good or interesting and read taviso's blog that is linked in the top comment (it gives a few concrete examples of these prefixes). They are modifiers that are part of the CPU instruction.
ajross 3y ago
"Prefixes" in this case mostly expand the instruction encoding space. So rarely-used addressing modes get a "segment prefix" that causes them to use a segment other than DS. Or x86_64 added a "REX" prefix that added more bits to the register fields allowing for 16 GPRs. Likewise the "LOCK" prefix (though poorly specified originally) causes (some!) memory operations to be atomic with respect to the rest of the system (c.f. "LOCK CMPXCHG" to effect a compare-and-set). All these things are operations other CPU architectures represent too, though they tend to pack them into the existing instruction space, requiring more bits to represent every instruction. Notably the "REP" prefix in question turns out to be the one exception. This is a microcoded repeat prefix left over from the ancient days. But it represents operations (c.f. memset/memmove) that are performance-sensitive even today, so it's worthwhile for CPU vendors to continue to optimize them. Which is how the bug in question seems to have happened.
jasonwatkinspdx 3y ago
You got some great answers already, but to your first point check out Hennessey and Patterson's books, namely Computer Architecture and Computer Organization and Design. The latter is probably more suited to you unless you wanna go on a dive into computer architecture itself. There's older editions available for free (authorized by the authors) on the web. I first read the 3rd edition of Computer Architecture and besides being one of the most clear textbooks I've ever read it vastly improved my understanding of what's going on in there in relation to OoO speculative execution, etc.

by xyst 3y ago

jeffbee 3y ago

It's just because x86 as an ISA has accreted over the course of 40+ years, and has variable-length instructions. Every time they extend the ISA they carve out part of the opcode space to squeeze in a new prefix. This will only continue, considering that Intel has proposed another new scheme this year.

shenberg 3y ago

Prefixes are modifiers to specific instructions executed by the processor, e.g. to control the size of the operands or enable locking for concurrency.

Tuna-Fish 3y ago

x86 was designed in 78, basically for the purpose of running a primitive laser printer (or other similar workloads). The big problem with this is that the encoding space for instructions was "efficiently utilized". When new instructions, or worse, additional registers were later added, you had to fit the new instruction variants in somehow, and you did this by tacking on prefixes.

mschuster91 3y ago
Nah, x86 goes even earlier in its heritage - it was, effectively, a bolt-on on Intel's way older designs, as a huge part of the 8086 was being ASM source-compatible with the older 8xxx chips, even as the instruction set itself changed [1]. What utterly amazes me is that the original 8086 was mostly designed by hand by a team of not even two dozen people - and today, we got hundreds if not thousands of people working on designing ASICs... [1] https://en.wikipedia.org/wiki/Intel_8086#The_first_x86_design https://en.wikipedia.org/wiki/Intel_8086#The_first_x86_desig...
- hulitu 3y ago
  It is because testing plays a bigger part today than back then. The complexity has also increased (people do not design at transistor level anymore).
- irdc 3y ago
  Acckkghtually, if you go back far enough you end up at the Datapoint 2200. If you want to understand where some of the crazier parts of the 8086 originate from, Ken Shirriff has a nice read: http://www.righto.com/2023/08/datapoint-to-8086.html http://www.righto.com/2023/08/datapoint-to-8086.html
thaumasiotes 3y ago
> x86 was designed in 78, basically for the purpose of running a primitive laser printer It's interesting that ASCII is transparently just a bunch of control codes for a physical printer/typewriter, combining things like "advance the paper one line", "advance the paper one inch", "reset the carriage position", and "strike an F at the carriage position", all of which are different mechanical actions that you might want a typewriter to do. But now we have Unicode, which is dedicated to the purpose of assigning ID numbers to visual glyphs, and ASCII has been interpreted as a bunch of glyph references instead of a bunch of machine instructions, and there are the control codes with no visual representation, sitting in Unicode, being inappropriate in every possible way. It's kind of like if Unicode were to incorporate "start microwave" as part of a set with "1", "2", "3", etc.
- rswail 3y ago
  ASCII was used by teletypes, not typewriters. They were "cylinder" heads, as compared to IBM's golfball typewriters. The endless CR/LF/CRLF line ending problem would have been solved if the RS (Record Separator) ASCII code was used instead of the physical CR = carriage return, ie move print head back to start of line, and LF = line feed, ie rotate paper up one line. But Unix decided on LF, Apple used CR, Windows used CRLF, and even today, I had to get a guy to stop setting his system to "Windows" because he was screwing up a git repo with extraneous CRs.
  - deleted 3y ago
    [deleted]

db48x 3y ago

Read https://wiki.osdev.org/X86-64_Instruction_Encoding#Legacy_Prefixes https://wiki.osdev.org/X86-64_Instruction_Encoding#Legacy_Pr... The REP prefixes are the most common; they just let you perform the same instruction a variable number of times. It looks in the CX register for the count. This makes many common loops really, really short, especially for moving objects around in memory. The memcpy function is often inlined as a single REP MOVS instruction, possibly with an instruction to copy the count into CX if it isn’t already there. I suppose the REX (operand size) prefix is pretty common too, since 64–bit programs will want to operate on 64–bit values and addresses pretty frequently. None of the prefixes toggle things that can be set globally, by the BIOS or otherwise. They all just specify things that the next instruction needs to do.

pclmulqdq 3y ago
The ModR/M and SIB prefixes are probably the most common prefixes in instructions. They are so common that assemblers elide their existence when you read code. REX is in the same boat: so common that it's usually elided. The VEX prefix is also really common (all of the V* AVX instructions, like VMOVDQ), and then the LOCK prefix (all atomics). After all of those, REP is not that uncommon of a prefix to run into, although many people prefer SIMD memcpy/memset to REP MOVSB/REP STOSB. It is slightly unusual.
- bonzini 3y ago
  ModRM and SIB are not a prefix, they're part of the opcode (second and third byte after all the prefixes and the 0Fh/0F38h/0F3Ah opcode map selectors)
  - EarlKing 3y ago
    More specifically, they're affixed to certain opcodes that require them. There are a number of byte-sized opcodes that do not require a ModRM or SIB byte (although a number of those got gobbled up to make the REX prefix, but that's another story). TL;DR Weeee! Intel machine language is crazy!
- 3y ago

epcoa 3y ago

That's a very poor summary of what prefixes are. My advice, just skip the original article which isn't very good or interesting and read taviso's blog that is linked in the top comment (it gives a few concrete examples of these prefixes). They are modifiers that are part of the CPU instruction.

ajross 3y ago

"Prefixes" in this case mostly expand the instruction encoding space. So rarely-used addressing modes get a "segment prefix" that causes them to use a segment other than DS. Or x86_64 added a "REX" prefix that added more bits to the register fields allowing for 16 GPRs. Likewise the "LOCK" prefix (though poorly specified originally) causes (some!) memory operations to be atomic with respect to the rest of the system (c.f. "LOCK CMPXCHG" to effect a compare-and-set). All these things are operations other CPU architectures represent too, though they tend to pack them into the existing instruction space, requiring more bits to represent every instruction. Notably the "REP" prefix in question turns out to be the one exception. This is a microcoded repeat prefix left over from the ancient days. But it represents operations (c.f. memset/memmove) that are performance-sensitive even today, so it's worthwhile for CPU vendors to continue to optimize them. Which is how the bug in question seems to have happened.

jasonwatkinspdx 3y ago

You got some great answers already, but to your first point check out Hennessey and Patterson's books, namely Computer Architecture and Computer Organization and Design. The latter is probably more suited to you unless you wanna go on a dive into computer architecture itself. There's older editions available for free (authorized by the authors) on the web. I first read the 3rd edition of Computer Architecture and besides being one of the most clear textbooks I've ever read it vastly improved my understanding of what's going on in there in relation to OoO speculative execution, etc.