5 ms·

I believe this was a design bug: the hardware followed the design but nobody realized that the design had a problem. Specifically, it takes two instructions to

by kens 4y ago

I believe this was a design bug: the hardware followed the design but nobody realized that the design had a problem. Specifically, it takes two instructions to move the stack pointer to a different segment (updating the SS and the SP). If you get an interrupt between these two instructions, everything is deterministic and "works". The problem is that the combination of the old SP and the new SS points to an unexpected place in memory, so your stack frame is going to clobber something. The hardware is doing the "right thing" but the behavior is unusable.

ajross 4y ago
Huh... well that's disappointing. That doesn't sound like a hardware bug to me at all. There's nothing unhandleable about this circumstance that I see, any combination of SS/SP points to a "valid" address, code just has to be sure the combination always points to enough memory to store an interrupt frame. In the overwhelmingly common situation where your stack started with an SP of 0, you just assign SP back to 0 before setting SS and nothing can go wrong (you are, after all, about to move off this stack and don't care about its contents). Even the weirdest setups can be handled with a quick thunk through an SP that overlaps in the same 64k region as the target. It's a clumsy interface but hardly something that Intel has traditionally shied away from (I mean, good grief, fast forward a half decade and they'll have inflicted protected mode transitions on us). I'm just surprised they felt this was worth a hardware patch. I guess I was hoping for something juicier.
- pm215 4y ago
  The problem is that in general "code" can't do that because the code that's switching stack doesn't necessarily conceptually control the stack it's switching to/from. One SS:SP might point to a userspace process, for instance, and the other SS:SP be for the kernel. Both stacks are "live" in that they still have data that will later be context switched back to. You might be able to come up with a convoluted workaround, but since "switch between stacks" was a design goal (and probably assumed by various OS designs they were hoping to get ports for), the race that meant you can't do it safely was definitely a design bug.
- munch117 4y ago
  > That doesn't sound like a hardware bug to me at all. It isn't. It's a rare case of a software bug being worked around in hardware. That alone makes it fascinating. The natural way of handling it in software would be to disable interrupts temporarily while changing the stack segment and pointer. You could just say that software developers should do that, but there's a performance cost, and more importantly: The rare random failures from the software already out there that didn't do this, would give your precious new CPU a reputation for being unstable. Better to just make it work.
  - cesarb 4y ago
    As others have already noted: yes, software could disable interrupts temporarily while changing the stack segment and pointer, but on these processors, there's a second kind of interrupt ("non-maskable interrupt" aka NMI) which cannot be disabled. (Some machines using these processors have hardware to externally gate the NMI input to the processor, but disabling the NMI through these methods defeats the whole purpose of the NMI, which is being able to interrupt the processor even when it has disabled interrupts.)
    - munch117 4y ago
      I saw those other 8087+NMI comments, but the 8087 is from 1980, and I thought this fix was older than that.
      - anyfoo 4y ago
        The 8087 just happened to be one particular source of NMIs because IBM decided to connect it to the 8086’s NMI line. NMIs can be used for many things. Without this fix, they’re breaking stuff, at least without another hideous hardware workaround.
- caf 4y ago
  In the overwhelmingly common situation where your stack started with an SP of 0, you just assign SP back to 0 before setting SS and nothing can go wrong (you are, after all, about to move off this stack and don't care about its contents). Consider the case when the MSDOS command interpreter is handing off execution to a user program. It can't just set SP to 0 first (well, really 0xFFFF since the stack grows down), because it's somewhere several frames deep in its own execution stack, an execution stack which will hopefully resume when the user program exits. Programs using the 'tiny' memory model, like all COM files, have SS == CS == DS so new_ss:old_sp could easily clobber the code or data from the loaded file.
  - ajross 4y ago
    Right, so you just thunk to a different region at the bottom of your own stack in that case. I'm not saying this is easy or simple, I'm saying it's possible to do it reliably entirely in software at the OS level without the need for patching a hardware mask. Edit, just because all the folks jumping in to downvote and tell me I'm wrong are getting under my skin, this (untested, obviously) sequence should work to safely set SP to zero in an interrupt-safe way, all it requires is that you have enough space under your current stack to align SP down to a 16 bit boundary and fit 16 bytes and one interrupt frame below that. Again, you wouldn't use this in production OS code, you'd just make sure there was a frame of space at the top of your stack or be otherwise able to recover from an interrupt. But you absolutely can do this. mov %sp, %ax // Align SP down to a segment boundary and %ax, ~0xf mov %ax, %sp loop: // Iteratively: mov %ss, %ax // + decrement the segment (safe) dec %ax mov %ax, %ss mov %sp, %ax // + increment the stack pointer by the same amount (safe) add %ax, 16 mov %ax, %sp jnz loop // + Until SP is zero (and points to the same spot!)
    - anyfoo 4y ago
      User software is allowed to switch their stack, too, and user software must be able to be interrupted. Sure, it may be possible to work around this in software with some careful elaborate protocol that involves making sure some value of SP is usable with any value of SS, and prescribing that anything at all adhere to that (because any code at all can be interrupted by an NMI), but that not only means the use of this protocol for all stack layouts, but also implies that every single “MOV SS” or “POP SS” becomes part of some elaborate sequence of instructions (something like “push some other register on the stack, save current SP, in that register, change SP to the magic value, change SS… extra fun if you wanted POP SS, restore SP from the other register, restore the other register”). That’s a lot to put up with, and in my mind qualifies as a design bug with no palpable software workaround. Not great for wanting folks to use your brand new CPU. The protected mode transitions that you mentioned are much more palpable because they only happen in very bespoke parts of the OS kernel, usually only in bootstrap in any non-ancient OS.

by kens 4y ago

ajross 4y ago

Huh... well that's disappointing. That doesn't sound like a hardware bug to me at all. There's nothing unhandleable about this circumstance that I see, any combination of SS/SP points to a "valid" address, code just has to be sure the combination always points to enough memory to store an interrupt frame. In the overwhelmingly common situation where your stack started with an SP of 0, you just assign SP back to 0 before setting SS and nothing can go wrong (you are, after all, about to move off this stack and don't care about its contents). Even the weirdest setups can be handled with a quick thunk through an SP that overlaps in the same 64k region as the target. It's a clumsy interface but hardly something that Intel has traditionally shied away from (I mean, good grief, fast forward a half decade and they'll have inflicted protected mode transitions on us). I'm just surprised they felt this was worth a hardware patch. I guess I was hoping for something juicier.

pm215 4y ago
The problem is that in general "code" can't do that because the code that's switching stack doesn't necessarily conceptually control the stack it's switching to/from. One SS:SP might point to a userspace process, for instance, and the other SS:SP be for the kernel. Both stacks are "live" in that they still have data that will later be context switched back to. You might be able to come up with a convoluted workaround, but since "switch between stacks" was a design goal (and probably assumed by various OS designs they were hoping to get ports for), the race that meant you can't do it safely was definitely a design bug.
munch117 4y ago
> That doesn't sound like a hardware bug to me at all. It isn't. It's a rare case of a software bug being worked around in hardware. That alone makes it fascinating. The natural way of handling it in software would be to disable interrupts temporarily while changing the stack segment and pointer. You could just say that software developers should do that, but there's a performance cost, and more importantly: The rare random failures from the software already out there that didn't do this, would give your precious new CPU a reputation for being unstable. Better to just make it work.
- cesarb 4y ago
  As others have already noted: yes, software could disable interrupts temporarily while changing the stack segment and pointer, but on these processors, there's a second kind of interrupt ("non-maskable interrupt" aka NMI) which cannot be disabled. (Some machines using these processors have hardware to externally gate the NMI input to the processor, but disabling the NMI through these methods defeats the whole purpose of the NMI, which is being able to interrupt the processor even when it has disabled interrupts.)
  - munch117 4y ago
    I saw those other 8087+NMI comments, but the 8087 is from 1980, and I thought this fix was older than that.
    - anyfoo 4y ago
      The 8087 just happened to be one particular source of NMIs because IBM decided to connect it to the 8086’s NMI line. NMIs can be used for many things. Without this fix, they’re breaking stuff, at least without another hideous hardware workaround.
caf 4y ago
In the overwhelmingly common situation where your stack started with an SP of 0, you just assign SP back to 0 before setting SS and nothing can go wrong (you are, after all, about to move off this stack and don't care about its contents). Consider the case when the MSDOS command interpreter is handing off execution to a user program. It can't just set SP to 0 first (well, really 0xFFFF since the stack grows down), because it's somewhere several frames deep in its own execution stack, an execution stack which will hopefully resume when the user program exits. Programs using the 'tiny' memory model, like all COM files, have SS == CS == DS so new_ss:old_sp could easily clobber the code or data from the loaded file.
- ajross 4y ago
  Right, so you just thunk to a different region at the bottom of your own stack in that case. I'm not saying this is easy or simple, I'm saying it's possible to do it reliably entirely in software at the OS level without the need for patching a hardware mask. Edit, just because all the folks jumping in to downvote and tell me I'm wrong are getting under my skin, this (untested, obviously) sequence should work to safely set SP to zero in an interrupt-safe way, all it requires is that you have enough space under your current stack to align SP down to a 16 bit boundary and fit 16 bytes and one interrupt frame below that. Again, you wouldn't use this in production OS code, you'd just make sure there was a frame of space at the top of your stack or be otherwise able to recover from an interrupt. But you absolutely can do this. mov %sp, %ax // Align SP down to a segment boundary and %ax, ~0xf mov %ax, %sp loop: // Iteratively: mov %ss, %ax // + decrement the segment (safe) dec %ax mov %ax, %ss mov %sp, %ax // + increment the stack pointer by the same amount (safe) add %ax, 16 mov %ax, %sp jnz loop // + Until SP is zero (and points to the same spot!)
  - anyfoo 4y ago
    User software is allowed to switch their stack, too, and user software must be able to be interrupted. Sure, it may be possible to work around this in software with some careful elaborate protocol that involves making sure some value of SP is usable with any value of SS, and prescribing that anything at all adhere to that (because any code at all can be interrupted by an NMI), but that not only means the use of this protocol for all stack layouts, but also implies that every single “MOV SS” or “POP SS” becomes part of some elaborate sequence of instructions (something like “push some other register on the stack, save current SP, in that register, change SP to the magic value, change SS… extra fun if you wanted POP SS, restore SP from the other register, restore the other register”). That’s a lot to put up with, and in my mind qualifies as a design bug with no palpable software workaround. Not great for wanting folks to use your brand new CPU. The protected mode transitions that you mentioned are much more palpable because they only happen in very bespoke parts of the OS kernel, usually only in bootstrap in any non-ancient OS.