5 ms·

The mysterious second parameter to the x86 ENTER instruction

151 points by matt_d 3y ago

082349872349872 3y ago
> I wonder if anybody uses the Intel design for nested functions TFA gives at least a partial answer to my earlier q: https://news.ycombinator.com/item?id=38544640 https://news.ycombinator.com/item?id=38544640
- tralarpa 3y ago
  The answers in this discussion https://stackoverflow.com/questions/26323215/do-any-languages-compilers-utilize-the-x86-enter-instruction-with-a-nonzero-ne https://stackoverflow.com/questions/26323215/do-any-language... mention some old compilers using the ENTER instruction with non-zero second argument.
  - tralarpa 3y ago
    I couldn't edit my above post in time because the server was unreachable. What I wanted to add: I think I misunderstood the blog author's question. He wasn't asking about the ENTER statement with a non-zero argument, but about the special ABI with arrays of frame pointers.
AdrianoKF 3y ago
I love how it was already clear that it would be Raymond Chen's blog just from the title and domain name. I really enjoy The Old New Thing and his coverage of obscure details that might otherwise be long forgotten.
- cristeigabriel 3y ago
  Seconded!:)
- reddalo 3y ago
  His blog is the best, especially when he talks about ancient versions of Windows. He talk about obscure details, but they're so interesting because they give you insight on how things were made!
- secondcoming 3y ago
  I’m impressed he remembers the minute details, I can’t recall what I did last month.
  - epcoa 3y ago
    Not sure about Raymond Chen’s memory capabilities but carefully maintained notes are an effective substitute.
  - kazinator 3y ago
    Well, I also can't recall what I did last month, but if you put me in front of an Apple II+, I know I can get into the machine language monitor with "CALL -151".
gpderetta 3y ago
> I suspect it’s silicon on the CPU that is completely wasted. I suspect it is completely microcoded and there is no dedicated silicon to implement it (well other than ROM space to store the microcode I guess).
- clausecker 3y ago
  Right on the money. Almost. Measurements suggest the microcode has special cases for 0 and 1 level of displays and these perform better, though not crazy well.
Joker_vD 3y ago
Ah, so it's for constructing displays. A nice feature to have if you expect e.g. Pascal to be the most popular high-level language used on your ISA, I guess. Probably could also be used to implement flat closures when combined with "Cheney on the M.T.A." allocation scheme?
sim7c00 3y ago
these kinds of details about x64 and x86 make it so hard to build something from intel or amd docs. what do you use, what do you omit, feature and instruction wise. so many ways to do the same things.. it takes a lot of experimentation and measurements to see whats efficient and whats simple to implement etc. you can peek at nix and other osses but i suppose tbat defeats the purpose of the excersize :d. anyway, nice blog all in all =) always happy to see people translate more complex constructs into the basic instructions. helps understand a lot what the cpu is actually doing.
- retrac 3y ago
  As I'm sure most of us know, the x86 lineage goes back nearly 50 years. Many ideas that went nowhere, were taken as design decisions, and are forever etched in silicon. One of my favourite examples is the x86 task switching support. The 386 had everything needed to implement context-switching and multitasking -- in hardware. (That notion was in vogue, once. See the ICL 2900.) If you set up the right structure in memory, and turn on the hardware task switching, x86 processors will automatically context switch all of the registers in and out of that in-memory structure, at the appropriate privilege boundaries (interrupt, return from interrupt, etc.) So your kernel can switch between processes like this (more or less): mov next_proc_ptr, tr iret It also provides IO port protection - there's another table that lets you specify which ports the process can access, which the processor will examine, when an IO instruction is executed in user mode, presumably for something microkernelish. Now that's the good ol' CISC spirit. It's quite a complex feature and yet I'm not sure any OS ever used it.
  - bombcar 3y ago
    Even if no OS used it I bet it has been used - the era of the 386 saw a decent amount of bare-metal specialized uses of the hardware. Some would nominally run on DOS, some didn’t bother with it at all.
  - dist-epoch 3y ago
    > I'm not sure any OS ever used it When nobody uses such a feature is typically either because it has worse performance then alternatives or some missing feature. There have been many such cases in x86.
bell-cot 3y ago
Not a compiler expert, nor chip designer, but... From everything I've seen, the x86 and x86-64 instruction sets are loaded with "it seemed like a good idea at the time", "if you couldn't count on FPU / MMX / SSE instructions being available...", and similar waste-of-silicon instructions. Plus, there's a feedback loop: Compilers writers restrict themselves to subset S of the full instruction set for cost-benefit reasons <==> CPU designers devote minimal time / effort / silicon to improving the performance of non-S instructions.
- mschuster91 3y ago
  > From everything I've seen, the x86 and x86-64 instruction sets are loaded with "it seemed like a good idea at the time", "if you couldn't count on FPU / MMX / SSE instructions being available...", and similar waste-of-silicon instructions. They are, but a lot of "instructions" these days don't even map onto native silicon logic anyway like they used to. It's all microcode these days.
  - bell-cot 3y ago
    True - "wastes of designers' time / room in the instruction set / testing / etc." would have been a far better description.
  - titzer 3y ago
    > It's all microcode these days. I know we're splitting hairs here, but really the only difference between microcoded instructions and "native" instructions is that the instruction decoder directly produces u-ops versus doing a micro code table lookup. The microcode table lookup just ends up producing u-ops, and those all go through the rest of the out-of-order engine just the same. It's really just the frontend that is the difference. For CPUs that have a trace (u-op) cache, I think the results of microcoded instructions can also be stored there.
    - dezgeg 3y ago
      I'd think it matters for the purposes of what behaviour can be changed by microcode updates, no?
HarHarVeryFunny 3y ago
Interesting ... Not sure how many languages other than Pascal support local functions and allow access to (potentially multiple levels of) parent variables in this way. The normal Pascal terminology for this "array of frame pointers" is the "display", and it wouldn't normally be copied during a function/procedure entry - just updated (assuming the function being entered had local functions that accessed parent variables).
- whoopdedo 3y ago
  Only most of the languages you use every day[1]. Pascal inherited nested functions from ALGOL so everything in that family does it. Lisp dialects with lexical scope will have it. You can even get it in C if you use GCC[2]. (And if you don't mind turning off the NX stack.) [1] https://en.wikipedia.org/wiki/Nested_function#Languages https://en.wikipedia.org/wiki/Nested_function#Languages [2] https://www.gnu.org/software/c-intro-and-ref/manual/html_node/Nested-Functions.html https://www.gnu.org/software/c-intro-and-ref/manual/html_nod...
  - HarHarVeryFunny 3y ago
    Well, I'm not actually using ALGOL or Lisp every day, but ok ... :) There's a variety of implementations there though - some seem to be talking about closures, and at least one (Rust) doesn't appear to support parent variable access.
- wmil 3y ago
  JavaScript seems like the obvious one. Isn't this just a lexical closure? It's common in scripting languages. https://wiki.c2.com/?LexicalClosure https://wiki.c2.com/?LexicalClosure
  - HarHarVeryFunny 3y ago
    I don't know how compilers typically implement closures, but that does seem to be what some languages on that list (e.g. D) support by way of "nested/local functions", and is obviously a bit different in terms of variable lifetime. In Pascal local functions are not closures and do not affect variable storage. All variables are still stored in the stack frame (unless optimized to be in registers), and local functions can access their parent's (& grandparent's etc) variables via pointers to their stack frames, which are normally held globally in an array of frame pointers called the display. For a Pascal depth-N nested local function to access it's parent's variables it would use the depth-(N-1) display entry, for it's grandparent's variables the depth-(N-2) entry etc. When a parent calls a nested function the only thing that needs to change in the display is to set the depth N-1 display pointer to it's own stack frame, and to replace it with the old N-1 pointer when the call is complete. The grandparent/etc entries don't need to change. This x86 ENTER instruction appears to be designed for a more general case than Pascal, since it's apparently copying the entire display (array of frame pointers), not just doing the more efficient incremental update/restore that is all Pascal needs. It's interesting to see Raymond Chen writing about this "almost always unused" ENTER parameter since it suggests that compilers are not using it even for those languages that might seem to be able to make use of it. Perhaps, as for Pascal, it's not the most efficient way, or perhaps just because languages such as JavaScript don't normally compile to machine code.
ianopolous 3y ago
For those interested, here is the implementation of enter for JPC emulator: https://github.com/ianopolous/JPC/blob/master/src/org/jpc/emulator/execution/opcodes/pm/enter_o32_Iw_Ib.java#L48 https://github.com/ianopolous/JPC/blob/master/src/org/jpc/em... https://github.com/ianopolous/JPC/blob/master/src/org/jpc/emulator/processor/Processor.java#L3055 https://github.com/ianopolous/JPC/blob/master/src/org/jpc/em...
- cylinder714 3y ago
  https://github.com/ianopolous/JPC https://github.com/ianopolous/JPC "JPC is a fast modern x86 PC emulator capable of booting Windows up to Windows 95 (and windows 98 in safe mode) and some graphical linuxes. It has a full featured graphical debugger with time travel mode along with standard features like break and watch points."
metanonsense 3y ago
Saw the headline and thought "This sounds like Raymond Chen". Saw "microsoft.com", I knew this was Raymond Chen.
- glonq 3y ago
  Raymond is a treasure. Not everybody who can do things can also teach/explain things so well.
  - jart 3y ago
    I second that. I can't count how many times Raymond Chen has helped me understand things while working on Windows polyfills for Cosmopolitan Libc. Microsoft is very fortunate to have him.
- Dwedit 3y ago
  Or you just mouseover and see "oldnewthing" in the URL...
deleted 3y ago
[deleted]
kazinator 3y ago
Interesting. The TXR Lisp VM uses displays, and has a similar concept with keeping track of levels. We can see that with code. We disable optimization so the compiler doesn't eliminate frames: 1> (let ((*opt-level* 0)) (compile-toplevel '(let (a) (let (b) (let (c) (list a b c)))))) #<sys:vm-desc: 9fb6870> 2> (disassemble *1) data: syms: 0: list code: 0: 04020001 frame 2 1 1: 04030001 frame 3 1 2: 04040001 frame 4 1 3: 20030002 gcall t2 0 v00000 v01000 v02000 4: 08000000 5: 10000C00 6: 10000002 end t2 7: 10000002 end t2 8: 10000002 end t2 9: 10000002 end t2 instruction count: 8 #<sys:vm-desc: 9fb6870> The frame 4 1 instruction means we are at depth 4, and there is 1 variable here.
satiric 3y ago
Interesting. To my untrained eye, this looks like its only purpose is to produce spaghetti code. What's a scenario where this is actually desirable? (I.e., where you actually want to be able to access the stack frame of a parent function call)

151 points by matt_d 3y ago

> I wonder if anybody uses the Intel design for nested functions TFA gives at least a partial answer to my earlier q: https://news.ycombinator.com/item?id=38544640 https://news.ycombinator.com/item?id=38544640

tralarpa 3y ago
The answers in this discussion https://stackoverflow.com/questions/26323215/do-any-languages-compilers-utilize-the-x86-enter-instruction-with-a-nonzero-ne https://stackoverflow.com/questions/26323215/do-any-language... mention some old compilers using the ENTER instruction with non-zero second argument.
- tralarpa 3y ago
  I couldn't edit my above post in time because the server was unreachable. What I wanted to add: I think I misunderstood the blog author's question. He wasn't asking about the ENTER statement with a non-zero argument, but about the special ABI with arrays of frame pointers.

AdrianoKF 3y ago

I love how it was already clear that it would be Raymond Chen's blog just from the title and domain name. I really enjoy The Old New Thing and his coverage of obscure details that might otherwise be long forgotten.

cristeigabriel 3y ago
Seconded!:)
reddalo 3y ago
His blog is the best, especially when he talks about ancient versions of Windows. He talk about obscure details, but they're so interesting because they give you insight on how things were made!
secondcoming 3y ago
I’m impressed he remembers the minute details, I can’t recall what I did last month.
- epcoa 3y ago
  Not sure about Raymond Chen’s memory capabilities but carefully maintained notes are an effective substitute.
- kazinator 3y ago
  Well, I also can't recall what I did last month, but if you put me in front of an Apple II+, I know I can get into the machine language monitor with "CALL -151".

gpderetta 3y ago

> I suspect it’s silicon on the CPU that is completely wasted. I suspect it is completely microcoded and there is no dedicated silicon to implement it (well other than ROM space to store the microcode I guess).

clausecker 3y ago
Right on the money. Almost. Measurements suggest the microcode has special cases for 0 and 1 level of displays and these perform better, though not crazy well.

Joker_vD 3y ago

Ah, so it's for constructing displays. A nice feature to have if you expect e.g. Pascal to be the most popular high-level language used on your ISA, I guess. Probably could also be used to implement flat closures when combined with "Cheney on the M.T.A." allocation scheme?

sim7c00 3y ago

these kinds of details about x64 and x86 make it so hard to build something from intel or amd docs. what do you use, what do you omit, feature and instruction wise. so many ways to do the same things.. it takes a lot of experimentation and measurements to see whats efficient and whats simple to implement etc. you can peek at nix and other osses but i suppose tbat defeats the purpose of the excersize :d. anyway, nice blog all in all =) always happy to see people translate more complex constructs into the basic instructions. helps understand a lot what the cpu is actually doing.

retrac 3y ago
As I'm sure most of us know, the x86 lineage goes back nearly 50 years. Many ideas that went nowhere, were taken as design decisions, and are forever etched in silicon. One of my favourite examples is the x86 task switching support. The 386 had everything needed to implement context-switching and multitasking -- in hardware. (That notion was in vogue, once. See the ICL 2900.) If you set up the right structure in memory, and turn on the hardware task switching, x86 processors will automatically context switch all of the registers in and out of that in-memory structure, at the appropriate privilege boundaries (interrupt, return from interrupt, etc.) So your kernel can switch between processes like this (more or less): mov next_proc_ptr, tr iret It also provides IO port protection - there's another table that lets you specify which ports the process can access, which the processor will examine, when an IO instruction is executed in user mode, presumably for something microkernelish. Now that's the good ol' CISC spirit. It's quite a complex feature and yet I'm not sure any OS ever used it.
- bombcar 3y ago
  Even if no OS used it I bet it has been used - the era of the 386 saw a decent amount of bare-metal specialized uses of the hardware. Some would nominally run on DOS, some didn’t bother with it at all.
- dist-epoch 3y ago
  > I'm not sure any OS ever used it When nobody uses such a feature is typically either because it has worse performance then alternatives or some missing feature. There have been many such cases in x86.

bell-cot 3y ago

Not a compiler expert, nor chip designer, but... From everything I've seen, the x86 and x86-64 instruction sets are loaded with "it seemed like a good idea at the time", "if you couldn't count on FPU / MMX / SSE instructions being available...", and similar waste-of-silicon instructions. Plus, there's a feedback loop: Compilers writers restrict themselves to subset S of the full instruction set for cost-benefit reasons <==> CPU designers devote minimal time / effort / silicon to improving the performance of non-S instructions.

mschuster91 3y ago
> From everything I've seen, the x86 and x86-64 instruction sets are loaded with "it seemed like a good idea at the time", "if you couldn't count on FPU / MMX / SSE instructions being available...", and similar waste-of-silicon instructions. They are, but a lot of "instructions" these days don't even map onto native silicon logic anyway like they used to. It's all microcode these days.
- bell-cot 3y ago
  True - "wastes of designers' time / room in the instruction set / testing / etc." would have been a far better description.
- titzer 3y ago
  > It's all microcode these days. I know we're splitting hairs here, but really the only difference between microcoded instructions and "native" instructions is that the instruction decoder directly produces u-ops versus doing a micro code table lookup. The microcode table lookup just ends up producing u-ops, and those all go through the rest of the out-of-order engine just the same. It's really just the frontend that is the difference. For CPUs that have a trace (u-op) cache, I think the results of microcoded instructions can also be stored there.
  - dezgeg 3y ago
    I'd think it matters for the purposes of what behaviour can be changed by microcode updates, no?

HarHarVeryFunny 3y ago

Interesting ... Not sure how many languages other than Pascal support local functions and allow access to (potentially multiple levels of) parent variables in this way. The normal Pascal terminology for this "array of frame pointers" is the "display", and it wouldn't normally be copied during a function/procedure entry - just updated (assuming the function being entered had local functions that accessed parent variables).

whoopdedo 3y ago
Only most of the languages you use every day[1]. Pascal inherited nested functions from ALGOL so everything in that family does it. Lisp dialects with lexical scope will have it. You can even get it in C if you use GCC[2]. (And if you don't mind turning off the NX stack.) [1] https://en.wikipedia.org/wiki/Nested_function#Languages https://en.wikipedia.org/wiki/Nested_function#Languages [2] https://www.gnu.org/software/c-intro-and-ref/manual/html_node/Nested-Functions.html https://www.gnu.org/software/c-intro-and-ref/manual/html_nod...
- HarHarVeryFunny 3y ago
  Well, I'm not actually using ALGOL or Lisp every day, but ok ... :) There's a variety of implementations there though - some seem to be talking about closures, and at least one (Rust) doesn't appear to support parent variable access.
wmil 3y ago
JavaScript seems like the obvious one. Isn't this just a lexical closure? It's common in scripting languages. https://wiki.c2.com/?LexicalClosure https://wiki.c2.com/?LexicalClosure
- HarHarVeryFunny 3y ago
  I don't know how compilers typically implement closures, but that does seem to be what some languages on that list (e.g. D) support by way of "nested/local functions", and is obviously a bit different in terms of variable lifetime. In Pascal local functions are not closures and do not affect variable storage. All variables are still stored in the stack frame (unless optimized to be in registers), and local functions can access their parent's (& grandparent's etc) variables via pointers to their stack frames, which are normally held globally in an array of frame pointers called the display. For a Pascal depth-N nested local function to access it's parent's variables it would use the depth-(N-1) display entry, for it's grandparent's variables the depth-(N-2) entry etc. When a parent calls a nested function the only thing that needs to change in the display is to set the depth N-1 display pointer to it's own stack frame, and to replace it with the old N-1 pointer when the call is complete. The grandparent/etc entries don't need to change. This x86 ENTER instruction appears to be designed for a more general case than Pascal, since it's apparently copying the entire display (array of frame pointers), not just doing the more efficient incremental update/restore that is all Pascal needs. It's interesting to see Raymond Chen writing about this "almost always unused" ENTER parameter since it suggests that compilers are not using it even for those languages that might seem to be able to make use of it. Perhaps, as for Pascal, it's not the most efficient way, or perhaps just because languages such as JavaScript don't normally compile to machine code.

ianopolous 3y ago

For those interested, here is the implementation of enter for JPC emulator: https://github.com/ianopolous/JPC/blob/master/src/org/jpc/emulator/execution/opcodes/pm/enter_o32_Iw_Ib.java#L48 https://github.com/ianopolous/JPC/blob/master/src/org/jpc/em... https://github.com/ianopolous/JPC/blob/master/src/org/jpc/emulator/processor/Processor.java#L3055 https://github.com/ianopolous/JPC/blob/master/src/org/jpc/em...

cylinder714 3y ago
https://github.com/ianopolous/JPC https://github.com/ianopolous/JPC "JPC is a fast modern x86 PC emulator capable of booting Windows up to Windows 95 (and windows 98 in safe mode) and some graphical linuxes. It has a full featured graphical debugger with time travel mode along with standard features like break and watch points."

metanonsense 3y ago

Saw the headline and thought "This sounds like Raymond Chen". Saw "microsoft.com", I knew this was Raymond Chen.

glonq 3y ago
Raymond is a treasure. Not everybody who can do things can also teach/explain things so well.
- jart 3y ago
  I second that. I can't count how many times Raymond Chen has helped me understand things while working on Windows polyfills for Cosmopolitan Libc. Microsoft is very fortunate to have him.
Dwedit 3y ago
Or you just mouseover and see "oldnewthing" in the URL...

deleted 3y ago

[deleted]

kazinator 3y ago

Interesting. The TXR Lisp VM uses displays, and has a similar concept with keeping track of levels. We can see that with code. We disable optimization so the compiler doesn't eliminate frames: 1> (let ((*opt-level* 0)) (compile-toplevel '(let (a) (let (b) (let (c) (list a b c)))))) #<sys:vm-desc: 9fb6870> 2> (disassemble *1) data: syms: 0: list code: 0: 04020001 frame 2 1 1: 04030001 frame 3 1 2: 04040001 frame 4 1 3: 20030002 gcall t2 0 v00000 v01000 v02000 4: 08000000 5: 10000C00 6: 10000002 end t2 7: 10000002 end t2 8: 10000002 end t2 9: 10000002 end t2 instruction count: 8 #<sys:vm-desc: 9fb6870> The frame 4 1 instruction means we are at depth 4, and there is 1 variable here.

satiric 3y ago

Interesting. To my untrained eye, this looks like its only purpose is to produce spaghetti code. What's a scenario where this is actually desirable? (I.e., where you actually want to be able to access the stack frame of a parent function call)