4 ms·

> Lineiform is just a wrapper function that takes a closure as input, and returns a closure as output. In the middle however, it will dynamically inline calls t

by disas 5y ago

> Lineiform is just a wrapper function that takes a closure as input, and returns a closure as output. In the middle however, it will dynamically inline calls to other closures that the input might be doing, using dynamic recompilation to lift, optimize, and then lower x86 instructions to a set of x86 instructions with the same operational semantics, but hopefully doing less work such as inlining small closure snippets so there isn’t call/ret and ABI overhead, or folding constants like 3+5. Isn't a magic step missing here? In order to "lift, optimize, and then lower x86 instructions" you need x86 instructions to start with. How does the leap from having a Rust closure object to a snippet of baseline machine code work?

chc4 5y ago
(I'm the author) Rust closures store a function pointer for the closure body. I'm using yaxpeax, which is a fast x86 disassembler made by one of my friends, in order to turn the function pointer into an x86 instruction stream. There's some junk that we have to do with parsing /proc/self/exe in order to get the symbol it points to so we know how big the function is, as well. It's kinda complex, because Rust closure internals aren't documented all that well, but it works fine and isn't UB (as far as I know). The bigger issue is actually that Rust closures use the Rust ABI, which is not stable, and so Lineiform shouldn't really assume which arguments are in which registers since it could (technically) change at any point: we end up having the disassembly entry point be a C ABI trampoline that calls the rust-call ABI so that rustc emits the cross-ABI register shuffling for us as a prelude, and emit a C ABI function to call as an output. The relevant code is https://github.com/chc4/lineiform/blob/4a104c994561282318179ec50a3a6b00b3f6e8c4/lineiform/src/lib.rs#L103 https://github.com/chc4/lineiform/blob/4a104c994561282318179.... It is very messy, sorry.
- disas 5y ago
  I see, thanks for the explanation. That's a neat trick!

by disas 5y ago

chc4 5y ago

(I'm the author) Rust closures store a function pointer for the closure body. I'm using yaxpeax, which is a fast x86 disassembler made by one of my friends, in order to turn the function pointer into an x86 instruction stream. There's some junk that we have to do with parsing /proc/self/exe in order to get the symbol it points to so we know how big the function is, as well. It's kinda complex, because Rust closure internals aren't documented all that well, but it works fine and isn't UB (as far as I know). The bigger issue is actually that Rust closures use the Rust ABI, which is not stable, and so Lineiform shouldn't really assume which arguments are in which registers since it could (technically) change at any point: we end up having the disassembly entry point be a C ABI trampoline that calls the rust-call ABI so that rustc emits the cross-ABI register shuffling for us as a prelude, and emit a C ABI function to call as an output. The relevant code is https://github.com/chc4/lineiform/blob/4a104c994561282318179ec50a3a6b00b3f6e8c4/lineiform/src/lib.rs#L103 https://github.com/chc4/lineiform/blob/4a104c994561282318179.... It is very messy, sorry.

disas 5y ago
I see, thanks for the explanation. That's a neat trick!