aengelke

7 ms·

31.
by aengelke 1y ago
> if we didn't settle on executing compiled machine code exactly as-is, and had a instruction-updating pass (less involved than a full VM byte code compilation) Apple tried something like this: they collected the LLVM bitcode of app
32.
by aengelke 1y ago
I agree; and the article seems to have also quite a few technical flaws: - Register width: we somewhat maxed out at 512 bits, with Intel going back to 256 bits for non-server CPUs. I don't see larger widths on the horizon (even if SVE
33.
by aengelke 2y ago
I'd add code quality as a reason. I find it much easier to understand and modify code in LLVM compared to GCC. Both have a fairly steep learning curve and not too much documentation, but often I (personally) find LLVM's architectu
34.
by aengelke 2y ago
libgccjit is, despite its name, just another front-end for GIMPLE. The JIT-part is realized through compiling the object file to a shared library and using dlopen on this. One big problem with libgccjit, despite its fairly bad compile-time
35.
by aengelke 2y ago
> attempting to separate them, like what most if not all compilers do, results in (sometimes very) suboptimal results Not separating them would have a big disadvantage: all register allocation decisions need to be strictly local, because
36.
by aengelke 2y ago
There was a paper about this at CGO earlier this year [1]. Correctness is a problem that is hard to solve, though; 50% accuracy might not be enough for serious use cases, especially given that the relation to the original input for manual i
37.
by aengelke 2y ago
Side note: Intel CPUs since Skylake and also recent AMD CPUs (since Zen 3 or so?) store a history for indirect branches. On such processors, using threaded jumps does not really improve performance anymore (I've even seen 1-2% slowdown
38.
by aengelke 2y ago
TZCNT and BSF are not completely identical even for non-zero input: BSF sets the ZF when the input is zero, TZCNT sets ZF when the output is zero (i.e., the least significant bit is set).
39.
by aengelke 2y ago
The behavior/penalty of 66/67 length-changing prefixes is documented in the Intel Optimization Reference Manual (3.4.2.3): > Assembly/Compiler Coding Rule 19. (MH impact, MH generality) Favor generating code using imm8 or
40.
by aengelke 2y ago
> how they interact with the FS/GS prefixes (64/65) For memory operations, they are ignored: 64-2e-65-3e gives 65 as segment override. (From my memory and the resulting implementation, I did some tests with hardware a few years
41.
by aengelke 2y ago
> Other architectures, like [...] ARMv8, are much more consistent. From an instruction/operation perspective, AArch64 is more clean. However, from an instruction operand and encoding perspective, AArch64 is a lot less consistent tha
42.
by aengelke 2y ago
Thanks for the pointer to QEMU's decoder! I actually never looked at it before. So you coded all the tables manually in C -- interesting, that's quite some effort. I opted to autogenerate the tables (and keep them as data only =&g
43.
by aengelke 2y ago
> A lot of this is done in software (microcode). No, that's not the case, since >30 years. Microcode is only used for implementing some complex instructions (mostly system instructions). Most regular instructions (and the rest of
44.
by aengelke 2y ago
Bonus quirk: there's BSF/BSR, for which the Intel SDM states that on zero input, the destination has an undefined value. (AMD documents that the destination is not modified in that case.) And then there's glibc, which happily
45.
by aengelke 2y ago
Nice summary! Additional changes I have planned: - Removing per-instruction timers, which add a measurable overhead even when disabled ( https://github.com/llvm/llvm-project/pull/97046 ) - Splitting AsmPrinterH
46.
by aengelke 3y ago
It works and is easier to read, but there's an important catch: it is not obvious when the compiler falls back to an inefficient scalar implementation that might even be less efficient than not using SIMD at all. Example: https:/
47.
by aengelke 3y ago
Thanks for the blog post, it's always nice to see performance improvements in Postgres! I'm curious: how much time is spent for LLVM on some real queries and how is LLVM configured (i.e., which passes, which back-end optimization,
48.
by aengelke 3y ago
It is an old idea with a new name. For example, QEMU orginally worked like this [1] and they already used relocations to patch in constants, before they later moved to TCG for higher-quality code. [1]: https://www.usenix.org&#x2F
49.
by aengelke 3y ago
The change was withdrawn to improve it for F41 [1]. There were concerns about using $PATH to find executables for v2/v3/v4, especially for containers and absolute path specifications. [1]: https://pagure.io/fesco&#
50.
by aengelke 3y ago
> That way the location of the .text section becomes reachable to the running program via the section header table. Not sure why you would want to find .text in the program, but if you do, the linker (at least ld.bfd and ld.lld do) adds
51.
by aengelke 3y ago
Nice visualization of the ELF headers. However, the article has a few inaccuracies: - ELF files are used not only for executables, but also object files, shared libraries, and also coredumps. Different parts of the ELF format serve differen
52.
by aengelke 3y ago
Dynamic linking doesn't require section headers, either, just a PT_DYNAMIC program header to refer to the .dynamic section (which in turn refers to .dynsym, .dynstr, .rela.dyn, .init_array, stc.).
53.
by aengelke 3y ago
> People think binary analysis is something special, it's not, just compilers. I (conceptually) agree! But there's different sorts of compilers, and they have different requirements, which (should) materialize in the code repre
54.
by aengelke 3y ago
> allow merging them, which QEMU does, even if just at basic block level. IIRC post-SIGSEGV state is not 100% correct in QEMU A quick and very limited testing shows that all registers and even status flags are correct in the ucontext in
55.
by aengelke 3y ago
> [LLVM in QEMU] has already been done, but never upstreamed. Yeah, that's a long history, and it never worked great, the architectures are too different. LLVM is great for compiling functions while QEMU compiles basic blocks, which
56.
by aengelke 3y ago
> The problem with all lifters is though that even a trivial "add rax, 1" generated a lot of IL code (probably 50-100 lines in LLVM IL) Why is this a problem? The addition is one LLVM-IR instruction (add), followed by flag comp
57.
by aengelke 3y ago
> Rolling your own infrastructure is not a great idea IMO For the start, I'd agree, but as soon as you want to do non-trivial things, I don't think that standard compiler IRs (e.g., LLVM) are a good way for binary analysis. (PO
58.
by aengelke 3y ago
Disclosure: I work in one of TUM's database groups. From a researcher's perspective, upstream (Linux) kernel development is not attractive: it's a very time-consuming and lengthy process, often involving several iterations, w
59.
by aengelke 3y ago
Userland exec was a very interesting read when I came across it some years ago; thanks for publishing it! The technique still mostly works, but on recent glibc+Linux, you also have to unregister the rseq area before cleaning out the address