5 ms·

Integrated assembler improvements in LLVM 19

104 points by MaskRay 2y ago

matrix_overload 2y ago
TLDR: building projects with Clang is now about 4% faster due to optimizations in the way it internally handles assembly.
- JonChesterfield 2y ago
  Perhaps more important, someone is going through MC and simplifying it. Decent chance that's a net reduction in bugs as well.
  - MaskRay 2y ago
    Thanks!
- deleted 2y ago
  [deleted]
aengelke 2y ago
Nice summary! Additional changes I have planned: - Removing per-instruction timers, which add a measurable overhead even when disabled (https://github.com/llvm/llvm-project/pull/97046 https://github.com/llvm/llvm-project/pull/97046) - Splitting AsmPrinterHandler (used for unwind info) and DebugHandler (used also for per-instruction location information) to avoid two virtual function calls per instruction (https://github.com/llvm/llvm-project/pull/96785 https://github.com/llvm/llvm-project/pull/96785) - Remove several maps from ELFObjectWriter, including some std::map (changed locally, need to make PR) - Faster section allocation, remove ELF "mergeable section info" hash maps (although this is called just ~40 times per object file, it is very measurable in JIT use cases when compiling many small objects) (planned) - X86 encoding in general; this consumes quite some time and looks very inefficient -- having written my own x86 encoder, I'm confident that there's a lot of improvement potential. (not started) Some takeaways on a higher level -- most of these aren't really surprising, but nonetheless are very frequent problems(/patterns) in the LLVM code base: - Maps/hash maps/sets are quite expensive when used frequently, and sometimes can be easily avoided, e.g., with a vector or, for pointer keys, a pointer dereference - Virtual functions(/abstraction) calls comes at a cost, especially when done frequently - raw_svector_ostream is slow, because writes are virtual function calls and don't get inlined (I previously replaced raw_svector_ostream with a SmallVector&: https://reviews.llvm.org/D145792 https://reviews.llvm.org/D145792) - Frequent heap allocations are costly, especially with glibc's malloc - Many small inefficiencies add up (=> many small improvements do, too)
- MaskRay 2y ago
  Big thanks for the recent performance changes! The "many small inefficiencies" point resonates – it definitely shows how performance is hurt in many small areas. (I aim to write blog posts every 2-3 weeks, but this latest one was postponed... I wrote this in relatively short time so that the gap would not be too long, and I really should take time to refine the post.)
brcmthrowaway 2y ago
[flagged]
- Smaug123 2y ago
  Did you have anything in mind? I must say, "we added LLMs to LLVM" is a scenario that fills me with horror.
  - pjmlp 2y ago
    Actually I expect that eventually we will have such scenario. Instead of LLM => some language generated output => its compiler => executable We will get LLM => magic pixie dust => executable The dream of many corporate overlords.
    - abainbridge 2y ago
      You missed out the input to the LLM, which would presumably be a requirements spec with all behaviour specified in exact detail, including all the tricky corner cases were someone has to think hard about which solution is most useful and least confusing to the customer. Natural language isn't great for expressing such things. A formal notation would be easier. Perhaps something that makes it easy to express if-this-then-that kinds of things. I wonder if a programming language would be good for that.
      - pjmlp 2y ago
        Indeed, that is why, based on offshoring experience, I see a future where the developers of tomorrow are mostly technical architects, with Star Trek style "Computer do XYZ". This has been tried before with UML, see Rational, Together or Enterprise Architect, however LLMs bring an additional automation step to the whole thing.
    - plingbang 2y ago
      Why not just an LLM-based interpreter that direclty executes a PDF spec plus edits received by email? No need to recomplile and restart the app. A DB is also not required - the LLM will naturally remember all user requests and figure out the current state. (We'll solve the limitations of context later)
Keyframe 2y ago
Side note, but I was looking for a pre-built binaries in releases of LLVM project. Specifically I was looking for clang+llvm releases for x86_64 linux (ubuntu preferably) in order to save some time (always had trouble compiling it) and to put it into my own `prefix` directory. It's kind of wild to see aarch64, armv7, powerpc64, x86_64_windows.. but not something like this. I am aware of https://apt.llvm.org/ https://apt.llvm.org/ and its llvm.sh - but as I said, I'd prefer it to live in its own `prefix`. Anyone knows where else there might be pre-builts? There used to be something just like that for v17, like https://github.com/llvm/llvm-project/releases/download/llvmorg-17.0.6/clang+llvm-17.0.6-x86_64-linux-gnu-ubuntu-22.04.tar.xz https://github.com/llvm/llvm-project/releases/download/llvmo...
- quic_bcain 2y ago
  > It's kind of wild to see aarch64, armv7, powerpc64, x86_64_windows.. but not something like this. Yeah, sorry, mostly my fault. I'd been producing these regularly and haven't done as well lately. I'll get one uploaded for 18 soon. :(
  - Keyframe 2y ago
    thank you, friend! you're awesome
    - quic_bcain 2y ago
      Ok, 18.1.8 uploaded [1] [2]. [1] https://github.com/llvm/llvm-project/releases/tag/llvmorg-18.1.8 https://github.com/llvm/llvm-project/releases/tag/llvmorg-18... [2] https://discourse.llvm.org/t/18-1-8-has-been-tagged/79726/10?u=androm3da https://discourse.llvm.org/t/18-1-8-has-been-tagged/79726/10...
      - Keyframe 2y ago
        Fantastic, thank you! Can you share the process how you build and package?
mncharity 2y ago
In the first sentence, "[Intro to the LLVM MC Project]" was likely intended to be a link[1]. [1] https://blog.llvm.org/2010/04/intro-to-llvm-mc-project.html https://blog.llvm.org/2010/04/intro-to-llvm-mc-project.html
- MaskRay 2y ago
  Thx. Fixed

104 points by MaskRay 2y ago

TLDR: building projects with Clang is now about 4% faster due to optimizations in the way it internally handles assembly.

JonChesterfield 2y ago
Perhaps more important, someone is going through MC and simplifying it. Decent chance that's a net reduction in bugs as well.
- MaskRay 2y ago
  Thanks!
deleted 2y ago
[deleted]

aengelke 2y ago

Nice summary! Additional changes I have planned: - Removing per-instruction timers, which add a measurable overhead even when disabled (https://github.com/llvm/llvm-project/pull/97046 https://github.com/llvm/llvm-project/pull/97046) - Splitting AsmPrinterHandler (used for unwind info) and DebugHandler (used also for per-instruction location information) to avoid two virtual function calls per instruction (https://github.com/llvm/llvm-project/pull/96785 https://github.com/llvm/llvm-project/pull/96785) - Remove several maps from ELFObjectWriter, including some std::map (changed locally, need to make PR) - Faster section allocation, remove ELF "mergeable section info" hash maps (although this is called just ~40 times per object file, it is very measurable in JIT use cases when compiling many small objects) (planned) - X86 encoding in general; this consumes quite some time and looks very inefficient -- having written my own x86 encoder, I'm confident that there's a lot of improvement potential. (not started) Some takeaways on a higher level -- most of these aren't really surprising, but nonetheless are very frequent problems(/patterns) in the LLVM code base: - Maps/hash maps/sets are quite expensive when used frequently, and sometimes can be easily avoided, e.g., with a vector or, for pointer keys, a pointer dereference - Virtual functions(/abstraction) calls comes at a cost, especially when done frequently - raw_svector_ostream is slow, because writes are virtual function calls and don't get inlined (I previously replaced raw_svector_ostream with a SmallVector&: https://reviews.llvm.org/D145792 https://reviews.llvm.org/D145792) - Frequent heap allocations are costly, especially with glibc's malloc - Many small inefficiencies add up (=> many small improvements do, too)

MaskRay 2y ago
Big thanks for the recent performance changes! The "many small inefficiencies" point resonates – it definitely shows how performance is hurt in many small areas. (I aim to write blog posts every 2-3 weeks, but this latest one was postponed... I wrote this in relatively short time so that the gap would not be too long, and I really should take time to refine the post.)

brcmthrowaway 2y ago

[flagged]

Smaug123 2y ago
Did you have anything in mind? I must say, "we added LLMs to LLVM" is a scenario that fills me with horror.
- pjmlp 2y ago
  Actually I expect that eventually we will have such scenario. Instead of LLM => some language generated output => its compiler => executable We will get LLM => magic pixie dust => executable The dream of many corporate overlords.
  - abainbridge 2y ago
    You missed out the input to the LLM, which would presumably be a requirements spec with all behaviour specified in exact detail, including all the tricky corner cases were someone has to think hard about which solution is most useful and least confusing to the customer. Natural language isn't great for expressing such things. A formal notation would be easier. Perhaps something that makes it easy to express if-this-then-that kinds of things. I wonder if a programming language would be good for that.
    - pjmlp 2y ago
      Indeed, that is why, based on offshoring experience, I see a future where the developers of tomorrow are mostly technical architects, with Star Trek style "Computer do XYZ". This has been tried before with UML, see Rational, Together or Enterprise Architect, however LLMs bring an additional automation step to the whole thing.
  - plingbang 2y ago
    Why not just an LLM-based interpreter that direclty executes a PDF spec plus edits received by email? No need to recomplile and restart the app. A DB is also not required - the LLM will naturally remember all user requests and figure out the current state. (We'll solve the limitations of context later)

Keyframe 2y ago

Side note, but I was looking for a pre-built binaries in releases of LLVM project. Specifically I was looking for clang+llvm releases for x86_64 linux (ubuntu preferably) in order to save some time (always had trouble compiling it) and to put it into my own `prefix` directory. It's kind of wild to see aarch64, armv7, powerpc64, x86_64_windows.. but not something like this. I am aware of https://apt.llvm.org/ https://apt.llvm.org/ and its llvm.sh - but as I said, I'd prefer it to live in its own `prefix`. Anyone knows where else there might be pre-builts? There used to be something just like that for v17, like https://github.com/llvm/llvm-project/releases/download/llvmorg-17.0.6/clang+llvm-17.0.6-x86_64-linux-gnu-ubuntu-22.04.tar.xz https://github.com/llvm/llvm-project/releases/download/llvmo...

quic_bcain 2y ago
> It's kind of wild to see aarch64, armv7, powerpc64, x86_64_windows.. but not something like this. Yeah, sorry, mostly my fault. I'd been producing these regularly and haven't done as well lately. I'll get one uploaded for 18 soon. :(
- Keyframe 2y ago
  thank you, friend! you're awesome
  - quic_bcain 2y ago
    Ok, 18.1.8 uploaded [1] [2]. [1] https://github.com/llvm/llvm-project/releases/tag/llvmorg-18.1.8 https://github.com/llvm/llvm-project/releases/tag/llvmorg-18... [2] https://discourse.llvm.org/t/18-1-8-has-been-tagged/79726/10?u=androm3da https://discourse.llvm.org/t/18-1-8-has-been-tagged/79726/10...
    - Keyframe 2y ago
      Fantastic, thank you! Can you share the process how you build and package?

mncharity 2y ago

In the first sentence, "[Intro to the LLVM MC Project]" was likely intended to be a link[1]. [1] https://blog.llvm.org/2010/04/intro-to-llvm-mc-project.html https://blog.llvm.org/2010/04/intro-to-llvm-mc-project.html

MaskRay 2y ago
Thx. Fixed