5 ms·

Link to page explaining LLD http://lld.llvm.org/NewLLD.html http://lld.llvm.org/NewLLD.html

by ktta 10y ago

jey 10y ago
EDIT: Disregard; See https://news.ycombinator.com/item?id=13671078 https://news.ycombinator.com/item?id=13671078 -- Actually, the page you linked seems to be about an older ELF and COFF specific implementation, despite its URL. Here's an overview of what makes the new one special: https://lld.llvm.org/design.html https://lld.llvm.org/design.html
- Joky 10y ago
  Actually no: `lld` is not a single linker, there has been a split between Elf/COFF folks and the MachO ones. The page you're linking is about the MachO project which is not very actively developed.
  - kibwen 10y ago
    Does this imply that LLD isn't going to support MacOS any time soon? That would be quite sad.
    - AndyKelley 10y ago
      I don't understand why it would be that way. Is there some inherent property of macho object files that make them outside the scope of LLD?
      - chisophugis 10y ago
        No, it is just that nobody is currently working on it. Last I talked with the Apple folks they are just busy with other stuff. Patches are definitely welcome for MachO improvements in LLD (as always in LLVM!). You should be aware though that the Apple folks feel strongly that the original "atom" linker design is the one that they want to use. If you want to start a MachO linker based on the COFF/ELF design (which has been very successful) you will want to ping the llvm-dev mailing list first to talk with the Apple folks (CC Lang Hames and Jim Grosbach).
- haberman 10y ago
  I am pretty familiar with object file formats, and I don't get how this "atom" stuff works. The site says that "atoms" are an improvement over simple section interlacing. But I don't get how you are going to make this leap without changing the object file format. Linkers work on the section level because that is how object files work. Object files have sections, not atoms. Compilers emit sections as their basic, atomic unit of output. Within a section, the code will assume that all offsets referring to other parts of the section will be stable, so you can't chop a section apart without breaking the code. How does the new linker work in terms of this new "atom" abstraction without changing the underlying object file format?
  - Someone 10y ago
    http://releases.llvm.org/3.9.0/tools/lld/docs/Readers.html http://releases.llvm.org/3.9.0/tools/lld/docs/Readers.html Seems to indicate that many object file formats contain the information you need to split them up in such a fine-grained manner. But maybe, they require you to create special versions of object files where even references internal to each library are referenced there as if they live in a different object file? Is that even possible? Edit: corrections welcome, but https://github.com/llvm-mirror/lld/tree/master/lib/ReaderWriter https://github.com/llvm-mirror/lld/tree/master/lib/ReaderWri... seems to indicate that this only sopports Mach-o (and a yaml-based format used for debugging and testing)
    - chisophugis 10y ago
      > But maybe, they require you to create special versions of object files where even references internal to each library are referenced there as if they live in a different object file? Is that even possible? The extra information that is needed for an ELF linker (any ELF linker; nothing LLD specific) to operate on functions and global data objects in a fine-grained manner is enabled by -ffunction-sections/-fdata-sections. For more information, see https://news.ycombinator.com/item?id=13673589 https://news.ycombinator.com/item?id=13673589
  - Joky 10y ago
    If you are familiar with object file formats in general, you may know that this is exactly how MachO works: it is based on atoms. If you want to map ELF to the atom model, you need somehow to build with -ffunction-section so that the compiler emits a single function per section (and similarly with -fdata-section) or model it by mapping one section of the object to an atom.
haberman 10y ago
Wow, this is claiming 1.2x to 2x faster than gold at 10% of the code size. That is a remarkable achievement. Gold was already faster than GNU ld. I am really surprised that there was this much room for optimization. Ian Lance Taylor who wrote Gold is a really smart guy, and speed was one of the primary goals.
- jblow 10y ago
  Linkers are mind-bogglingly slow. I don't understand why they are so slow. lld is still slow, it is just less slow than the other linkers. This is not to disparage anyone working on linkers or say they are not smart. I think they just don't tend to be performance-oriented programmers, and culturally there has become some kind of ingrained acceptance of how much time it is okay for a linker to take.
  - comex 10y ago
    Linking is also very easy to do incrementally, but for some reason incremental linking is not popular in the Unix world. GNU ld and Apple ld64 can't do it. GNU gold can do it, but only if you pass a special linker flag, which typical build systems don't. LLD can't do it, despite being the spiffy new thing. So you end up with big projects where most of the time taken by incremental debug builds is spent linking - relinking the same object files to each other over and over and over. Awful. I don't use Windows, but I hear Visual Studio does the right thing and links debug builds incrementally by default. Wish the rest of the world would catch on.
    - Joky 10y ago
      ld64 links clang (a multi-millions line project) in roughly 2s on my laptop. Do we need incremental linking? MSVC incremental link is really not a model I would take as an example: the final binary is not the same as the one you get from a clean build, which is not a world I would want to live in.
      - comex 10y ago

Link to page explaining LLD http://lld.llvm.org/NewLLD.html http://lld.llvm.org/NewLLD.html

by ktta 10y ago

Link to page explaining LLD http://lld.llvm.org/NewLLD.html http://lld.llvm.org/NewLLD.html

jey 10y ago

EDIT: Disregard; See https://news.ycombinator.com/item?id=13671078 https://news.ycombinator.com/item?id=13671078 -- Actually, the page you linked seems to be about an older ELF and COFF specific implementation, despite its URL. Here's an overview of what makes the new one special: https://lld.llvm.org/design.html https://lld.llvm.org/design.html

Joky 10y ago
Actually no: `lld` is not a single linker, there has been a split between Elf/COFF folks and the MachO ones. The page you're linking is about the MachO project which is not very actively developed.
- kibwen 10y ago
  Does this imply that LLD isn't going to support MacOS any time soon? That would be quite sad.
  - AndyKelley 10y ago
    I don't understand why it would be that way. Is there some inherent property of macho object files that make them outside the scope of LLD?
    - chisophugis 10y ago
      No, it is just that nobody is currently working on it. Last I talked with the Apple folks they are just busy with other stuff. Patches are definitely welcome for MachO improvements in LLD (as always in LLVM!). You should be aware though that the Apple folks feel strongly that the original "atom" linker design is the one that they want to use. If you want to start a MachO linker based on the COFF/ELF design (which has been very successful) you will want to ping the llvm-dev mailing list first to talk with the Apple folks (CC Lang Hames and Jim Grosbach).
haberman 10y ago
I am pretty familiar with object file formats, and I don't get how this "atom" stuff works. The site says that "atoms" are an improvement over simple section interlacing. But I don't get how you are going to make this leap without changing the object file format. Linkers work on the section level because that is how object files work. Object files have sections, not atoms. Compilers emit sections as their basic, atomic unit of output. Within a section, the code will assume that all offsets referring to other parts of the section will be stable, so you can't chop a section apart without breaking the code. How does the new linker work in terms of this new "atom" abstraction without changing the underlying object file format?
- Someone 10y ago
  http://releases.llvm.org/3.9.0/tools/lld/docs/Readers.html http://releases.llvm.org/3.9.0/tools/lld/docs/Readers.html Seems to indicate that many object file formats contain the information you need to split them up in such a fine-grained manner. But maybe, they require you to create special versions of object files where even references internal to each library are referenced there as if they live in a different object file? Is that even possible? Edit: corrections welcome, but https://github.com/llvm-mirror/lld/tree/master/lib/ReaderWriter https://github.com/llvm-mirror/lld/tree/master/lib/ReaderWri... seems to indicate that this only sopports Mach-o (and a yaml-based format used for debugging and testing)
  - chisophugis 10y ago
    > But maybe, they require you to create special versions of object files where even references internal to each library are referenced there as if they live in a different object file? Is that even possible? The extra information that is needed for an ELF linker (any ELF linker; nothing LLD specific) to operate on functions and global data objects in a fine-grained manner is enabled by -ffunction-sections/-fdata-sections. For more information, see https://news.ycombinator.com/item?id=13673589 https://news.ycombinator.com/item?id=13673589
- Joky 10y ago
  If you are familiar with object file formats in general, you may know that this is exactly how MachO works: it is based on atoms. If you want to map ELF to the atom model, you need somehow to build with -ffunction-section so that the compiler emits a single function per section (and similarly with -fdata-section) or model it by mapping one section of the object to an atom.

haberman 10y ago

Wow, this is claiming 1.2x to 2x faster than gold at 10% of the code size. That is a remarkable achievement. Gold was already faster than GNU ld. I am really surprised that there was this much room for optimization. Ian Lance Taylor who wrote Gold is a really smart guy, and speed was one of the primary goals.

jblow 10y ago
Linkers are mind-bogglingly slow. I don't understand why they are so slow. lld is still slow, it is just less slow than the other linkers. This is not to disparage anyone working on linkers or say they are not smart. I think they just don't tend to be performance-oriented programmers, and culturally there has become some kind of ingrained acceptance of how much time it is okay for a linker to take.
- comex 10y ago
  Linking is also very easy to do incrementally, but for some reason incremental linking is not popular in the Unix world. GNU ld and Apple ld64 can't do it. GNU gold can do it, but only if you pass a special linker flag, which typical build systems don't. LLD can't do it, despite being the spiffy new thing. So you end up with big projects where most of the time taken by incremental debug builds is spent linking - relinking the same object files to each other over and over and over. Awful. I don't use Windows, but I hear Visual Studio does the right thing and links debug builds incrementally by default. Wish the rest of the world would catch on.
  - Joky 10y ago
    ld64 links clang (a multi-millions line project) in roughly 2s on my laptop. Do we need incremental linking? MSVC incremental link is really not a model I would take as an example: the final binary is not the same as the one you get from a clean build, which is not a world I would want to live in.
    - comex 10y ago