12 ms·
LLD is included in the upcoming LLVM 4.0 release
- smlacy 10y agolld is the LLVM linker. More details here: https://lld.llvm.org/ https://lld.llvm.org/
- AndyKelley 10y agoPosting this news article is my way to get the attention of Debian, Arch, Gentoo, Homebrew, NixOS, and other package maintainers. As an upstream compiler developer [1], I want to depend on LLD instead of the system linker, and the sooner LLD becomes ubiquitous in the various package managers, the sooner I can depend on it. (Of course I could ask my users to compile it from source, but that increases overhead of people getting started.) Further evidence that it is time for LLD to be distributed along with LLVM: http://lists.llvm.org/pipermail/llvm-dev/2017-February/110302.html http://lists.llvm.org/pipermail/llvm-dev/2017-February/11030... [1]: http://ziglang.org/ http://ziglang.org/
- woodrowbarlow 10y agocan you elaborate on the advantages offered by using LLD over the system linker? is it purely for the sake of consistency, or does this decrease the size of your compiler's frontend component?
- AndyKelley 10y agoTo me, the biggest advantage is cross compiling. I started working on a proof of concept operating system, and according to the OS Dev Wiki, the first thing you have to do to get started is to build a cross compiler. This is because by default the C compiler and linker are compiled only for the current system. When I started making the OS in Zig, however, I did not have to build a special cross compilation version of Zig, because it supports cross compilation out of the box for every target and OS. For example, unlike in C, compiler_rt/libgcc are shipped with the compiler in source form and built lazily for the desired target. So not having to build or obtain a special gcc was nice, however, I did have to build a cross compiled GNU binutils so that I could use an ARM linker on an x86_64 host. If I can depend on LDD, then I can support cross compiling for every target with no other dependencies needed.
- qznc 10y agoThis "first build a cross compiler" is ingrained for OS/embedded people. It is a shame, because it is just a GCC weirdness.
- pjmlp 10y agoYes, I still remember the days where each compiler had its own linker, optimized for the specific language semantics. Using the system linker means throwing away those benefits, in name of lowest common denominator.
- floatboth 10y agoGo (well, its main compiler, not gccgo) is one of the few languages that does everything by itself. It's awesome. I can just tell it to compile to FreeBSD/arm on a Darwin/amd64 box and it will just fucking do it!
- pjmlp 10y agoSystem linkers are a UNIX thing. On the majority of other OSes, each programming language used to have its own linker. This is why although I dislike some of Go's decisions, I am quite supportive of its uptake. - Bostraped language, with very few dependencies on C - Runtime library makes direct use of syscalls, instead of taking the easy way and depend on libc - It has its own linker - Despite the GC hate and the whole discussion what is "systems programming", people are indeed slowly using for such purposes
- binarycrusader 10y agoTo me, the biggest advantage is cross compiling Not all system linkers have this problem. For example, Solaris ld(1) is perfectly capable of cross-linking any valid ELF file.
- saurik 10y agoI spend essentially all of my time cross compiling stuff (as in, 99% of my development is cross compilation; I have two projects with a local target, on earth of which is never used on that target an is one of which is seldom used on that target: almost all of my users are using my software on non-native targets from any of my development systems). I have one project in particular which targets 20 different operating system / platform combinations as part of one build process. I can tell you that even when I am using clang and LLVM to target another system, except for minor variations where I am targeting the same operating system and the same vendor with the same target executable format across only different CPU vendors (i.e.: Apple), I still have to end up with different full cross-compiler tool chains, often based on entirely different forks of the code base for LLVM. Even if it were possible, and it is not clear to me that it is or that it would even be desirable (as you still need a set of highly platform-specific libraries and header files, even for weirdly quirked-up core libraries like the moral equivalents of libgcc that are needed for every subplatform, due to symbol visibility), in practice we just aren't there yet, as there are too many extremely subtle platform differences... the only way we could even get close to this dream would really be to have a mindset that a platform is a massive file of configuration warts that you pass to the compiler, as opposed to some trivial flags like -mllvm -arm-reserve-r9.
- MaulingMonkey 10y ago> can you elaborate on the advantages offered by using LLD over the system linker? Performance. >> (usually more than 2x faster than GNU gold). I switched from bfd to gold to realize a ~6x performance gain on a project (~1 minute linking -> ~10 seconds linking, times 10-20 subprojects, times a few build configurations - really adds up!) I'll keep in mind that I now also have the option to switch to LLD for another 2x (12x total?)
- pjmlp 10y agoIn addition to that, I don't know anything about LLD, but when a compiler toolchain for language X also has its own linker, it means the linker can take advantage of the language semantics, which aren't necessarily properly supported in a system linker. As an example, many of the C++'s drawbacks regarding build systems was because Bjarne wanted that the generated object files and libraries could be transparently linked by a linker that only understood C.
- oso2k 10y agoWhat makes LLD faster than gold? gold's improvement on gnu ld was focusing on ELF output and removing BFD internal representation. And shouldn't the name be llld?
- X86BSD 10y agoThe amount of work it took to replace the gnu linker with LLD in freebsd was non trivial. But it's just about ready to be turned on in base.
- rleigh 10y agoI'm really looking forward to this. I've hit the 2^16 symbol limit in the ancient GNU ld in base several times over the last few years (the version in ports is fine). It will be great to have a default linker without such 16-bit limits.
- josteink 10y agoGiven his current attitudes on LLVM, I'm sure Stallman will se exceeding such a limit as an attack on free software ;) Seriously though. I guess the majority of major FOSS software out there have a goal of retaining GNU compatibility, so I'd be surprised if this turned out to be an issue.
- kibwen 10y agoAs someone mildly informed of Rust language development, I can confirm that they as well would be over the moon if LLD were usable on all LLVM targets. It's been a long, long anticipated feature.
- umanwizard 10y agoPackage maintainers aren't like the President. You don't need news media to get their attention. You can just e-mail them.
- AndyKelley 10y agoThat sounds like an O(N) solution. More seriously, being a package maintainer myself I know that the goal is to serve the users and create packages that users expect. So, the best way to get package maintainers to include LLD is to get users to expect it.
- pantalaimon 10y agoWell they kind of are, it's up to them to decide what kind of software millions of users are able to install.
- ktta 10y agoLink to page explaining LLD http://lld.llvm.org/NewLLD.html http://lld.llvm.org/NewLLD.html
- jey 10y agoEDIT: Disregard; See https://news.ycombinator.com/item?id=13671078 https://news.ycombinator.com/item?id=13671078 -- Actually, the page you linked seems to be about an older ELF and COFF specific implementation, despite its URL. Here's an overview of what makes the new one special: https://lld.llvm.org/design.html https://lld.llvm.org/design.html
- Joky 10y agoActually no: `lld` is not a single linker, there has been a split between Elf/COFF folks and the MachO ones. The page you're linking is about the MachO project which is not very actively developed.
- kibwen 10y agoDoes this imply that LLD isn't going to support MacOS any time soon? That would be quite sad.
- AndyKelley 10y agoI don't understand why it would be that way. Is there some inherent property of macho object files that make them outside the scope of LLD?
- chisophugis 10y agoNo, it is just that nobody is currently working on it. Last I talked with the Apple folks they are just busy with other stuff. Patches are definitely welcome for MachO improvements in LLD (as always in LLVM!). You should be aware though that the Apple folks feel strongly that the original "atom" linker design is the one that they want to use. If you want to start a MachO linker based on the COFF/ELF design (which has been very successful) you will want to ping the llvm-dev mailing list first to talk with the Apple folks (CC Lang Hames and Jim Grosbach).
- alvarop 10y ago"The ELF support is in progress and is able to link large programs such as Clang or LLD itself. Unless your program depends on linker scripts, you can expect it to be linkable with LLD." :( The one feature I need most for embedded development
- rui314 10y agoThe document is outdated and needs updating. Linker script support has improved a lot since 3.9, and that's why it can now link the entire FreeBSD base system including kernel. It's still in active development (e.g. it can't link Linux kernel yet) but it's not that bad now.
- alvarop 10y agoNice! I mostly want to use it on Cortex-M0/3/4 microcontrollers, which are much less complex.
- nightcracker 10y agoThis can not come too soon. Looking forward to it.
- nneonneo 10y agoThe LLD webpage touts a big speed improvement over gold, which is extremely commendable. What I'm curious about, though, is memory usage. On relatively constrained systems, linking large projects can take ages due to swapping - anecdotally, linking is responsible for upwards of 80% of the compilation times I see for a certain very large software project on my developer machine (which, at 16GB RAM, isn't huge, but fairly typical). Worse, memory pressure from linking makes the process rather non-parallelizable, which also hurts throughput. Having a memory-efficient linker could significantly speed up compilation beyond just a 2x improvement in such environments.
- rui314 10y agoI shouldn't be bad, although I didn't actually measure its heap usage. Allocating memory and copying data is slow, so in LLD we mmap input object files and use the files directly as much as possible. This increases process virtual memory size (because large files are kept mapped to memory), but the actual memory usage is much smaller than that.
- wyldfire 10y agoOf note is the fact that George Rimar, Dmitry Golovin et al are working on getting Linux to boot linked by LLD. They've making a lot of progress -- when he started it didn't even successfully link.
- ThisIs_MyName 10y agoWoohoo, I've been using lld instead of GNU ld and gold for a while now and I'd love to have everyone who builds my code do the same.
- fithisux 10y agoEven though I don't care about usable Linux linker I care about a replacement for Microsoft's linker for windows. LDC is an example where you need the closed link.exe in order to link. Hopefully all the llvm projects (lldb, libc++, ...) mature for windows in order to have a replacement for msvc's tools. Until then (and beyond then) GNU tools gcc + ld will be my go to tools if msvc compatibility is not a constraint. They work perfectly well with Golang. But it happens people to release msvc link libraries. Even dmd for windows-x64 needs microsoft linker to work). Unilink is an option, but its closed source. And openwatcom-v2 will take time to get there.
- AceJohnny2 10y agoThis is not the "atom-based" LLD http://lld.llvm.org/AtomLLD.html http://lld.llvm.org/AtomLLD.html While conventional linkers work at the compilation-unit level (one source file, usually), placing that whole source file's functions adjacently in memory [1], an atom-based linker is able to take the smallest linkable units (individual functions, each static/global variable...), and arrange those optimally. As I recall, the OS X ld is based on this model. However it remains more limited as it doesn't support GNU ld's linker scripts and only limited command-line parameters, so it's not exposing all the power of the flexibility it would provide. As far as I know, AtomLLD remains an experimental project with only 1 or two people working part-time on it. [1] although modern linkers also add LTO (Link-Time Optimization) to rearrange things after everything's been integrated/
- chisophugis 10y ago> While conventional linkers work at the compilation-unit level (one source file, usually), placing that whole source file's functions adjacently in memory [1], an atom-based linker is able to take the smallest linkable units (individual functions, each static/global variable...), and arrange those optimally. This isn't quite right. It's just that traditionally (i.e. when not using -ffunction-sections/-fdata-sections) when compilers output an ELF relocatable object, they group all the functions into a single "smallest linkable unit" (a single .text section) and so the linker can't actually do any reordering because the information that the functions are distinct has been lost. I describe this in more detail here: http://lists.llvm.org/pipermail/llvm-dev/2016-December/108140.html http://lists.llvm.org/pipermail/llvm-dev/2016-December/10814... This was actually one of the key technical issues that prevented the Atom design from being used for ELF and COFF. """ the ELF and COFF notion of "section" is a strict superset of its [the Atom LLD's] core "atom" abstraction (an indivisible chunk of data with a 1:1 correspondence with a symbol name). Therefore the entire design was impossible to use for those formats. In ELF and COFF, a section is decoupled from symbols, and there can be arbitrarily many symbols pointing to arbitrary parts of a section (whereas in MachO, "section" means something different; the analogous concept to an ELF/COFF section is basically "the thing that you get when you split a MachO section on symbol boundaries, assuming that the compiler/assembler hasn't relaxed relocations across those boundaries" which is not as powerful as ELF/COFF's notion). This resulted in severe contortions that ultimately made it untenable to keep working in that codebase. """
- mioelnir 10y agoI'm just going to shamelessly quote the FreeBSD Quarterly Status Report, Q4/2016 that came out a few days ago. LLD developers made significant progress over the last quarter. With changes committed to both LLD and FreeBSD we reached a major milestone: it is now possible to link the entire FreeBSD/amd64 base system (kernel and userland) with LLD. Now that the base system links with LLD, we have started investigating linking applications in the ports tree with LLD. Through this process we are identifying limitations or bugs in both LLD and a number of FreeBSD ports. With a few work-in-progress patches we can link approximately 95% of the ports collection with LLD on amd64. That is 95% of ~27000 ports. Which means for very many things, lld as linker will just work once those patches have been applied at the appropriate places. [1]: https://lists.freebsd.org/pipermail/freebsd-announce/2017-February/001781.html https://lists.freebsd.org/pipermail/freebsd-announce/2017-Fe...
- deleted 10y ago[deleted]
- gigatexal 10y agoFor the uninitiated what is LLD? What does it provide/ are it's benefits?