4 ms·

Ive always struggled to understand the need to have linker Like, you could easily write your compiler to do not have to rely on such machinery Meanwhile they

Ive always struggled to understand the need to have linker Like, you could easily write your compiler to do not have to rely on such machinery Meanwhile they add complexity and decrease quality of error messages (in cpp)

mschuster91 2y ago
> Like, you could easily write your compiler to do not have to rely on such machinery You need a linker as soon as you are dealing with either multiple languages in one project (say, C++ and ASM) or if you include other libraries.
Joker_vD 2y ago
Separate compilation. Of course, if your compiler is fast enough to rebuild the whole universe in 6 seconds and then rest on the seventh — an approach Wirth advocated in one of his papers about an implementation of Pascal system — you won't need a linker. But most compilers are not that fast. Besides, there is more than one programming language, so that's something we have to deal with somehow. And to be fair, merging modules in the compiler, as you go by, while not that difficult, is just annoying. If you link them properly together, into big amalgamated text/rodata/data sections, then you need to apply relocations (and have them in the first place). If you just place them next to each other, then you have to organize the inter-module calls via some moral equivalent of GOT/PLT. In any case, all this logic really doesn't have much to do with code generation proper, it's administrativia — and logic for dealing with has already been written for you and packed in the so called "link editor".
- uptownfunk 2y ago
  What are the bottlenecks that make this so slow
ChadNauseam 2y ago
When I first came to C++ from Rust I was surprised by the regularity of linker errors. Rust must be compiled with a linker as well but I don't think I've ever seen a linker error, except when doing exotic things far outside of my typical day-to-day. I guess rustc detects the situations in which the linker would throw an error and then throws its own error preemptively. It leads to a much better user experience than the C++ one, since the error messages produced by the linker are always unnecessarily terrible
- Joker_vD 2y ago
  > I guess rustc detects the situations in which the linker would throw an error and then throws its own error preemptively. Pretty much. The crucial difference between C and Rust which enables Rust to do this sort of detection is that in Rust, the extern things are anchored in modules (crates? whatever), and so when you import things, you have to say where you are importing them from. extern void *magic_init(int); extern void *magic_stuff(void*, const char*, int); extern void magic_fini(void*); versus use crate_of_magic::{init, stuff, fini}; It even enables one to actually type-check against the imported crate during the compilation (IIRC if Cargo can't locate the imported crate to look into it, it will refuse to build the project), as opposed to hoping that the headers you've included are correctly describing the object file you'll be linking against.
- 0x457 2y ago
  Only time I get linker errors in rust is when it's linking some dynamic library written in C.
wyldfire 2y ago
All but the most trivial programs require a linker to resolve references among object files. And while "int main() {}" might seem like a trivial C program, it's not (by that definition, at least). Your favorite toolchain will often include archives and objects that you might take for granted like crt0.o, init.o, fini.o, libgcc/clang_rt.builtins and more. The compiler's design is simplified by not having to resolve references among symbols. The assembler can do this for references within a section and linkers can do it among sections. Linkers might have to add trampolines/thunks for relocations that span a distance longer than the opcode could reach. Loaders do this symbol resolution and relocation at runtime.
- high_na_euv 2y ago
  So, if hello world is not trivial Then you can easily generate e.g web assembly from your own lang input without linker and it will run fine Aint the same viable for x86?
  - wyldfire 2y ago
    > So, if hello world is not trivial Note that the nontrivial C program I showed above was not "hello world". Emitting output is yet more complicated still. > you can easily generate e.g web assembly from your own lang input without linker IIRC some things that generate wasm generate object files (e.g. C translation units). Those wasm objects require a linker to resolve symbol references to create the executable. > it will run fine ...on a wasm virtual machine. IIRC Java doesn't have anything like a traditional linker either. > Aint the same viable for x86? I mean, you can do anything you want inventing a new compiler that does much more than traditional compilers. But consider perhaps that there's a lot of value in the status quo that you might appreciate more if you were more familiar with the design of a toolchain. It's not that linkers are an absolute drop-dead necessity, it's that having one allows you to have distinct assembler and compiler functions that are much simpler to design by focusing on one kind of transformation.

mschuster91 2y ago

> Like, you could easily write your compiler to do not have to rely on such machinery You need a linker as soon as you are dealing with either multiple languages in one project (say, C++ and ASM) or if you include other libraries.

Joker_vD 2y ago

Separate compilation. Of course, if your compiler is fast enough to rebuild the whole universe in 6 seconds and then rest on the seventh — an approach Wirth advocated in one of his papers about an implementation of Pascal system — you won't need a linker. But most compilers are not that fast. Besides, there is more than one programming language, so that's something we have to deal with somehow. And to be fair, merging modules in the compiler, as you go by, while not that difficult, is just annoying. If you link them properly together, into big amalgamated text/rodata/data sections, then you need to apply relocations (and have them in the first place). If you just place them next to each other, then you have to organize the inter-module calls via some moral equivalent of GOT/PLT. In any case, all this logic really doesn't have much to do with code generation proper, it's administrativia — and logic for dealing with has already been written for you and packed in the so called "link editor".

uptownfunk 2y ago
What are the bottlenecks that make this so slow

ChadNauseam 2y ago

When I first came to C++ from Rust I was surprised by the regularity of linker errors. Rust must be compiled with a linker as well but I don't think I've ever seen a linker error, except when doing exotic things far outside of my typical day-to-day. I guess rustc detects the situations in which the linker would throw an error and then throws its own error preemptively. It leads to a much better user experience than the C++ one, since the error messages produced by the linker are always unnecessarily terrible

Joker_vD 2y ago
> I guess rustc detects the situations in which the linker would throw an error and then throws its own error preemptively. Pretty much. The crucial difference between C and Rust which enables Rust to do this sort of detection is that in Rust, the extern things are anchored in modules (crates? whatever), and so when you import things, you have to say where you are importing them from. extern void *magic_init(int); extern void *magic_stuff(void*, const char*, int); extern void magic_fini(void*); versus use crate_of_magic::{init, stuff, fini}; It even enables one to actually type-check against the imported crate during the compilation (IIRC if Cargo can't locate the imported crate to look into it, it will refuse to build the project), as opposed to hoping that the headers you've included are correctly describing the object file you'll be linking against.
0x457 2y ago
Only time I get linker errors in rust is when it's linking some dynamic library written in C.

wyldfire 2y ago

All but the most trivial programs require a linker to resolve references among object files. And while "int main() {}" might seem like a trivial C program, it's not (by that definition, at least). Your favorite toolchain will often include archives and objects that you might take for granted like crt0.o, init.o, fini.o, libgcc/clang_rt.builtins and more. The compiler's design is simplified by not having to resolve references among symbols. The assembler can do this for references within a section and linkers can do it among sections. Linkers might have to add trampolines/thunks for relocations that span a distance longer than the opcode could reach. Loaders do this symbol resolution and relocation at runtime.

high_na_euv 2y ago
So, if hello world is not trivial Then you can easily generate e.g web assembly from your own lang input without linker and it will run fine Aint the same viable for x86?
- wyldfire 2y ago
  > So, if hello world is not trivial Note that the nontrivial C program I showed above was not "hello world". Emitting output is yet more complicated still. > you can easily generate e.g web assembly from your own lang input without linker IIRC some things that generate wasm generate object files (e.g. C translation units). Those wasm objects require a linker to resolve symbol references to create the executable. > it will run fine ...on a wasm virtual machine. IIRC Java doesn't have anything like a traditional linker either. > Aint the same viable for x86? I mean, you can do anything you want inventing a new compiler that does much more than traditional compilers. But consider perhaps that there's a lot of value in the status quo that you might appreciate more if you were more familiar with the design of a toolchain. It's not that linkers are an absolute drop-dead necessity, it's that having one allows you to have distinct assembler and compiler functions that are much simpler to design by focusing on one kind of transformation.