3 ms·

So if I take the address of printf in one library, and ask another library to compare it with their address of printf, will they not be equal?

bombela 5y ago
Correct. They won't be equal. Since each library has its own table.
- KMag 5y ago
  Except I was wrong... sorry. I haven't been able to find documentation that lazy ELF symbol binding is disabled when taking the address of an extern fuction, but using objdump to dump the .plt section, it's clear printf is no longer lazily bound when you need to take its address. Instead, the entry is marked to be resolved at load time, and the global function entry point is just read directly from the GOT entry. Part of my assumption that the PLT entry was used as the function address came from trying to figure out why the AMD engineers didn't include an ip-relative indirect addressing mode for the call instruction when they designed x86_64. An ip-relative indirect call could directly call through the GOT and avoid wasting an instruction cache line on the PLT entry. The PLT entry would still be used once for lazy symbol binding, but after that wouldn't be used and wouldn't cause any more cache evictions. Such an indirect call would need to be broken down into several micro-ops internally, but would save instruction cache space. I did a bit of thinking and came to the conclusion that the PLT entry was still necessary for taking the function address of a lazily bound ELF symbol, so such an addressing mode would almost never be used. Now that I see the PLT entry is actually never used for taking the function address, I'm a bit surprised that AMD when designing long mode (x86_64) didn't include an ip-relative indirect addressing mode for the call instruction. I'm just a mechanical engineer by training who's way too much self-taught about software, so I'm sure the AMD hardware engineers (and maybe also RISC-V folks... I'm less familiar with RISC-V addressing) had very good reasons for not having ip-relative indirect function calls. My best guess is that the complexity increase wasn't worth it and/or there was some hidden performance cost that's not obvious to me.
andrewf 5y ago
I just tried to make this work and couldn't. I might be missing something? gcc 7.5, Ubuntu 18.04, x86_64. The third-from-bottom line is two libraries showing the same address. $ cat *.h *.c // common.h #include <stdio.h> typedef int (*printf_ptr)(const char * restrict format, ...); printf_ptr firstA(); printf_ptr firstB(); printf_ptr secondA(); printf_ptr secondB(); // first.c #include "common.h" printf_ptr firstA() { return &printf; } printf_ptr firstB() { return &printf; } // second.c #include "common.h" printf_ptr secondA() { return &printf; } printf_ptr secondB() { return &printf; } // third.c #include "common.h" int main() { printf("%p %p %p %p\n", firstA(), firstB(), secondA(), secondB()); firstA()("hello world %d\n", 1); secondA()("hello world %d\n", 2); return 0; } $ make gcc -fPIC -c *.c gcc -shared -o first.so first.o -lc gcc -shared -o second.so second.o -lc gcc -o atprintf third.o ./first.so ./second.so ./atprintf 0x7f6d24e08f70 0x7f6d24e08f70 0x7f6d24e08f70 0x7f6d24e08f70 hello world 1 hello world 2
- KMag 5y ago
  Ahh... I was wrong! There's one detail I wasn't aware of: if you take the address of a dynamic function, it appears to disable ELF lazy dynamic symbol binding for that function, so that it's guaranteed the GOT entry has been resolved to the global function address before main() is entered. It then just unconditionally uses the GOT entry as the function address. If we print the address of main and never take the address of printf, we get lazy symbol resolution for printf like I expected: #include <stdio.h> int main() { printf("main is at %p\n", main); return 0; } objdump --section .plt -d a.out shows 2 PLT entries: 0000000000001020 <.plt>: 1020: ff 35 e2 2f 00 00 pushq 0x2fe2(%rip) # 4008 <_GLOBAL_OFFSET_TABLE_+0x8> 1026: ff 25 e4 2f 00 00 jmpq *0x2fe4(%rip) # 4010 <_GLOBAL_OFFSET_TABLE_+0x10> 102c: 0f 1f 40 00 nopl 0x0(%rax) 0000000000001030 <printf@plt>: 1030: ff 25 e2 2f 00 00 jmpq *0x2fe2(%rip) # 4018 <printf@GLIBC_2.2.5> 1036: 68 00 00 00 00 pushq $0x0 103b: e9 e0 ff ff ff jmpq 1020 <.plt> That jmpq *0x2fe2(%rip) in the second PLT entry is an indirect jump through the prnitf GOT entry. The printf GOT entry is actually initialized to point right back at the pushq $0x0 inside pritf@plt. That pushes 0 on the stack so the dynamic symbol resolution knows which GOT entry it's lazily resolving. The jmpq 1020 <.plt> jumps to the first PLT entry, which then uses the 0 at the top of the stack to know it's resolving the printf GOT entry. But, if we ever actually take the address of printf, then printf ceases to be a lazily bound ELF symbol: #include <stdio.h> int main() { printf("printf is at %p\n", printf); return 0; } Note that we lose the dynamic resolution PLT stub for printf (objdump --section .plt -d a.out): 0000000000001020 <.plt>: 1020: ff 35 e2 2f 00 00 pushq 0x2fe2(%rip) # 4008 <_GLOBAL_OFFSET_TABLE_+0x8> 1026: ff 25 e4 2f 00 00 jmpq *0x2fe4(%rip) # 4010 <_GLOBAL_OFFSET_TABLE_+0x10> 102c: 0f 1f 40 00 nopl 0x0(%rax) And gcc -O2 -S main.c shows it's just unconditionally loading the GOT entry (printf@GOTPCREL) to use as the function address. (Note the dissasembly shows printf@PLT, but objdump doesn't show this PLT entry. I guess the linker does some link-time optimization there to remove the actual PLT entry.) main: .LFB11: .cfi_startproc subq $8, %rsp .cfi_def_cfa_offset 16 movq printf@GOTPCREL(%rip), %rsi leaq .LC0(%rip), %rdi xorl %eax, %eax call printf@PLT https://refspecs.linuxfoundation.org/ELF/zSeries/lzsabi0_zSeries/x2251.html https://refspecs.linuxfoundation.org/ELF/zSeries/lzsabi0_zSe... (section named Function Addresses) mentions that things (at least for IBM zSeries) basically work as I originally expected, but further vaguely mentions some special steps are taken to make function addresses compare as expected. It doesn't specifically mention disabling ELF dynamic symbol resolution.*

bombela 5y ago

Correct. They won't be equal. Since each library has its own table.

KMag 5y ago
Except I was wrong... sorry. I haven't been able to find documentation that lazy ELF symbol binding is disabled when taking the address of an extern fuction, but using objdump to dump the .plt section, it's clear printf is no longer lazily bound when you need to take its address. Instead, the entry is marked to be resolved at load time, and the global function entry point is just read directly from the GOT entry. Part of my assumption that the PLT entry was used as the function address came from trying to figure out why the AMD engineers didn't include an ip-relative indirect addressing mode for the call instruction when they designed x86_64. An ip-relative indirect call could directly call through the GOT and avoid wasting an instruction cache line on the PLT entry. The PLT entry would still be used once for lazy symbol binding, but after that wouldn't be used and wouldn't cause any more cache evictions. Such an indirect call would need to be broken down into several micro-ops internally, but would save instruction cache space. I did a bit of thinking and came to the conclusion that the PLT entry was still necessary for taking the function address of a lazily bound ELF symbol, so such an addressing mode would almost never be used. Now that I see the PLT entry is actually never used for taking the function address, I'm a bit surprised that AMD when designing long mode (x86_64) didn't include an ip-relative indirect addressing mode for the call instruction. I'm just a mechanical engineer by training who's way too much self-taught about software, so I'm sure the AMD hardware engineers (and maybe also RISC-V folks... I'm less familiar with RISC-V addressing) had very good reasons for not having ip-relative indirect function calls. My best guess is that the complexity increase wasn't worth it and/or there was some hidden performance cost that's not obvious to me.

andrewf 5y ago

I just tried to make this work and couldn't. I might be missing something? gcc 7.5, Ubuntu 18.04, x86_64. The third-from-bottom line is two libraries showing the same address. $ cat *.h *.c // common.h #include <stdio.h> typedef int (*printf_ptr)(const char * restrict format, ...); printf_ptr firstA(); printf_ptr firstB(); printf_ptr secondA(); printf_ptr secondB(); // first.c #include "common.h" printf_ptr firstA() { return &printf; } printf_ptr firstB() { return &printf; } // second.c #include "common.h" printf_ptr secondA() { return &printf; } printf_ptr secondB() { return &printf; } // third.c #include "common.h" int main() { printf("%p %p %p %p\n", firstA(), firstB(), secondA(), secondB()); firstA()("hello world %d\n", 1); secondA()("hello world %d\n", 2); return 0; } $ make gcc -fPIC -c *.c gcc -shared -o first.so first.o -lc gcc -shared -o second.so second.o -lc gcc -o atprintf third.o ./first.so ./second.so ./atprintf 0x7f6d24e08f70 0x7f6d24e08f70 0x7f6d24e08f70 0x7f6d24e08f70 hello world 1 hello world 2

KMag 5y ago
Ahh... I was wrong! There's one detail I wasn't aware of: if you take the address of a dynamic function, it appears to disable ELF lazy dynamic symbol binding for that function, so that it's guaranteed the GOT entry has been resolved to the global function address before main() is entered. It then just unconditionally uses the GOT entry as the function address. If we print the address of main and never take the address of printf, we get lazy symbol resolution for printf like I expected: #include <stdio.h> int main() { printf("main is at %p\n", main); return 0; } objdump --section .plt -d a.out shows 2 PLT entries: 0000000000001020 <.plt>: 1020: ff 35 e2 2f 00 00 pushq 0x2fe2(%rip) # 4008 <_GLOBAL_OFFSET_TABLE_+0x8> 1026: ff 25 e4 2f 00 00 jmpq *0x2fe4(%rip) # 4010 <_GLOBAL_OFFSET_TABLE_+0x10> 102c: 0f 1f 40 00 nopl 0x0(%rax) 0000000000001030 <printf@plt>: 1030: ff 25 e2 2f 00 00 jmpq *0x2fe2(%rip) # 4018 <printf@GLIBC_2.2.5> 1036: 68 00 00 00 00 pushq $0x0 103b: e9 e0 ff ff ff jmpq 1020 <.plt> That jmpq *0x2fe2(%rip) in the second PLT entry is an indirect jump through the prnitf GOT entry. The printf GOT entry is actually initialized to point right back at the pushq $0x0 inside pritf@plt. That pushes 0 on the stack so the dynamic symbol resolution knows which GOT entry it's lazily resolving. The jmpq 1020 <.plt> jumps to the first PLT entry, which then uses the 0 at the top of the stack to know it's resolving the printf GOT entry. But, if we ever actually take the address of printf, then printf ceases to be a lazily bound ELF symbol: #include <stdio.h> int main() { printf("printf is at %p\n", printf); return 0; } Note that we lose the dynamic resolution PLT stub for printf (objdump --section .plt -d a.out): 0000000000001020 <.plt>: 1020: ff 35 e2 2f 00 00 pushq 0x2fe2(%rip) # 4008 <_GLOBAL_OFFSET_TABLE_+0x8> 1026: ff 25 e4 2f 00 00 jmpq *0x2fe4(%rip) # 4010 <_GLOBAL_OFFSET_TABLE_+0x10> 102c: 0f 1f 40 00 nopl 0x0(%rax) And gcc -O2 -S main.c shows it's just unconditionally loading the GOT entry (printf@GOTPCREL) to use as the function address. (Note the dissasembly shows printf@PLT, but objdump doesn't show this PLT entry. I guess the linker does some link-time optimization there to remove the actual PLT entry.) main: .LFB11: .cfi_startproc subq $8, %rsp .cfi_def_cfa_offset 16 movq printf@GOTPCREL(%rip), %rsi leaq .LC0(%rip), %rdi xorl %eax, %eax call printf@PLT https://refspecs.linuxfoundation.org/ELF/zSeries/lzsabi0_zSeries/x2251.html https://refspecs.linuxfoundation.org/ELF/zSeries/lzsabi0_zSe... (section named Function Addresses) mentions that things (at least for IBM zSeries) basically work as I originally expected, but further vaguely mentions some special steps are taken to make function addresses compare as expected. It doesn't specifically mention disabling ELF dynamic symbol resolution.*