5 ms·

I'm not a language lawyer, but I'm pretty sure with x86 and x86_64 ELF dynamic symbols, &func is the address of the program linkage table (PLT) entry for the dy

by KMag 5y ago

I'm not a language lawyer, but I'm pretty sure with x86 and x86_64 ELF dynamic symbols, &func is the address of the program linkage table (PLT) entry for the dynamic symbol, in the current ELF executable/library. The executable has its own PLT. Each library has its own PLT. The executable has its own global offset table (GOT). Each library has its own GOT. A given dynamically linked function will have one PLT entry and one GOT entry in each library that calls it. After the function's actual global entry point has been resolved, its address will be in the GOT entry. However, symbol resolution can be done lazily, and we don't want taking the address of a function to potentially perform the lookup, so I think the function address is just the start of the PLT entry. For instance, if the executable and several libraries it loads all call printf, they'll each have their own tiny machine code stub (PLT entry) that (after dynamic symbol resolution) just jumps to the actual printf implementation. The executable will use the start of its PLT entry for printf as its address for printf. Each library will use the start of its PLT entry for printf as its address for printf. The PLT entry is a tiny machine code stub within the same library or executable that does an indirect jump to the address held in a global offset table (GOT) entry. For lazily resolved symbols, the GOT is initialized to the address of a stub that does the dynamic symbol resolution and overwrites the GOT entry with the correct address (and then jumps to that address). Any later function call will still hit the PLT and jump to the address in the GOT, but the GOT will now point to the actual start of the function.

pritambaral 5y ago
So if I take the address of printf in one library, and ask another library to compare it with their address of printf, will they not be equal?
- bombela 5y ago
  Correct. They won't be equal. Since each library has its own table.
  - KMag 5y ago
    Except I was wrong... sorry. I haven't been able to find documentation that lazy ELF symbol binding is disabled when taking the address of an extern fuction, but using objdump to dump the .plt section, it's clear printf is no longer lazily bound when you need to take its address. Instead, the entry is marked to be resolved at load time, and the global function entry point is just read directly from the GOT entry. Part of my assumption that the PLT entry was used as the function address came from trying to figure out why the AMD engineers didn't include an ip-relative indirect addressing mode for the call instruction when they designed x86_64. An ip-relative indirect call could directly call through the GOT and avoid wasting an instruction cache line on the PLT entry. The PLT entry would still be used once for lazy symbol binding, but after that wouldn't be used and wouldn't cause any more cache evictions. Such an indirect call would need to be broken down into several micro-ops internally, but would save instruction cache space. I did a bit of thinking and came to the conclusion that the PLT entry was still necessary for taking the function address of a lazily bound ELF symbol, so such an addressing mode would almost never be used. Now that I see the PLT entry is actually never used for taking the function address, I'm a bit surprised that AMD when designing long mode (x86_64) didn't include an ip-relative indirect addressing mode for the call instruction. I'm just a mechanical engineer by training who's way too much self-taught about software, so I'm sure the AMD hardware engineers (and maybe also RISC-V folks... I'm less familiar with RISC-V addressing) had very good reasons for not having ip-relative indirect function calls. My best guess is that the complexity increase wasn't worth it and/or there was some hidden performance cost that's not obvious to me.
- andrewf 5y ago
  I just tried to make this work and couldn't. I might be missing something? gcc 7.5, Ubuntu 18.04, x86_64. The third-from-bottom line is two libraries showing the same address. $ cat *.h *.c // common.h #include <stdio.h> typedef int (*printf_ptr)(const char * restrict format, ...); printf_ptr firstA(); printf_ptr firstB(); printf_ptr secondA(); printf_ptr secondB(); // first.c #include "common.h" printf_ptr firstA() { return &printf; } printf_ptr firstB() { return &printf; } // second.c #include "common.h" printf_ptr secondA() { return &printf; } printf_ptr secondB() { return &printf; } // third.c #include "common.h" int main() { printf("%p %p %p %p\n", firstA(), firstB(), secondA(), secondB()); firstA()("hello world %d\n", 1); secondA()("hello world %d\n", 2); return 0; } $ make gcc -fPIC -c *.c gcc -shared -o first.so first.o -lc gcc -shared -o second.so second.o -lc gcc -o atprintf third.o ./first.so ./second.so ./atprintf 0x7f6d24e08f70 0x7f6d24e08f70 0x7f6d24e08f70 0x7f6d24e08f70 hello world 1 hello world 2
  - KMag 5y ago
    Ahh... I was wrong! There's one detail I wasn't aware of: if you take the address of a dynamic function, it appears to disable ELF lazy dynamic symbol binding for that function, so that it's guaranteed the GOT entry has been resolved to the global function address before main() is entered. It then just unconditionally uses the GOT entry as the function address. If we print the address of main and never take the address of printf, we get lazy symbol resolution for printf like I expected: #include <stdio.h> int main() { printf("main is at %p\n", main); return 0; } objdump --section .plt -d a.out shows 2 PLT entries: 0000000000001020 <.plt>: 1020: ff 35 e2 2f 00 00 pushq 0x2fe2(%rip) # 4008 <_GLOBAL_OFFSET_TABLE_+0x8> 1026: ff 25 e4 2f 00 00 jmpq *0x2fe4(%rip) # 4010 <_GLOBAL_OFFSET_TABLE_+0x10> 102c: 0f 1f 40 00 nopl 0x0(%rax) 0000000000001030 <printf@plt>: 1030: ff 25 e2 2f 00 00 jmpq *0x2fe2(%rip) # 4018 <printf@GLIBC_2.2.5> 1036: 68 00 00 00 00 pushq $0x0 103b: e9 e0 ff ff ff jmpq 1020 <.plt> That jmpq *0x2fe2(%rip) in the second PLT entry is an indirect jump through the prnitf GOT entry. The printf GOT entry is actually initialized to point right back at the pushq $0x0 inside pritf@plt. That pushes 0 on the stack so the dynamic symbol resolution knows which GOT entry it's lazily resolving. The jmpq 1020 <.plt> jumps to the first PLT entry, which then uses the 0 at the top of the stack to know it's resolving the printf GOT entry. But, if we ever actually take the address of printf, then printf ceases to be a lazily bound ELF symbol: #include <stdio.h> int main() { printf("printf is at %p\n", printf); return 0; } Note that we lose the dynamic resolution PLT stub for printf (objdump --section .plt -d a.out): 0000000000001020 <.plt>: 1020: ff 35 e2 2f 00 00 pushq 0x2fe2(%rip) # 4008 <_GLOBAL_OFFSET_TABLE_+0x8> 1026: ff 25 e4 2f 00 00 jmpq *0x2fe4(%rip) # 4010 <_GLOBAL_OFFSET_TABLE_+0x10> 102c: 0f 1f 40 00 nopl 0x0(%rax) And gcc -O2 -S main.c shows it's just unconditionally loading the GOT entry (printf@GOTPCREL) to use as the function address. (Note the dissasembly shows printf@PLT, but objdump doesn't show this PLT entry. I guess the linker does some link-time optimization there to remove the actual PLT entry.) main: .LFB11: .cfi_startproc subq $8, %rsp .cfi_def_cfa_offset 16 movq printf@GOTPCREL(%rip), %rsi leaq .LC0(%rip), %rdi xorl %eax, %eax call printf@PLT https://refspecs.linuxfoundation.org/ELF/zSeries/lzsabi0_zSeries/x2251.html https://refspecs.linuxfoundation.org/ELF/zSeries/lzsabi0_zSe... (section named Function Addresses) mentions that things (at least for IBM zSeries) basically work as I originally expected, but further vaguely mentions some special steps are taken to make function addresses compare as expected. It doesn't specifically mention disabling ELF dynamic symbol resolution.*

by KMag 5y ago

pritambaral 5y ago

So if I take the address of printf in one library, and ask another library to compare it with their address of printf, will they not be equal?

bombela 5y ago
Correct. They won't be equal. Since each library has its own table.
- KMag 5y ago
  Except I was wrong... sorry. I haven't been able to find documentation that lazy ELF symbol binding is disabled when taking the address of an extern fuction, but using objdump to dump the .plt section, it's clear printf is no longer lazily bound when you need to take its address. Instead, the entry is marked to be resolved at load time, and the global function entry point is just read directly from the GOT entry. Part of my assumption that the PLT entry was used as the function address came from trying to figure out why the AMD engineers didn't include an ip-relative indirect addressing mode for the call instruction when they designed x86_64. An ip-relative indirect call could directly call through the GOT and avoid wasting an instruction cache line on the PLT entry. The PLT entry would still be used once for lazy symbol binding, but after that wouldn't be used and wouldn't cause any more cache evictions. Such an indirect call would need to be broken down into several micro-ops internally, but would save instruction cache space. I did a bit of thinking and came to the conclusion that the PLT entry was still necessary for taking the function address of a lazily bound ELF symbol, so such an addressing mode would almost never be used. Now that I see the PLT entry is actually never used for taking the function address, I'm a bit surprised that AMD when designing long mode (x86_64) didn't include an ip-relative indirect addressing mode for the call instruction. I'm just a mechanical engineer by training who's way too much self-taught about software, so I'm sure the AMD hardware engineers (and maybe also RISC-V folks... I'm less familiar with RISC-V addressing) had very good reasons for not having ip-relative indirect function calls. My best guess is that the complexity increase wasn't worth it and/or there was some hidden performance cost that's not obvious to me.
andrewf 5y ago
I just tried to make this work and couldn't. I might be missing something? gcc 7.5, Ubuntu 18.04, x86_64. The third-from-bottom line is two libraries showing the same address. $ cat *.h *.c // common.h #include <stdio.h> typedef int (*printf_ptr)(const char * restrict format, ...); printf_ptr firstA(); printf_ptr firstB(); printf_ptr secondA(); printf_ptr secondB(); // first.c #include "common.h" printf_ptr firstA() { return &printf; } printf_ptr firstB() { return &printf; } // second.c #include "common.h" printf_ptr secondA() { return &printf; } printf_ptr secondB() { return &printf; } // third.c #include "common.h" int main() { printf("%p %p %p %p\n", firstA(), firstB(), secondA(), secondB()); firstA()("hello world %d\n", 1); secondA()("hello world %d\n", 2); return 0; } $ make gcc -fPIC -c *.c gcc -shared -o first.so first.o -lc gcc -shared -o second.so second.o -lc gcc -o atprintf third.o ./first.so ./second.so ./atprintf 0x7f6d24e08f70 0x7f6d24e08f70 0x7f6d24e08f70 0x7f6d24e08f70 hello world 1 hello world 2
- KMag 5y ago
  Ahh... I was wrong! There's one detail I wasn't aware of: if you take the address of a dynamic function, it appears to disable ELF lazy dynamic symbol binding for that function, so that it's guaranteed the GOT entry has been resolved to the global function address before main() is entered. It then just unconditionally uses the GOT entry as the function address. If we print the address of main and never take the address of printf, we get lazy symbol resolution for printf like I expected: #include <stdio.h> int main() { printf("main is at %p\n", main); return 0; } objdump --section .plt -d a.out shows 2 PLT entries: 0000000000001020 <.plt>: 1020: ff 35 e2 2f 00 00 pushq 0x2fe2(%rip) # 4008 <_GLOBAL_OFFSET_TABLE_+0x8> 1026: ff 25 e4 2f 00 00 jmpq *0x2fe4(%rip) # 4010 <_GLOBAL_OFFSET_TABLE_+0x10> 102c: 0f 1f 40 00 nopl 0x0(%rax) 0000000000001030 <printf@plt>: 1030: ff 25 e2 2f 00 00 jmpq *0x2fe2(%rip) # 4018 <printf@GLIBC_2.2.5> 1036: 68 00 00 00 00 pushq $0x0 103b: e9 e0 ff ff ff jmpq 1020 <.plt> That jmpq *0x2fe2(%rip) in the second PLT entry is an indirect jump through the prnitf GOT entry. The printf GOT entry is actually initialized to point right back at the pushq $0x0 inside pritf@plt. That pushes 0 on the stack so the dynamic symbol resolution knows which GOT entry it's lazily resolving. The jmpq 1020 <.plt> jumps to the first PLT entry, which then uses the 0 at the top of the stack to know it's resolving the printf GOT entry. But, if we ever actually take the address of printf, then printf ceases to be a lazily bound ELF symbol: #include <stdio.h> int main() { printf("printf is at %p\n", printf); return 0; } Note that we lose the dynamic resolution PLT stub for printf (objdump --section .plt -d a.out): 0000000000001020 <.plt>: 1020: ff 35 e2 2f 00 00 pushq 0x2fe2(%rip) # 4008 <_GLOBAL_OFFSET_TABLE_+0x8> 1026: ff 25 e4 2f 00 00 jmpq *0x2fe4(%rip) # 4010 <_GLOBAL_OFFSET_TABLE_+0x10> 102c: 0f 1f 40 00 nopl 0x0(%rax) And gcc -O2 -S main.c shows it's just unconditionally loading the GOT entry (printf@GOTPCREL) to use as the function address. (Note the dissasembly shows printf@PLT, but objdump doesn't show this PLT entry. I guess the linker does some link-time optimization there to remove the actual PLT entry.) main: .LFB11: .cfi_startproc subq $8, %rsp .cfi_def_cfa_offset 16 movq printf@GOTPCREL(%rip), %rsi leaq .LC0(%rip), %rdi xorl %eax, %eax call printf@PLT https://refspecs.linuxfoundation.org/ELF/zSeries/lzsabi0_zSeries/x2251.html https://refspecs.linuxfoundation.org/ELF/zSeries/lzsabi0_zSe... (section named Function Addresses) mentions that things (at least for IBM zSeries) basically work as I originally expected, but further vaguely mentions some special steps are taken to make function addresses compare as expected. It doesn't specifically mention disabling ELF dynamic symbol resolution.*