4 ms·

This CFI implementation makes &func be different values in 2 different loadable kernel modules. C says they should be the same value. What do the language lawye

by yosefk 5y ago

This CFI implementation makes &func be different values in 2 different loadable kernel modules. C says they should be the same value. What do the language lawyers among the compiler writers, who explain why the execution of an entire program is rendered meaningless by undefined behavior, tell us about this? What is the definition of the language compiled with CFI enabled (it's not C but some close relative)?

Joker_vD 5y ago
Could you elaborate the scenario a bit? Two different loadable kernel modules would define two different (even if named the same) functions "func" so "&func" has to be two different pointers... or do you mean that if that "func" was defined in the kernel, and both of those modules are linked against the kernel dynamically, then "&func" should yield the same value? But even now they don't even without CFI: when "func" is defined in the dinamically-loaded library, the "&func" value is the pointer into the (calling object file's) PLT.
derefr 5y ago
> What is the definition of the language compiled with CFI enabled (it's not C but some close relative)? I think I saw mentioned, in the comments of a recent HN post about undefined behavior in C, that OS kernels aren't using "standard C", as they use a bunch of compiler flags to constrain what the compiler does in the face of UB, contrary to what the standard says the compiler is allowed to do. As such, OS kernels have always kind of been using a "close relative of C." (Same syntax, slightly different semantics.)
bcoates 5y ago
Loadable modules aren't part of the C spec. Whatever the meaning of &func in 2 different modules potentially compiled by 2 different compilers is is specified by the ABI. CFI is a breaking change to the contract the Linux kernel makes with kernel modules, but it doesn't have anything in particular to do with C the language.
KMag 5y ago
I'm not a language lawyer, but I'm pretty sure with x86 and x86_64 ELF dynamic symbols, &func is the address of the program linkage table (PLT) entry for the dynamic symbol, in the current ELF executable/library. The executable has its own PLT. Each library has its own PLT. The executable has its own global offset table (GOT). Each library has its own GOT. A given dynamically linked function will have one PLT entry and one GOT entry in each library that calls it. After the function's actual global entry point has been resolved, its address will be in the GOT entry. However, symbol resolution can be done lazily, and we don't want taking the address of a function to potentially perform the lookup, so I think the function address is just the start of the PLT entry. For instance, if the executable and several libraries it loads all call printf, they'll each have their own tiny machine code stub (PLT entry) that (after dynamic symbol resolution) just jumps to the actual printf implementation. The executable will use the start of its PLT entry for printf as its address for printf. Each library will use the start of its PLT entry for printf as its address for printf. The PLT entry is a tiny machine code stub within the same library or executable that does an indirect jump to the address held in a global offset table (GOT) entry. For lazily resolved symbols, the GOT is initialized to the address of a stub that does the dynamic symbol resolution and overwrites the GOT entry with the correct address (and then jumps to that address). Any later function call will still hit the PLT and jump to the address in the GOT, but the GOT will now point to the actual start of the function.
- pritambaral 5y ago
  So if I take the address of printf in one library, and ask another library to compare it with their address of printf, will they not be equal?
  - bombela 5y ago
    Correct. They won't be equal. Since each library has its own table.
    - KMag 5y ago
      Except I was wrong... sorry. I haven't been able to find documentation that lazy ELF symbol binding is disabled when taking the address of an extern fuction, but using objdump to dump the .plt section, it's clear printf is no longer lazily bound when you need to take its address. Instead, the entry is marked to be resolved at load time, and the global function entry point is just read directly from the GOT entry. Part of my assumption that the PLT entry was used as the function address came from trying to figure out why the AMD engineers didn't include an ip-relative indirect addressing mode for the call instruction when they designed x86_64. An ip-relative indirect call could directly call through the GOT and avoid wasting an instruction cache line on the PLT entry. The PLT entry would still be used once for lazy symbol binding, but after that wouldn't be used and wouldn't cause any more cache evictions. Such an indirect call would need to be broken down into several micro-ops internally, but would save instruction cache space. I did a bit of thinking and came to the conclusion that the PLT entry was still necessary for taking the function address of a lazily bound ELF symbol, so such an addressing mode would almost never be used. Now that I see the PLT entry is actually never used for taking the function address, I'm a bit surprised that AMD when designing long mode (x86_64) didn't include an ip-relative indirect addressing mode for the call instruction. I'm just a mechanical engineer by training who's way too much self-taught about software, so I'm sure the AMD hardware engineers (and maybe also RISC-V folks... I'm less familiar with RISC-V addressing) had very good reasons for not having ip-relative indirect function calls. My best guess is that the complexity increase wasn't worth it and/or there was some hidden performance cost that's not obvious to me.

by yosefk 5y ago

Joker_vD 5y ago

Could you elaborate the scenario a bit? Two different loadable kernel modules would define two different (even if named the same) functions "func" so "&func" has to be two different pointers... or do you mean that if that "func" was defined in the kernel, and both of those modules are linked against the kernel dynamically, then "&func" should yield the same value? But even now they don't even without CFI: when "func" is defined in the dinamically-loaded library, the "&func" value is the pointer into the (calling object file's) PLT.

derefr 5y ago

> What is the definition of the language compiled with CFI enabled (it's not C but some close relative)? I think I saw mentioned, in the comments of a recent HN post about undefined behavior in C, that OS kernels aren't using "standard C", as they use a bunch of compiler flags to constrain what the compiler does in the face of UB, contrary to what the standard says the compiler is allowed to do. As such, OS kernels have always kind of been using a "close relative of C." (Same syntax, slightly different semantics.)

bcoates 5y ago

Loadable modules aren't part of the C spec. Whatever the meaning of &func in 2 different modules potentially compiled by 2 different compilers is is specified by the ABI. CFI is a breaking change to the contract the Linux kernel makes with kernel modules, but it doesn't have anything in particular to do with C the language.

KMag 5y ago

I'm not a language lawyer, but I'm pretty sure with x86 and x86_64 ELF dynamic symbols, &func is the address of the program linkage table (PLT) entry for the dynamic symbol, in the current ELF executable/library. The executable has its own PLT. Each library has its own PLT. The executable has its own global offset table (GOT). Each library has its own GOT. A given dynamically linked function will have one PLT entry and one GOT entry in each library that calls it. After the function's actual global entry point has been resolved, its address will be in the GOT entry. However, symbol resolution can be done lazily, and we don't want taking the address of a function to potentially perform the lookup, so I think the function address is just the start of the PLT entry. For instance, if the executable and several libraries it loads all call printf, they'll each have their own tiny machine code stub (PLT entry) that (after dynamic symbol resolution) just jumps to the actual printf implementation. The executable will use the start of its PLT entry for printf as its address for printf. Each library will use the start of its PLT entry for printf as its address for printf. The PLT entry is a tiny machine code stub within the same library or executable that does an indirect jump to the address held in a global offset table (GOT) entry. For lazily resolved symbols, the GOT is initialized to the address of a stub that does the dynamic symbol resolution and overwrites the GOT entry with the correct address (and then jumps to that address). Any later function call will still hit the PLT and jump to the address in the GOT, but the GOT will now point to the actual start of the function.

pritambaral 5y ago
So if I take the address of printf in one library, and ask another library to compare it with their address of printf, will they not be equal?
- bombela 5y ago
  Correct. They won't be equal. Since each library has its own table.
  - KMag 5y ago
    Except I was wrong... sorry. I haven't been able to find documentation that lazy ELF symbol binding is disabled when taking the address of an extern fuction, but using objdump to dump the .plt section, it's clear printf is no longer lazily bound when you need to take its address. Instead, the entry is marked to be resolved at load time, and the global function entry point is just read directly from the GOT entry. Part of my assumption that the PLT entry was used as the function address came from trying to figure out why the AMD engineers didn't include an ip-relative indirect addressing mode for the call instruction when they designed x86_64. An ip-relative indirect call could directly call through the GOT and avoid wasting an instruction cache line on the PLT entry. The PLT entry would still be used once for lazy symbol binding, but after that wouldn't be used and wouldn't cause any more cache evictions. Such an indirect call would need to be broken down into several micro-ops internally, but would save instruction cache space. I did a bit of thinking and came to the conclusion that the PLT entry was still necessary for taking the function address of a lazily bound ELF symbol, so such an addressing mode would almost never be used. Now that I see the PLT entry is actually never used for taking the function address, I'm a bit surprised that AMD when designing long mode (x86_64) didn't include an ip-relative indirect addressing mode for the call instruction. I'm just a mechanical engineer by training who's way too much self-taught about software, so I'm sure the AMD hardware engineers (and maybe also RISC-V folks... I'm less familiar with RISC-V addressing) had very good reasons for not having ip-relative indirect function calls. My best guess is that the complexity increase wasn't worth it and/or there was some hidden performance cost that's not obvious to me.