4 ms·

I am pretty familiar with object file formats, and I don't get how this "atom" stuff works. The site says that "atoms" are an improvement over simple section i

I am pretty familiar with object file formats, and I don't get how this "atom" stuff works. The site says that "atoms" are an improvement over simple section interlacing. But I don't get how you are going to make this leap without changing the object file format. Linkers work on the section level because that is how object files work. Object files have sections, not atoms. Compilers emit sections as their basic, atomic unit of output. Within a section, the code will assume that all offsets referring to other parts of the section will be stable, so you can't chop a section apart without breaking the code. How does the new linker work in terms of this new "atom" abstraction without changing the underlying object file format?

Someone 10y ago
http://releases.llvm.org/3.9.0/tools/lld/docs/Readers.html http://releases.llvm.org/3.9.0/tools/lld/docs/Readers.html Seems to indicate that many object file formats contain the information you need to split them up in such a fine-grained manner. But maybe, they require you to create special versions of object files where even references internal to each library are referenced there as if they live in a different object file? Is that even possible? Edit: corrections welcome, but https://github.com/llvm-mirror/lld/tree/master/lib/ReaderWriter https://github.com/llvm-mirror/lld/tree/master/lib/ReaderWri... seems to indicate that this only sopports Mach-o (and a yaml-based format used for debugging and testing)
- chisophugis 10y ago
  > But maybe, they require you to create special versions of object files where even references internal to each library are referenced there as if they live in a different object file? Is that even possible? The extra information that is needed for an ELF linker (any ELF linker; nothing LLD specific) to operate on functions and global data objects in a fine-grained manner is enabled by -ffunction-sections/-fdata-sections. For more information, see https://news.ycombinator.com/item?id=13673589 https://news.ycombinator.com/item?id=13673589
Joky 10y ago
If you are familiar with object file formats in general, you may know that this is exactly how MachO works: it is based on atoms. If you want to map ELF to the atom model, you need somehow to build with -ffunction-section so that the compiler emits a single function per section (and similarly with -fdata-section) or model it by mapping one section of the object to an atom.
- haberman 10y ago
  Hmm, in my experience with Mach-O, I have never come across atoms. For example, in this file format reference, atoms are not mentioned -- only the more traditional segment/section hierarchy: https://github.com/aidansteele/osx-abi-macho-file-format-reference https://github.com/aidansteele/osx-abi-macho-file-format-ref... What am I missing? I do note that on OS X, -ffunction-sections appears to do nothing.
  - Joky 10y ago
    Sorry, I was wrong to characterize the atom being a core part of the MachO object format while it is only a core part of how ld64 works. The compiler is following some convention though that ld64 takes advantages of. Other than ld64 source code (available on opensource.apple.com) I can only point to some design document in the source repo: https://opensource.apple.com/source/ld64/ld64-253.6/doc/design/linker.html https://opensource.apple.com/source/ld64/ld64-253.6/doc/desi...
jeffbush 10y ago
They abandoned the atom concept for ELF & PE/COFF for the reason you described. https://llvm.cc/t/llvmdev-lld-improvement-plan/274 https://llvm.cc/t/llvmdev-lld-improvement-plan/274 "The atom model is not the best model for some architectures The atom model makes sense only for Mach-O, but it’s used everywhere. I guess that we originally expected that we would be able to model the linker’s behavior beautifully using the atom model because the atom model seemed like a superset of the section model. Although it can, it turned out that it’s not necessarily natural and efficient model for ELF or PE/COFF on which section-based linking is expected."

Someone 10y ago

http://releases.llvm.org/3.9.0/tools/lld/docs/Readers.html http://releases.llvm.org/3.9.0/tools/lld/docs/Readers.html Seems to indicate that many object file formats contain the information you need to split them up in such a fine-grained manner. But maybe, they require you to create special versions of object files where even references internal to each library are referenced there as if they live in a different object file? Is that even possible? Edit: corrections welcome, but https://github.com/llvm-mirror/lld/tree/master/lib/ReaderWriter https://github.com/llvm-mirror/lld/tree/master/lib/ReaderWri... seems to indicate that this only sopports Mach-o (and a yaml-based format used for debugging and testing)

chisophugis 10y ago
> But maybe, they require you to create special versions of object files where even references internal to each library are referenced there as if they live in a different object file? Is that even possible? The extra information that is needed for an ELF linker (any ELF linker; nothing LLD specific) to operate on functions and global data objects in a fine-grained manner is enabled by -ffunction-sections/-fdata-sections. For more information, see https://news.ycombinator.com/item?id=13673589 https://news.ycombinator.com/item?id=13673589

Joky 10y ago

If you are familiar with object file formats in general, you may know that this is exactly how MachO works: it is based on atoms. If you want to map ELF to the atom model, you need somehow to build with -ffunction-section so that the compiler emits a single function per section (and similarly with -fdata-section) or model it by mapping one section of the object to an atom.

haberman 10y ago
Hmm, in my experience with Mach-O, I have never come across atoms. For example, in this file format reference, atoms are not mentioned -- only the more traditional segment/section hierarchy: https://github.com/aidansteele/osx-abi-macho-file-format-reference https://github.com/aidansteele/osx-abi-macho-file-format-ref... What am I missing? I do note that on OS X, -ffunction-sections appears to do nothing.
- Joky 10y ago
  Sorry, I was wrong to characterize the atom being a core part of the MachO object format while it is only a core part of how ld64 works. The compiler is following some convention though that ld64 takes advantages of. Other than ld64 source code (available on opensource.apple.com) I can only point to some design document in the source repo: https://opensource.apple.com/source/ld64/ld64-253.6/doc/design/linker.html https://opensource.apple.com/source/ld64/ld64-253.6/doc/desi...

jeffbush 10y ago

They abandoned the atom concept for ELF & PE/COFF for the reason you described. https://llvm.cc/t/llvmdev-lld-improvement-plan/274 https://llvm.cc/t/llvmdev-lld-improvement-plan/274 "The atom model is not the best model for some architectures The atom model makes sense only for Mach-O, but it’s used everywhere. I guess that we originally expected that we would be able to model the linker’s behavior beautifully using the atom model because the atom model seemed like a superset of the section model. Although it can, it turned out that it’s not necessarily natural and efficient model for ELF or PE/COFF on which section-based linking is expected."