4 ms·

Windows and macOS both use a form of two level name-spacing, which does the same sort of direct binding to a target library for each symbol. Retrofitting that i

by lgg 2y ago

Windows and macOS both use a form of two level name-spacing, which does the same sort of direct binding to a target library for each symbol. Retrofitting that into a binary format is pretty simple, but retrofitting it into an ecosystem that depends on the existing flat namespace look up semantics is not. I think it is pretty clever that the author noticed the static nature of the nix store allows them to statically evaluate the symbol resolutions and get the launch time benefits of two level namespaces. I do wonder if it might make more sense to rewrite the binaries to use Direct Binding[1]. That is an existing encoding of library targets for symbols in ELF that has been used by Solaris for a number of years. 1: https://en.wikipedia.org/wiki/Direct_binding https://en.wikipedia.org/wiki/Direct_binding

JonChesterfield 2y ago
That is much better than the Linux model! Not only is there less crawling around looking for symbols, you're no longer in trouble when two libraries export the same symbol. Especially given libraries are found by name, and symbols by name, where "type information" or "is that actually the library I wanted" are afterthoughts.
- cryptonector 2y ago
  > you're no longer in trouble when two libraries export the same symbol. Whether you use direct binding or symbol versioning, either way you don't have a problem with multiple libraries exporting the same symbol. By the way, this is the fundamental problem with static linking for C: it's still stuck with 1970s semantics and you can't get the same symbol conflict resolution semantics as with ELF because the static linker-editors do not record dependencies in static link archives. The key insight is that when you link-edit your libraries and programs you should provide only the direct dependencies, and the linker-editor should then record in its output which one of those provided which symbol. Compare to static linking where only the final edit gets the dependency information and that dependency tree has to get flattened (because it has to fit on a command-line, which is linear in nature).
- o11c 2y ago
  The advantage of the Linux model is that you can refactor which library actually contains a function, which is done quite often.
  - cryptonector 2y ago
    Filters and auxiliary filters also do that, but Linux doesn't support them well, which is really sad because they make for a very neat system.
glandium 2y ago
I think you can get an effect similar to direct binding with symbol versioning.
- cryptonector 2y ago
  You can get the same semantics as for direct binding using symbol versioning, but direct binding is faster. Also, symbol versioning is only really better than direct binding if you end up having multiple versions of the same symbol provided by the same object, but that's relatively hard to use, so it's really only ever used for things like the C library itself. Mind you, that is a very valuable feature when you need it. In Solaris itself when we needed to deal with the various different behaviors of snprintf() there just wasn't a good way to do it, and only symbol versioning with support for multiple versions of a symbol would have helped.
- lgg 2y ago
  Not really... symbol versioning is a form of namespacing, but it is somewhat orthogonal to this. Symbol versioning allows you to have multiple symbols with the same name namespaced by version, but you still have no control over what library in the search path they will be found in. So it does not improve the speed of the runtime searching (since they could be in any library an the search path and you still need to search for them in order), and it does not provide the the same binary compatibility support and dylib hijacking protection (since again, any dylibs earlier in the search path could declare a symbol with he same name. One could use symbol versioning to construct a system where you had the same binary protection guarantees, but it would involve every library declaring a unique version string, and guaranteeing there are no collisions. The obvious way to do that would be to use the file path as the symbol version, at which point you have reinvented mach-o install names, except: 1. You still do not get the runtime speed ups unless you change the dynamic linker behavior to use the version string as the search path, which would require ecosystem wide changes. 2. You can't actually use symbol versioning to do versioned symbols any more, since you overloaded the use of version strings (mach-o binaries end up accomplishing symbol versioning through header tricks with `asmname`, so it is not completely intractable to do even without explicit support).
  - 2y ago

by lgg 2y ago

JonChesterfield 2y ago

That is much better than the Linux model! Not only is there less crawling around looking for symbols, you're no longer in trouble when two libraries export the same symbol. Especially given libraries are found by name, and symbols by name, where "type information" or "is that actually the library I wanted" are afterthoughts.

cryptonector 2y ago
> you're no longer in trouble when two libraries export the same symbol. Whether you use direct binding or symbol versioning, either way you don't have a problem with multiple libraries exporting the same symbol. By the way, this is the fundamental problem with static linking for C: it's still stuck with 1970s semantics and you can't get the same symbol conflict resolution semantics as with ELF because the static linker-editors do not record dependencies in static link archives. The key insight is that when you link-edit your libraries and programs you should provide only the direct dependencies, and the linker-editor should then record in its output which one of those provided which symbol. Compare to static linking where only the final edit gets the dependency information and that dependency tree has to get flattened (because it has to fit on a command-line, which is linear in nature).
o11c 2y ago
The advantage of the Linux model is that you can refactor which library actually contains a function, which is done quite often.
- cryptonector 2y ago
  Filters and auxiliary filters also do that, but Linux doesn't support them well, which is really sad because they make for a very neat system.

glandium 2y ago

I think you can get an effect similar to direct binding with symbol versioning.

cryptonector 2y ago
You can get the same semantics as for direct binding using symbol versioning, but direct binding is faster. Also, symbol versioning is only really better than direct binding if you end up having multiple versions of the same symbol provided by the same object, but that's relatively hard to use, so it's really only ever used for things like the C library itself. Mind you, that is a very valuable feature when you need it. In Solaris itself when we needed to deal with the various different behaviors of snprintf() there just wasn't a good way to do it, and only symbol versioning with support for multiple versions of a symbol would have helped.
lgg 2y ago
Not really... symbol versioning is a form of namespacing, but it is somewhat orthogonal to this. Symbol versioning allows you to have multiple symbols with the same name namespaced by version, but you still have no control over what library in the search path they will be found in. So it does not improve the speed of the runtime searching (since they could be in any library an the search path and you still need to search for them in order), and it does not provide the the same binary compatibility support and dylib hijacking protection (since again, any dylibs earlier in the search path could declare a symbol with he same name. One could use symbol versioning to construct a system where you had the same binary protection guarantees, but it would involve every library declaring a unique version string, and guaranteeing there are no collisions. The obvious way to do that would be to use the file path as the symbol version, at which point you have reinvented mach-o install names, except: 1. You still do not get the runtime speed ups unless you change the dynamic linker behavior to use the version string as the search path, which would require ecosystem wide changes. 2. You can't actually use symbol versioning to do versioned symbols any more, since you overloaded the use of version strings (mach-o binaries end up accomplishing symbol versioning through header tricks with `asmname`, so it is not completely intractable to do even without explicit support).
- 2y ago