6 ms·

Why did dlclose not unload the library? (2023)

39 points by mattrighetti 1y ago

sgbeal 1y ago
There is no safe, reliable, cross-environment way to deal with closing a DLL. A DLL initialization function can allocate arbitrary resources, some of which may be in use by clients of the DLL when it is closed. The only safe, consistent, reliable approach is not to close DLLs.
- 10000truths 1y ago
  You can run the DLL in a "shim" subprocess that proxies function calls over IPC. Then the DLL can muck about with global state all it wants, and the OS will clean up after it when you "unload" the DLL by killing the subprocess.
  - sgbeal 1y ago
    > You can run the DLL in a "shim" subprocess that proxies function calls over IPC. That doesn't address the need of some DLLs to malloc() resources in the context of the applications linking to them. This problem _cannot_ be solved _generically_. Any solutions are extremely API-specific and impose restrictions on their users (the linking applications) which, if violated, will lead to Undefined Behavior. Edit: as an example of cases which must bind resources in the application's context: see the Classloading in C++ paper at <https://wanderinghorse.net/computing/papers/index.html#classloading_cpp https://wanderinghorse.net/computing/papers/index.html#class...> (disclosure: i wrote that article).
    - 10000truths 1y ago
      True, it's application specific. The typical reason an application would need to load/unload DLLs cleanly and repeatedly (where the caveats of dlclose are in full force) is for a plugin system with hot-reload functionality. And for that use case, the expected API/ABI is known in advance, so the shim can be tailored for it.
    - LegionMammal978 1y ago
      "Extremely" API-specific is a bit much. One way to do it would be to guard all DLL-implemented functions, and all access to DLL-associated resources, with a reference-counted token, or otherwise use a language-level mechanism (such as Rust's lifetimes) to ensure that all resources are dead by the time of closure. This would take some tedious wrapper-writing, but it's not so complex that it couldn't be done by a source generator or whatever. Of course, C/C++ applications written in the traditional model with static data everwhere would have difficulty not leaking tokens and holding the DLL open, but it's still far from impossible to write such a safe API. > That doesn't address the need of some DLLs to malloc() resources in the context of the applications linking to them. If there is a context boundary and really such a need, then the DLL can keep a list of all such resources, and destroy all those resources once closed. Access to them would similarly have to be protected by a token.
      - zbentley 1y ago
        > One way to do it would be to guard all DLL-implemented functions, and all access to DLL-associated resources, with a reference-counted token, or otherwise use a language-level mechanism (such as Rust's lifetimes) to ensure that all resources are dead by the time of closure. That's true, but those approaches are only viable if you trust the DLL in question. External libraries are fundamentally opaque/could contain anything, and if you're in a tinfoil-hat mood, it's quite easy to make new libraries that emulate the ABI of the intended library but do different (maybe malicious, maybe just LD_PRELOAD tricksy) things. Consider: an evil wrapper library could put the thinnest possible shim around the "real" version of the library and just not properly account for resources, exposing library (un)loaders to use-after-free without much work, even if the library loaders relied upon the approaches proposed. Since there aren't good cross-platform and race-condition-free ways of saying "authenticate this external library via checksum/codesigning, then load it", there are some situations where the proposed approaches aren't good enough. Sure, most situations probably don't need that paranoia level (or control the code/provenance of the target library implicitly). But the number of situations where that security risk does come up is larger than you'd think, especially given automatic look-up-library-by-name-via-LD_LIBRARY_PATH-ish behavior.
- AndrewStephens 1y ago
  This is the way. As this article details, there are so many circumstances that preclude DLLs being unloaded completely that I was surprised that their design actually worked at all. So many language constructs do not play nicely with the idea that code and static data can just disappear at runtime.
- dataflow 1y ago
  I feel like I wouldn't frame it like this. Rather, the underlying assumption of the problem has to be that the DLL's resources are already released, otherwise the problem of what happens when the resources are used afterward is itself ill-posed. The problem is really how to ensure that.
- immibis 1y ago
  There is no safe, reliable, cross-environment way to deal with deallocating memory. A memory block can be referenced from arbitrary locations, some of which may be on the stack by clients of the memory block when it isndeallocated. The only safe, consistent, reliable approach is not to deallocate memory.
  - zbentley 1y ago
    You joke, but there are programs that do exactly this.
  - SkiFire13 1y ago
    The difference is that memory won't do anything under your nose, it can't run arbitrary code by itself. It won't spawn threads, create thread locals, or store data in global variables. And it's normal to track the lifetime of memory, much less the lifetime of code and function pointers passed around.
    - immibis 1y ago
      The same is true of DLLs. They don't do anything by themselves; they are merely blocks of bytes mapped into memory. Why is tracking the lifetime of a function pointer different from tracking the lifetime of any other pointer? FreeLibrary on Windows unloads libraries when the reference count is zero.
      - SkiFire13 1y ago
        > The same is true of DLLs. They don't do anything by themselves; they are merely blocks of bytes mapped into memory. FYI just loading a DLL can execute code. > Why is tracking the lifetime of a function pointer different from tracking the lifetime of any other pointer? Technically speaking none, but historically this is something that nobody did, so we are full of code that assumes function pointers to have a static lifetime. It's a social problem.
    - AstralStorm 1y ago
      You could do that, but then you're running valgrind.
- justincormack 1y ago
  thats why Musl libc has dlclose as a no-op [0] [0] https://wiki.musl-libc.org/functional-differences-from-glibc.html https://wiki.musl-libc.org/functional-differences-from-glibc...
- kazinator 1y ago
  The DLL consists of code and static data. If nothing needs the code or the static data, it can be blown away. If that library allocated something which it is responsible for freeing, but that something still exists even though that library's very code is no longer referenced (nobody will call into it), it almost certainly means there is a memory leak. A memory leak caused by a library isn't necessarily something that should prevent it from being unloaded. Unless it is the leaked objects that are holding a reference! (Then we need a full blown GC system to detect cycles.)
duped 1y ago
I'm curious what design led them to split code between two shared libraries but also require the state of them to be synchronized across calls to dlopen and dlclose. It sounds like that state should be in one library and not two.
- theamk 1y ago
  they kinda mention this - one of those is Rust, other is C++. My guess they made an all-new Rust plugin which used an existing C++ library. Pretty common case when existing code base is slowly being converted to rust.
  - duped 1y ago
    But why are they dynamically linking to each other?
  - 01HNNWZ0MV43FF 1y ago
    I guess the C++ library might be binary-only or difficult to compile but, for curious readers, you can definitely link C++ and Rust into a single binary library or exe
  - saurik 1y ago
    But how/why is libA failing if libB has already been initialized? That's the thing which seems broken.
zbentley 1y ago
Not only is dlclose(3) not guaranteed to close/unload the library, close(2) isn't guaranteed to close a file!. If the same file descriptor is open in another process (e.g. one that has it via fork(2) FD sharing either without child processes exec(2)ing or with CLOEXEC not set, or some older and esoteric abuses of fdpassing over UNIX sockets), then close(2) just decrements a refcount. The actual file isn't closed until the last holder of a reference to its file descriptor calls close(2). This is rarely relevant, but when it comes up, it sure is wild. Common sources of confusion and pain due to this behavior are: files opened for "global library reasons" (e.g. /dev/shm buffers) in preforking servers, signalfd descriptors in preforking servers, processes that fork off and daemonize but leave their spawner around for e.g. CoW memory sharing efficiencies (looking at you, Python multiprocessing backends--the docs make it sound like they all act more or less the same, but in this regard they very much do not), libraries that usually swap STDIN/OUT/ERR file descriptors around with CLOEXEC disabled for a manually fork+exec'd child (e.g. situations where posix_spawn doesn't support needed pre-exec setup) but that are then used as part of larger applications that fork/spawn for other reasons and don't realize that file descriptor allocation/forking needs care and synchronization with the manually-fork/execing library in question: mixing forks and threads is one of those things that everyone says is a fast-track ticket to nasal demons, but that everyone also does regularly, I've found--if this describes you, be careful! If you end up in one of those situations, suddenly invariants like "I called unlink on this path and then close(2)'d the descriptor to it, so that (maybe large) chunk of allocated space isn't taking up space on the filesystem/buffers any more" and "this externally-observable lockfile/directory is now unlocked due to its absence, now processes coordinating using that file on NFS will work as expected" no longer hold. https://www.ibm.com/docs/en/aix/7.1.0?topic=domains-unix-domain-properties https://www.ibm.com/docs/en/aix/7.1.0?topic=domains-unix-dom... I know that close(2)'s weirdness isn't a superset of dlopen(3)'s and there are different reasons for both behaviors. But it's still interesting that they "rhyme" as it were.
- tux3 1y ago
  >or some older and esoteric abuses of fdpassing over UNIX sockets One of the less well designed APIs, but as an aside it is still widely used for IPC.
- 1y ago
pharrington 1y ago
Call the library's shutdown procedure before you dlclose. A dynamically linked library is a resource like any other. Ya gotta close it properly. Of course, if the library doesn't probably attempt to close libraries it's responsible for during its shutdown procedure, that's another can of worms.
kazinator 1y ago
There needs to be a way to forcibly run the thread destructors for a library that is going away. Since the library is going away, it means it's not used any more. Not being used means that no thread that is currently running will call into that library any more. If no thread that is currently running will use the code of that library, it has no business hanging on to the data belonging to the library; that data should only be manipulable via that code. Therefore, it should be forcibly taken away. The reason they flunk on this issue is that there isn't a nice way to enumerate through all the threads and snipe away a particular class of thread local storage from each. The architecture is oriented around the POSIX-induced brain-damaged idea that a thread must clean up all thread-specific storage after itself. Fight me!
abstractspoon 1y ago
What if lib.B had another dependency? To deal with this dlclose would have to query every loaded module for its dependencies and then decide if lib.B could safely be unloaded. What if lib.B had been loaded explicitly somewhere else such that it did not appear in any other module's dependency list?
- LegionMammal978 1y ago
  Do you mean, what if lib.B had another dependent? Every loaded library has a reference count, which counts both dependents and explicit dlopen() calls. Unless the caller is the last user (and the other conditions are satisfied), dlclose() has no effect except to decrement the reference count.

39 points by mattrighetti 1y ago

There is no safe, reliable, cross-environment way to deal with closing a DLL. A DLL initialization function can allocate arbitrary resources, some of which may be in use by clients of the DLL when it is closed. The only safe, consistent, reliable approach is not to close DLLs.

10000truths 1y ago
You can run the DLL in a "shim" subprocess that proxies function calls over IPC. Then the DLL can muck about with global state all it wants, and the OS will clean up after it when you "unload" the DLL by killing the subprocess.
- sgbeal 1y ago
  > You can run the DLL in a "shim" subprocess that proxies function calls over IPC. That doesn't address the need of some DLLs to malloc() resources in the context of the applications linking to them. This problem _cannot_ be solved _generically_. Any solutions are extremely API-specific and impose restrictions on their users (the linking applications) which, if violated, will lead to Undefined Behavior. Edit: as an example of cases which must bind resources in the application's context: see the Classloading in C++ paper at <https://wanderinghorse.net/computing/papers/index.html#classloading_cpp https://wanderinghorse.net/computing/papers/index.html#class...> (disclosure: i wrote that article).
  - 10000truths 1y ago
    True, it's application specific. The typical reason an application would need to load/unload DLLs cleanly and repeatedly (where the caveats of dlclose are in full force) is for a plugin system with hot-reload functionality. And for that use case, the expected API/ABI is known in advance, so the shim can be tailored for it.
  - LegionMammal978 1y ago
    "Extremely" API-specific is a bit much. One way to do it would be to guard all DLL-implemented functions, and all access to DLL-associated resources, with a reference-counted token, or otherwise use a language-level mechanism (such as Rust's lifetimes) to ensure that all resources are dead by the time of closure. This would take some tedious wrapper-writing, but it's not so complex that it couldn't be done by a source generator or whatever. Of course, C/C++ applications written in the traditional model with static data everwhere would have difficulty not leaking tokens and holding the DLL open, but it's still far from impossible to write such a safe API. > That doesn't address the need of some DLLs to malloc() resources in the context of the applications linking to them. If there is a context boundary and really such a need, then the DLL can keep a list of all such resources, and destroy all those resources once closed. Access to them would similarly have to be protected by a token.
    - zbentley 1y ago
      > One way to do it would be to guard all DLL-implemented functions, and all access to DLL-associated resources, with a reference-counted token, or otherwise use a language-level mechanism (such as Rust's lifetimes) to ensure that all resources are dead by the time of closure. That's true, but those approaches are only viable if you trust the DLL in question. External libraries are fundamentally opaque/could contain anything, and if you're in a tinfoil-hat mood, it's quite easy to make new libraries that emulate the ABI of the intended library but do different (maybe malicious, maybe just LD_PRELOAD tricksy) things. Consider: an evil wrapper library could put the thinnest possible shim around the "real" version of the library and just not properly account for resources, exposing library (un)loaders to use-after-free without much work, even if the library loaders relied upon the approaches proposed. Since there aren't good cross-platform and race-condition-free ways of saying "authenticate this external library via checksum/codesigning, then load it", there are some situations where the proposed approaches aren't good enough. Sure, most situations probably don't need that paranoia level (or control the code/provenance of the target library implicitly). But the number of situations where that security risk does come up is larger than you'd think, especially given automatic look-up-library-by-name-via-LD_LIBRARY_PATH-ish behavior.
AndrewStephens 1y ago
This is the way. As this article details, there are so many circumstances that preclude DLLs being unloaded completely that I was surprised that their design actually worked at all. So many language constructs do not play nicely with the idea that code and static data can just disappear at runtime.
dataflow 1y ago
I feel like I wouldn't frame it like this. Rather, the underlying assumption of the problem has to be that the DLL's resources are already released, otherwise the problem of what happens when the resources are used afterward is itself ill-posed. The problem is really how to ensure that.
immibis 1y ago
There is no safe, reliable, cross-environment way to deal with deallocating memory. A memory block can be referenced from arbitrary locations, some of which may be on the stack by clients of the memory block when it isndeallocated. The only safe, consistent, reliable approach is not to deallocate memory.
- zbentley 1y ago
  You joke, but there are programs that do exactly this.
- SkiFire13 1y ago
  The difference is that memory won't do anything under your nose, it can't run arbitrary code by itself. It won't spawn threads, create thread locals, or store data in global variables. And it's normal to track the lifetime of memory, much less the lifetime of code and function pointers passed around.
  - immibis 1y ago
    The same is true of DLLs. They don't do anything by themselves; they are merely blocks of bytes mapped into memory. Why is tracking the lifetime of a function pointer different from tracking the lifetime of any other pointer? FreeLibrary on Windows unloads libraries when the reference count is zero.
    - SkiFire13 1y ago
      > The same is true of DLLs. They don't do anything by themselves; they are merely blocks of bytes mapped into memory. FYI just loading a DLL can execute code. > Why is tracking the lifetime of a function pointer different from tracking the lifetime of any other pointer? Technically speaking none, but historically this is something that nobody did, so we are full of code that assumes function pointers to have a static lifetime. It's a social problem.
  - AstralStorm 1y ago
    You could do that, but then you're running valgrind.
justincormack 1y ago
thats why Musl libc has dlclose as a no-op [0] [0] https://wiki.musl-libc.org/functional-differences-from-glibc.html https://wiki.musl-libc.org/functional-differences-from-glibc...
kazinator 1y ago
The DLL consists of code and static data. If nothing needs the code or the static data, it can be blown away. If that library allocated something which it is responsible for freeing, but that something still exists even though that library's very code is no longer referenced (nobody will call into it), it almost certainly means there is a memory leak. A memory leak caused by a library isn't necessarily something that should prevent it from being unloaded. Unless it is the leaked objects that are holding a reference! (Then we need a full blown GC system to detect cycles.)

duped 1y ago

I'm curious what design led them to split code between two shared libraries but also require the state of them to be synchronized across calls to dlopen and dlclose. It sounds like that state should be in one library and not two.

theamk 1y ago
they kinda mention this - one of those is Rust, other is C++. My guess they made an all-new Rust plugin which used an existing C++ library. Pretty common case when existing code base is slowly being converted to rust.
- duped 1y ago
  But why are they dynamically linking to each other?
- 01HNNWZ0MV43FF 1y ago
  I guess the C++ library might be binary-only or difficult to compile but, for curious readers, you can definitely link C++ and Rust into a single binary library or exe
- saurik 1y ago
  But how/why is libA failing if libB has already been initialized? That's the thing which seems broken.

zbentley 1y ago

Not only is dlclose(3) not guaranteed to close/unload the library, close(2) isn't guaranteed to close a file!. If the same file descriptor is open in another process (e.g. one that has it via fork(2) FD sharing either without child processes exec(2)ing or with CLOEXEC not set, or some older and esoteric abuses of fdpassing over UNIX sockets), then close(2) just decrements a refcount. The actual file isn't closed until the last holder of a reference to its file descriptor calls close(2). This is rarely relevant, but when it comes up, it sure is wild. Common sources of confusion and pain due to this behavior are: files opened for "global library reasons" (e.g. /dev/shm buffers) in preforking servers, signalfd descriptors in preforking servers, processes that fork off and daemonize but leave their spawner around for e.g. CoW memory sharing efficiencies (looking at you, Python multiprocessing backends--the docs make it sound like they all act more or less the same, but in this regard they very much do not), libraries that usually swap STDIN/OUT/ERR file descriptors around with CLOEXEC disabled for a manually fork+exec'd child (e.g. situations where posix_spawn doesn't support needed pre-exec setup) but that are then used as part of larger applications that fork/spawn for other reasons and don't realize that file descriptor allocation/forking needs care and synchronization with the manually-fork/execing library in question: mixing forks and threads is one of those things that everyone says is a fast-track ticket to nasal demons, but that everyone also does regularly, I've found--if this describes you, be careful! If you end up in one of those situations, suddenly invariants like "I called unlink on this path and then close(2)'d the descriptor to it, so that (maybe large) chunk of allocated space isn't taking up space on the filesystem/buffers any more" and "this externally-observable lockfile/directory is now unlocked due to its absence, now processes coordinating using that file on NFS will work as expected" no longer hold. https://www.ibm.com/docs/en/aix/7.1.0?topic=domains-unix-domain-properties https://www.ibm.com/docs/en/aix/7.1.0?topic=domains-unix-dom... I know that close(2)'s weirdness isn't a superset of dlopen(3)'s and there are different reasons for both behaviors. But it's still interesting that they "rhyme" as it were.

tux3 1y ago
>or some older and esoteric abuses of fdpassing over UNIX sockets One of the less well designed APIs, but as an aside it is still widely used for IPC.
1y ago

pharrington 1y ago

Call the library's shutdown procedure before you dlclose. A dynamically linked library is a resource like any other. Ya gotta close it properly. Of course, if the library doesn't probably attempt to close libraries it's responsible for during its shutdown procedure, that's another can of worms.

kazinator 1y ago

There needs to be a way to forcibly run the thread destructors for a library that is going away. Since the library is going away, it means it's not used any more. Not being used means that no thread that is currently running will call into that library any more. If no thread that is currently running will use the code of that library, it has no business hanging on to the data belonging to the library; that data should only be manipulable via that code. Therefore, it should be forcibly taken away. The reason they flunk on this issue is that there isn't a nice way to enumerate through all the threads and snipe away a particular class of thread local storage from each. The architecture is oriented around the POSIX-induced brain-damaged idea that a thread must clean up all thread-specific storage after itself. Fight me!

abstractspoon 1y ago

What if lib.B had another dependency? To deal with this dlclose would have to query every loaded module for its dependencies and then decide if lib.B could safely be unloaded. What if lib.B had been loaded explicitly somewhere else such that it did not appear in any other module's dependency list?

LegionMammal978 1y ago
Do you mean, what if lib.B had another dependent? Every loaded library has a reference count, which counts both dependents and explicit dlopen() calls. Unless the caller is the last user (and the other conditions are satisfied), dlclose() has no effect except to decrement the reference count.