4 ms·

On linux, none of these libraries are duplicated, they are loaded once, and shared across all processes that need them.

barrkel 7y ago
Even when the processes are loading the libraries from different paths, in different filesystems, in different containers? How does it page in data on demand if the first container that loaded the library is killed and its filesystem unloaded? It's not easy to share libraries across containers, unless they can be built to share a base layer in a stacked union filesystem approach.
- saagarjha 7y ago
  Why can’t the linker deduplicate libraries as they’re loaded?
  - pas 7y ago
    kernel samepage merging already exists
- deleted 7y ago
  [deleted]
- bnegreve 7y ago
  >Even when the processes are loading the libraries from different paths, Well, in my original post (GGGP) I've defined "a library" as "a .so file" so what I can say is that the 872 distinct .so files used on my laptop will be shared among the different processes that use them. If you assume the same library can be duplicated in two different .so files, then 872 is just an upper bound on the number of distinct libraries and further sharing could be done. Eitherway, that is a significant amount of code sharing, which was the original question in this thread.
berkut 7y ago
Only if they're the same version (which using the older built-in package system all packages are likely built against the same version)... If the containerised versions of apps all have different versions of dependencies (quite likely IMO as they'll have the freedom to), there won't be any sharing.
- viraptor 7y ago
  Or even different locations. (unless they're hardlinks, or COW files probably) If it's not the same block on the disk, it's going to be duplicated in cache - whether it's the same contents or not.
  - berkut 7y ago
    If the soname is the same between the duplicate copies, and another copy is in $LD_LIBRARY_PATH it shouldn't matter about location.

Even when the processes are loading the libraries from different paths, in different filesystems, in different containers? How does it page in data on demand if the first container that loaded the library is killed and its filesystem unloaded? It's not easy to share libraries across containers, unless they can be built to share a base layer in a stacked union filesystem approach.

saagarjha 7y ago
Why can’t the linker deduplicate libraries as they’re loaded?
- pas 7y ago
  kernel samepage merging already exists
deleted 7y ago
[deleted]
bnegreve 7y ago
>Even when the processes are loading the libraries from different paths, Well, in my original post (GGGP) I've defined "a library" as "a .so file" so what I can say is that the 872 distinct .so files used on my laptop will be shared among the different processes that use them. If you assume the same library can be duplicated in two different .so files, then 872 is just an upper bound on the number of distinct libraries and further sharing could be done. Eitherway, that is a significant amount of code sharing, which was the original question in this thread.

berkut 7y ago

Only if they're the same version (which using the older built-in package system all packages are likely built against the same version)... If the containerised versions of apps all have different versions of dependencies (quite likely IMO as they'll have the freedom to), there won't be any sharing.

viraptor 7y ago
Or even different locations. (unless they're hardlinks, or COW files probably) If it's not the same block on the disk, it's going to be duplicated in cache - whether it's the same contents or not.
- berkut 7y ago
  If the soname is the same between the duplicate copies, and another copy is in $LD_LIBRARY_PATH it shouldn't matter about location.