7 ms·

But dynamically linked library only needs to be loaded to RAM once whereas with static linking you'd be loading the same code many times (unless you compile eve

by marwis 3y ago

But dynamically linked library only needs to be loaded to RAM once whereas with static linking you'd be loading the same code many times (unless you compile everything to single binary like BusyBox). This also gets you better cache utilization. Also I think inlining would typically increase the total size of output rather than decrease it.

jhallenworld 3y ago
So you only need to load duplicated code for each different statically linked program. If there are many processes running the same program, they will all share the same physical pages for the code. So for example, having 100s of "bash" instances running does not use that much memory. You can see this by running "pmap <pid> -XX" (the output is very wide- probably load it into an editor). Look at the shared vs. private pages. Also: There is another way to automatically share pages between different programs: de-duplication. This would require common libraries to be statically linked on page boundaries. The OS would quickly de-duplicate during loading by hashing the pages. VMs use this technique to increase effective memory when there are many guest OS running.
- marwis 3y ago
  Yes but most processes are unique. The only bash process I have running is my interactive shell.
arghwhat 3y ago
Static linking gives you better instruction cache utilization as you are executing local code linearly rather than going through indirection with more boilerplate. This indirection costs a few cycles too. Inlining external code reduces the size not only by saving the call, PLT and and stack dance, but also through specialization (removal of unused conditional, pruning of no longer referenced symbols) as the code is locally optimized. This further reduction in size further improves cache behavior and performance. Duplication can be an issue (not necessarily for performance, but for total binary size), but compilers have heuristics for that. Even just having the symbol local saves some space and call overhead though (no PLT). The case for the shared library having better caching implies multiple processes that are distinct executables (otherwise they share program memory regardless of linkage) trying to hammer it at once, sharing the continued caching, but such scenario is hurt by the call overhead and lower optimization opportunities, negating the result.
- marwis 3y ago
  > The case for the shared library having better caching implies multiple processes that are distinct executables But this is the most common case for desktops/multipurpose systems. On my desktop there are tens or hundreds distinct processes sharing most of their code.
  - arghwhat 3y ago
    No it is not. Depending on your CPU, you might have, say, 32KB of 8-way associative instruction cache per core. Just being shared does not make it fit in the cache. A shared library would only be there across processes of different executable images if its users primarily, continuously execute the same paths in shared libs rather than anything unique in their own executable image - e.g., they'd more or less need to be stuck in the same processing-intensive shared routine in the lib. There would also have to be no other processing done in between by other processes that would have trashed the cache. On the other hand, the severe cache penalty of longer code paths for each executable and the larger PLT call overhead will universally lead to a loss in performance for all library usage. The scenarios you may hit where different processes are actually executing the same shared code paths to the point of benefiting from shared cache utilization would be cases where they share executable image as well. E.g., browser processes, threads, compilers. Electron too if using system-packaged electron binaries.
    - marwis 3y ago
      > A shared library would only be there across processes of different executable images if its users primarily, continuously execute the same paths in shared libs rather than anything unique in their own executable image Yes, like libc/WinApi, Cairo/whatever graphics library of your OS, Gtk/Qt etc All of the apps on my desktop (I don't use electron) share vast majority of their code (see above) and spend most of their time drawing UI through shared libs or doing their own business logic but that's part of their code and not subject to shared lib overhead.
      - arghwhat 3y ago

by marwis 3y ago

jhallenworld 3y ago

So you only need to load duplicated code for each different statically linked program. If there are many processes running the same program, they will all share the same physical pages for the code. So for example, having 100s of "bash" instances running does not use that much memory. You can see this by running "pmap <pid> -XX" (the output is very wide- probably load it into an editor). Look at the shared vs. private pages. Also: There is another way to automatically share pages between different programs: de-duplication. This would require common libraries to be statically linked on page boundaries. The OS would quickly de-duplicate during loading by hashing the pages. VMs use this technique to increase effective memory when there are many guest OS running.

marwis 3y ago
Yes but most processes are unique. The only bash process I have running is my interactive shell.

arghwhat 3y ago

Static linking gives you better instruction cache utilization as you are executing local code linearly rather than going through indirection with more boilerplate. This indirection costs a few cycles too. Inlining external code reduces the size not only by saving the call, PLT and and stack dance, but also through specialization (removal of unused conditional, pruning of no longer referenced symbols) as the code is locally optimized. This further reduction in size further improves cache behavior and performance. Duplication can be an issue (not necessarily for performance, but for total binary size), but compilers have heuristics for that. Even just having the symbol local saves some space and call overhead though (no PLT). The case for the shared library having better caching implies multiple processes that are distinct executables (otherwise they share program memory regardless of linkage) trying to hammer it at once, sharing the continued caching, but such scenario is hurt by the call overhead and lower optimization opportunities, negating the result.

marwis 3y ago
> The case for the shared library having better caching implies multiple processes that are distinct executables But this is the most common case for desktops/multipurpose systems. On my desktop there are tens or hundreds distinct processes sharing most of their code.
- arghwhat 3y ago
  No it is not. Depending on your CPU, you might have, say, 32KB of 8-way associative instruction cache per core. Just being shared does not make it fit in the cache. A shared library would only be there across processes of different executable images if its users primarily, continuously execute the same paths in shared libs rather than anything unique in their own executable image - e.g., they'd more or less need to be stuck in the same processing-intensive shared routine in the lib. There would also have to be no other processing done in between by other processes that would have trashed the cache. On the other hand, the severe cache penalty of longer code paths for each executable and the larger PLT call overhead will universally lead to a loss in performance for all library usage. The scenarios you may hit where different processes are actually executing the same shared code paths to the point of benefiting from shared cache utilization would be cases where they share executable image as well. E.g., browser processes, threads, compilers. Electron too if using system-packaged electron binaries.
  - marwis 3y ago
    > A shared library would only be there across processes of different executable images if its users primarily, continuously execute the same paths in shared libs rather than anything unique in their own executable image Yes, like libc/WinApi, Cairo/whatever graphics library of your OS, Gtk/Qt etc All of the apps on my desktop (I don't use electron) share vast majority of their code (see above) and spend most of their time drawing UI through shared libs or doing their own business logic but that's part of their code and not subject to shared lib overhead.
    - arghwhat 3y ago