4 ms·

I'd love to see stats on how many libraries are really shared, I'd imagine it might be less than you hope (other than the big graphical ones)

• bnegreve 7y ago
  On my laptop: How many libraries are loaded: $ sudo cat /proc/[0-9]*/maps | grep '\.so' | grep 'r-xp' | tr -s ' ' | cut -d ' ' -f 6 | wc -l 15429 How many unique library names: $ sudo cat /proc/[0-9]*/maps | grep '\.so' | grep 'r-xp' | tr -s ' ' | cut -d ' ' -f 6 | sort | uniq | wc -l 872 Top 10 most shared libraries: sudo cat /proc/[0-9]*/maps | grep '\.so' | grep 'r-xp' | tr -s ' ' | cut -d ' ' -f 6 | awk '{count[$0]++}END{for (i in count) print i, count[i]}' | sort -k 2 -n -r | head -n 10 /lib/x86_64-linux-gnu/libc-2.28.so 299 /lib/x86_64-linux-gnu/ld-2.28.so 299 /lib/x86_64-linux-gnu/libdl-2.28.so 262 /lib/x86_64-linux-gnu/libpthread-2.28.so 237 /lib/x86_64-linux-gnu/librt-2.28.so 227 /lib/x86_64-linux-gnu/libuuid.so.1.3.0 205 /lib/x86_64-linux-gnu/libz.so.1.2.11 200 /lib/x86_64-linux-gnu/libpcre.so.3.13.3 186 /lib/x86_64-linux-gnu/libresolv-2.28.so 170 /lib/x86_64-linux-gnu/libgpg-error.so.0.26.1 169 EDIT: add grep r-xp to count code segment only, values in previous edit were overestimated.
  • the_pwner224 7y ago
    When you're running a GUI those libraries also get shared. For example Qt might not be used by as many processes as libc, but it saves hundreds of megabytes of memory having that shared across my email/calendar/terminals/torrent client/windowManager/guiShell/pdfReader/webBrowser. The GP did mention the 'big graphical ones,' but the amount of memory that gets saved was a bit stunning the first time I saw it.
  • viraptor 7y ago
    That's a good idea to measure the sharing! Fortunately even if every app gets containerised, not all of those libraries will be duplicated. Specifically each app which forks on its own will still preserve sharing. For example 11 firefox processes I'm running now would share the libraries, whether it's running directly or from docker.
    • bnegreve 7y ago
      On linux, none of these libraries are duplicated, they are loaded once, and shared across all processes that need them.
      • barrkel 7y ago
        Even when the processes are loading the libraries from different paths, in different filesystems, in different containers? How does it page in data on demand if the first container that loaded the library is killed and its filesystem unloaded? It's not easy to share libraries across containers, unless they can be built to share a base layer in a stacked union filesystem approach.
        • saagarjha 7y ago
          Why can’t the linker deduplicate libraries as they’re loaded?
          • pas 7y ago
            kernel samepage merging already exists
        • deleted 7y ago
          [deleted]
        • bnegreve 7y ago
          >Even when the processes are loading the libraries from different paths, Well, in my original post (GGGP) I've defined "a library" as "a .so file" so what I can say is that the 872 distinct .so files used on my laptop will be shared among the different processes that use them. If you assume the same library can be duplicated in two different .so files, then 872 is just an upper bound on the number of distinct libraries and further sharing could be done. Eitherway, that is a significant amount of code sharing, which was the original question in this thread.
      • berkut 7y ago
        Only if they're the same version (which using the older built-in package system all packages are likely built against the same version)... If the containerised versions of apps all have different versions of dependencies (quite likely IMO as they'll have the freedom to), there won't be any sharing.