3 ms·

Because statically linking everything has several negative consequences: * increased storage space * increased memory usage * increased downtime for upda

Because statically linking everything has several negative consequences: * increased storage space * increased memory usage * increased downtime for updates (since more files have to be updated) * increased bandwidth usage (total size of download for update) * potentially increased security risks In short, trading off all of the above to simply avoid proper release engineering and simplified dependency management is the wrong answer. Our systems need less downtime and better security; not the opposite, which is what static linking brings. That's why Solaris, as an example, does not provide static archives for almost any of the components that are provided, especially libc.

sanbor 11y ago
Thank you for enumerating those downsides. Clearly I'd avoid statically linked software if I would have the option to use `apt-get install` to get that software. But in the situation that a new version of the software is released, and I think it would benefit me right now, I'd trade all those downsides for being able to use that software right now than waiting to be included in my distro (like it happened to me with Gimp some years ago). I'm plenty of memory, storage and bandwidth but not so much of time to compile it by hand. I have to admit that this is just a workaround while we find a better software release and dependency management system.
- binarycrusader 11y ago
  But in the situation that a new version of the software is released, and I think it would benefit me right now, I'd trade all those downsides for being able to use that software right now than waiting to be included in my distro (like it happened to me with Gimp some years ago). I don't understand what static linking vs. dynamic linking has to do with waiting for gimp to be included in your distro? I'm plenty of memory, storage and bandwidth but not so much of time to compile it by hand. No one is suggesting you have to do that, but someone will have to do that, and it does require resources (time, people, hardware, etc.). I have to admit that this is just a workaround while we find a better software release and dependency management system. If by system, you're referring to technology, then I disagree. In other words, the technological constraints of today's package management systems are not the primary issue, rather it is how the software itself is being developed and managed and the available resources to do so. Ultimately though, most of this all comes about because the underlying systems that applications depend upon in a typical Linux distribution are not properly release engineered. They don't properly version the shared libraries, they don't carefully avoid incompatible changes in interfaces, and they don't have sufficient regression testing. So to me, the right answer is to fix the root of the problem, not paper over it by pretending there isn't one by essentially embedding copies of every dependency into an application. In short, start encouraging developers to reduce their dependency chains, properly release manage their software, ensure that core system components offer stable interfaces, and provide timely updates.
gens 11y ago
statically linking to properly(!) made libraries will only maybe increase storage space it will decrease memory usage (as most loaders load the whole library, even when just one function is used) more things to upgrade, yes more bandwidth, yes (not much if one uses binary diffs) potentially increased security risks, yes. although shared libraries are bigger security risks i made an acc just to reply to this. why do people never understand static linking ?
- binarycrusader 11y ago
  statically linking to properly(!) made libraries will only maybe increase storage space it will decrease memory usage (as most loaders load the whole library, even when just one function is used) Most OS linkers already do efficient loading of only the relevant portions of a shared library since they typically mmap the library file. In aggregate, static linking the same dependency for multiple programs will increase memory usage as well despite your assertions to the contrary since the pages will not generally be shared (yes, I'm aware some OS have page dedup or compression, etc., but I'm talking about the general case here). more things to upgrade, yes more bandwidth, yes (not much if one uses binary diffs) A binary diff of 100 programs all linking to the same library will still be larger than the diffs for a single library. In addition, updating 100 binaries, even with diffs still takes longer than uprating a single library, which means increased downtime during system updates. potentially increased security risks, yes. although shared libraries are bigger security risks Code is code; how is shared linking a greater risk? Have any data to back that up? i made an acc just to reply to this. why do people never understand static linking ? I understand static linking quite well, since my day job is part of a commercial OS team that coincidentally pioneered dynamic linking technology in UNIX. So let's not base our arguments on assumptions about knowledge just because we disagree.
  - gens 11y ago
    >Most OS linkers already do efficient loading of only the relevant portions of a shared library since they typically mmap the library file. a page is 4kB (on x86 at least). a function is typically, what, 60 bytes ? 120 maybe ? 240 ? some functions use other functions (fopen()->_open() in libc) so you can have more then two pages maybe you are thinking about swapping (or not even lazy loading in the block layer) ? code has to be in RAM, everything else is insecure >In aggregate, static linking the same dependency for multiple programs will increase memory usage as well despite your assertions to the contrary since the pages will not generally be shared (yes, I'm aware some OS have page dedup or compression, etc., but I'm talking about the general case here). not pages, functions (and/or variable objects). when you make a .la on unices you are making a ar(1) archive out of .o object files. when the linker is linking the final binary it opens that .la archive and pulls out the relevant .o objects so "statically linking to properly(!) made libraries" will reduce overall memory usage, unless a lot of programs use that (small) library (rarely the case) OSX linker does the right thing with their libc. in that out of the many optimized memcpy()'s it will only load the one that is fast on the particular cpu GNU linker loads them all >Code is code; how is shared linking a greater risk? Have any data to back that up? data to back it up ? do you have any data to back up anything you said ? LD_PRELOAD can be used to replace functions and the user/admin would not notice it. while the only security flaw with static linking is an administrative mistake >I understand static linking quite well, since my day job is part of a commercial OS team that coincidentally pioneered dynamic linking technology in UNIX. >So let's not base our arguments on assumptions about knowledge just because we disagree. so go ask them about it
clevernickname 11y ago
A tradeoff that few consider is that dynamic linking can lessen the motivation of library developers to reduce code bloat and complex dependency trees. Who cares about 20 megs here or there if there is only one shared library instance? Who cares about bringing in a tree of 50 dependencies for this console program if the user probably has them installed already? And thus we end up with the modern Linux desktop, where nearly everything depends on a gigabyte or more of dependencies. Whereas with static linking, you can directly see the effect that any library you use has on the binary size. This encourages library and application devs alike to be more judicious with their dependencies.
- binarycrusader 11y ago
  A tradeoff that few consider is that dynamic linking can lessen the motivation of library developers to reduce code bloat and complex dependency trees. Architecting an operating system and its packages as a psychological forcing function to improve software development is not a worthwhile tradeoff for the efficiency, security, or reliability of a system. Who cares about 20 megs here or there if there is only one shared library instance? Who cares about bringing in a tree of 50 dependencies for this console program if the user probably has them installed already? And thus we end up with the modern Linux desktop, where nearly everything depends on a gigabyte or more of dependencies. I find it amusing that someone is arguing for static linking as a way to make it obvious how bloated programs are, since usually the counter-argument to dynamic linking from many is "who cares about disk space!". There are plenty of programs that are distributed essentially statically-linked today, that still have 50 dependencies or more -- there is no data that I am aware of to support the idea that dynamic vs. static linking would encourage developers to reduce their dependencies. Whereas with static linking, you can directly see the effect that any library you use has on the binary size. This encourages library and application devs alike to be more judicious with their dependencies. This is a dubious argument at best; you can still easily see the size of your program today even if it's dynamically linked simply by looking at top, or by using the appropriate developer tools. I can empathize with the frustrations that some feel in dealing with some operating systems and applications, but it is not due to a technological issue -- it is due to cultural and process issues present not only in the development of the software they rely on but in the distributions that provide the software themselves. "papering over" those issues by static linking only makes the problem worse -- not better. This a cultural and process issue that cannot be addressed solely through technological means. Also, I want to be clear that I am not linking against static linking completely; I'm arguing for dynamic linking (or more generally, shared objects) for those dependencies shared between a sufficient number of components that have sufficiently stable interfaces to share them. By all means, if there is a component with a sufficiently unstable interface, then by definition, it is not appropriate for sharing between components, and at that point whether you statically or dynamically-link the dependency is moot, since you can have private dynamically-linked dependencies just as easily statically-linking them into binaries.

sanbor 11y ago

Thank you for enumerating those downsides. Clearly I'd avoid statically linked software if I would have the option to use `apt-get install` to get that software. But in the situation that a new version of the software is released, and I think it would benefit me right now, I'd trade all those downsides for being able to use that software right now than waiting to be included in my distro (like it happened to me with Gimp some years ago). I'm plenty of memory, storage and bandwidth but not so much of time to compile it by hand. I have to admit that this is just a workaround while we find a better software release and dependency management system.

binarycrusader 11y ago
But in the situation that a new version of the software is released, and I think it would benefit me right now, I'd trade all those downsides for being able to use that software right now than waiting to be included in my distro (like it happened to me with Gimp some years ago). I don't understand what static linking vs. dynamic linking has to do with waiting for gimp to be included in your distro? I'm plenty of memory, storage and bandwidth but not so much of time to compile it by hand. No one is suggesting you have to do that, but someone will have to do that, and it does require resources (time, people, hardware, etc.). I have to admit that this is just a workaround while we find a better software release and dependency management system. If by system, you're referring to technology, then I disagree. In other words, the technological constraints of today's package management systems are not the primary issue, rather it is how the software itself is being developed and managed and the available resources to do so. Ultimately though, most of this all comes about because the underlying systems that applications depend upon in a typical Linux distribution are not properly release engineered. They don't properly version the shared libraries, they don't carefully avoid incompatible changes in interfaces, and they don't have sufficient regression testing. So to me, the right answer is to fix the root of the problem, not paper over it by pretending there isn't one by essentially embedding copies of every dependency into an application. In short, start encouraging developers to reduce their dependency chains, properly release manage their software, ensure that core system components offer stable interfaces, and provide timely updates.

gens 11y ago

statically linking to properly(!) made libraries will only maybe increase storage space it will decrease memory usage (as most loaders load the whole library, even when just one function is used) more things to upgrade, yes more bandwidth, yes (not much if one uses binary diffs) potentially increased security risks, yes. although shared libraries are bigger security risks i made an acc just to reply to this. why do people never understand static linking ?

binarycrusader 11y ago
statically linking to properly(!) made libraries will only maybe increase storage space it will decrease memory usage (as most loaders load the whole library, even when just one function is used) Most OS linkers already do efficient loading of only the relevant portions of a shared library since they typically mmap the library file. In aggregate, static linking the same dependency for multiple programs will increase memory usage as well despite your assertions to the contrary since the pages will not generally be shared (yes, I'm aware some OS have page dedup or compression, etc., but I'm talking about the general case here). more things to upgrade, yes more bandwidth, yes (not much if one uses binary diffs) A binary diff of 100 programs all linking to the same library will still be larger than the diffs for a single library. In addition, updating 100 binaries, even with diffs still takes longer than uprating a single library, which means increased downtime during system updates. potentially increased security risks, yes. although shared libraries are bigger security risks Code is code; how is shared linking a greater risk? Have any data to back that up? i made an acc just to reply to this. why do people never understand static linking ? I understand static linking quite well, since my day job is part of a commercial OS team that coincidentally pioneered dynamic linking technology in UNIX. So let's not base our arguments on assumptions about knowledge just because we disagree.
- gens 11y ago
  >Most OS linkers already do efficient loading of only the relevant portions of a shared library since they typically mmap the library file. a page is 4kB (on x86 at least). a function is typically, what, 60 bytes ? 120 maybe ? 240 ? some functions use other functions (fopen()->_open() in libc) so you can have more then two pages maybe you are thinking about swapping (or not even lazy loading in the block layer) ? code has to be in RAM, everything else is insecure >In aggregate, static linking the same dependency for multiple programs will increase memory usage as well despite your assertions to the contrary since the pages will not generally be shared (yes, I'm aware some OS have page dedup or compression, etc., but I'm talking about the general case here). not pages, functions (and/or variable objects). when you make a .la on unices you are making a ar(1) archive out of .o object files. when the linker is linking the final binary it opens that .la archive and pulls out the relevant .o objects so "statically linking to properly(!) made libraries" will reduce overall memory usage, unless a lot of programs use that (small) library (rarely the case) OSX linker does the right thing with their libc. in that out of the many optimized memcpy()'s it will only load the one that is fast on the particular cpu GNU linker loads them all >Code is code; how is shared linking a greater risk? Have any data to back that up? data to back it up ? do you have any data to back up anything you said ? LD_PRELOAD can be used to replace functions and the user/admin would not notice it. while the only security flaw with static linking is an administrative mistake >I understand static linking quite well, since my day job is part of a commercial OS team that coincidentally pioneered dynamic linking technology in UNIX. >So let's not base our arguments on assumptions about knowledge just because we disagree. so go ask them about it

clevernickname 11y ago

A tradeoff that few consider is that dynamic linking can lessen the motivation of library developers to reduce code bloat and complex dependency trees. Who cares about 20 megs here or there if there is only one shared library instance? Who cares about bringing in a tree of 50 dependencies for this console program if the user probably has them installed already? And thus we end up with the modern Linux desktop, where nearly everything depends on a gigabyte or more of dependencies. Whereas with static linking, you can directly see the effect that any library you use has on the binary size. This encourages library and application devs alike to be more judicious with their dependencies.

binarycrusader 11y ago
A tradeoff that few consider is that dynamic linking can lessen the motivation of library developers to reduce code bloat and complex dependency trees. Architecting an operating system and its packages as a psychological forcing function to improve software development is not a worthwhile tradeoff for the efficiency, security, or reliability of a system. Who cares about 20 megs here or there if there is only one shared library instance? Who cares about bringing in a tree of 50 dependencies for this console program if the user probably has them installed already? And thus we end up with the modern Linux desktop, where nearly everything depends on a gigabyte or more of dependencies. I find it amusing that someone is arguing for static linking as a way to make it obvious how bloated programs are, since usually the counter-argument to dynamic linking from many is "who cares about disk space!". There are plenty of programs that are distributed essentially statically-linked today, that still have 50 dependencies or more -- there is no data that I am aware of to support the idea that dynamic vs. static linking would encourage developers to reduce their dependencies. Whereas with static linking, you can directly see the effect that any library you use has on the binary size. This encourages library and application devs alike to be more judicious with their dependencies. This is a dubious argument at best; you can still easily see the size of your program today even if it's dynamically linked simply by looking at top, or by using the appropriate developer tools. I can empathize with the frustrations that some feel in dealing with some operating systems and applications, but it is not due to a technological issue -- it is due to cultural and process issues present not only in the development of the software they rely on but in the distributions that provide the software themselves. "papering over" those issues by static linking only makes the problem worse -- not better. This a cultural and process issue that cannot be addressed solely through technological means. Also, I want to be clear that I am not linking against static linking completely; I'm arguing for dynamic linking (or more generally, shared objects) for those dependencies shared between a sufficient number of components that have sufficiently stable interfaces to share them. By all means, if there is a component with a sufficiently unstable interface, then by definition, it is not appropriate for sharing between components, and at that point whether you statically or dynamically-link the dependency is moot, since you can have private dynamically-linked dependencies just as easily statically-linking them into binaries.