7 ms·

No End to DLL Hell

53 points by bdfh42 16y ago

AndrewDucker 16y ago
Actually, as the article itself points out, there's a very easy way to avoid DLL hell - don't share DLLs with other apps. All of the programs we produce here include all of their DLLs in their own folder. There's nothing shared, so no other program can cause a problem for them.
- mapleoin 16y ago
  Doesn't that defeat the purpose of using DDLs in the first place?
  - cstuder 16y ago
    Two main arguments for using DLLs were saving disk space and saving RAM. When was the last time you ran out of disk space because of too many DLLs? And nowadays, our RAM is bigger than the disks back then, so that shouldn't be a problem either.
    - wzdd 16y ago
      The other main argument is so that bug fixes can be deployed to all affected applications by replacing a shared DLL. The importance of this aspect is also going away now that pretty much everything automatically updates itself via the Internet.
      - ciupicri 16y ago
        Why should I waste bandwidth because DLLs aren't shared anymore?
        roel_v 16y ago
        It's a trade off between using bandwidth (+ the extra hassles for the auto update infrastructure etc., but that is carried by the vendor) and reducing the risk of deployment issues that may not be caught by QA because they are machine-specific. Plus, application dll's are in the magnitude of 100's of kilobytes - I 'waste' more bandwidth with the POP3 polling that my phone does each week than that. I would say that in this context the amount 'wasted' is so small that it can hardly be called a true 'waste'. (I rinse my cup before pouring in new coffee. Is that 'wasted' water? Technically yes, since it goes into the dishwasher at the end of the day anyway and reusing the cup during one day doesn't pose a health risk. That 'waste' won't prevent me from doing so though).
        ciupicri 16y ago
        Of course it's a trade off, I didn't deny that. I see it just like (micro) optimizations: in some cases it's worth doing it, in others it's not. Regarding the size of DLLs, some of them are indeed small, but others have a couple of megabytes, e.g. XUL (from Mozilla Firefox), Qt, GTK+. Multiply this with the number of applications and it can be come an issue if you have a slow Internet connection.
        tomjen3 16y ago
        Because bandwidth is cheap. Programmers and system administers are not. And neither is your time when you have to fix the broken system.
      - viraptor 16y ago
        > pretty much everything automatically updates itself via the Internet But will the vendor care about every update from the system the program is running on? What happens if someone discovers a bug, your system is updated, but vendor a) doesn't care (or doesn't exist anymore) b) linked statically to some library? You're left with no protection against a known vulnerability, even though the patches are available and published.
    - wazoox 16y ago
      This may not seem a problem on the desktop, but RAM still is a major limitation on servers, and don't even talk of virtual machines.
      - JoachimSchipper 16y ago
        Libraries are still shared among multiple instances of the same process, and just how big are libraries anyway? Most servers run only a handful of (very busy) processes...
      - djacobs 16y ago
        That may be true, but RAM is not so limited (even on VMs/servers) that DLLs are worth the extreme hassle. Case in point: Linux (for me) runs much better on a minimal VM than any version of Windows.
        wazoox 16y ago
        Does not compute. Linux too uses dynamic shared libraries, and you can also link programs to a specific version of a .so, or bind it with your binary distribution (see Firefox, Chrome), such binding posing problems to distributions : Debian just dropped chromium from Squeeze exactly because of that.
    - rubyrescue 16y ago
      also from the article: It is worth noting that DLLs no longer save disk space. In fact, the opposite is the case. If you have a look into the side-by-side repository (found in Windows' WinSXS directory), you will find, that its 5 GB on Windows 7 and even 7 GB on Windows Server 2008, containing 40.000 files with numerous versions of roughly all Microsoft Libraries. That's about 50% of the disk space used by Windows in total. We were needing a solution, but we created a monster
      - roel_v 16y ago
        I think this argument is wrong. Those 5GB are of separate versions of each DLL. If program A needs version 1 of a library and programs B and C need version 2, you only have to have 2 files - those of 1 and 2. B and C can share their copy. So they do save space. Now I don't know if it saves as much as the 5GB that the different versions take. This 5 GB is because there are so many versions. You could argue that they shouldn't be all installed, and that applications should install them on-demand, but that would require major deployment issues at vendors (especially those so small that they don't have specialized deployment engineers). I think the trade off they (=Microsoft) made here is quite reasonable (i.e. use more disk space to prevent deployment issues).
        malkia 16y ago
        That 5gb is 20gb on my vista 64 bit machine. I have tons of different 64bit, debug and release dlls. Could be from the fact that I have 3 different versions of studio installed? Anyway...it does not serve it's purpose, and only wastes user money - 20gb cost a lot on a latest SSD drive
        contextfree 16y ago
        It's probably not really 20GB. Most of what's in WinSxS is links to files in other directories, often multiple links to the same file, and the Windows shell "Properties" dialog will include every "copy" when calculating total size.
        16y ago
  - xentronium 16y ago
    You don't always have access to DLL source code. Actually, most of the time you don't.
  - AndrewDucker 16y ago
    We can still use a DLL with multiple applications - they're just shared between the different apps, and live with them. So if I have two apps Foo and Bar that both use PrettyTreeView.DLL, then I just put a copy of it in both projects, and save it in the same repository. Means I can keep the functionality nicely carved off in one place, but I don't have to worry about updating one app and breaking the other. Also, the DLL is only loaded in as needed - which saves on memory.
  - billpg 16y ago
    Keeping things in their own DLLs are handy to update modules without having to update the entire app.
- elblanco 16y ago
  Considering how cheap storage is today (~$65/TB) is there really a reason to use shared DLLs in this day and age on the desktop?
  - qjz 16y ago
    Ideally, when a bugfix or security update is applied to the shared DLL, your application immediately benefits and you won't have to roll out your own update. I would imagine that some 3rd party enhancements (plugins, for example) also rely on shared DLLs.
    - jpr 16y ago
      Aren't dynamic loading (for plugins etc.) and dynamic linking two different things?
      - roel_v 16y ago
        Depends on the exact definitions. 'dynamic loading' (whic I would define as with ::LoadLibrary() ), is a subset of 'dynamic linking' because dynamic linking can be done explicitly (with ::LoadLibrary()) or implicitly by the application's startup code, which is generated/inserted into the executable by the linker. Then again I wouldn't have a problem with someone defining the terms differently so that they are the same or overlap in a different way. I wouldn't say though that they are very different - they are related no matter how you slice it.
  - roel_v 16y ago
    Yes, so that one component can be upgraded and be used by other programs. Recall the zlib bug of a few years ago. Everybody who linked zlib statically had to provide a new build (or include the fix in their next release). There have to be thousands of applications out there who weren't upgraded (old version, vendor out of business, whatever) and who are vulnerable to a years-old security flaw. That said, I ship my own software with all required dll's in one directory, too.
    - 16y ago
- jimbobimbo 16y ago
  Unfortunately, it's a problem which is surprisingly hard to solve for code that uses CRT, MFC or ATL. And it is especially hard, if you link to the code which in turn links to some other version of these libs.
  - malkia 16y ago
    Also if for some reason you would like to deploy debug version along with release. Normally you don't have to, but sometimes to inspect a problem on coworker's machine (say he's an artist, and the dll is some plugin for some editor) and then you end up with installing studio on his machine :(
roel_v 16y ago
He didn't even mention some of the fun tricks that Microsoft pulled with the versioning of the C and MFC runtime dll's. For example in the summer of 2008 with the release of SP1 for Visual Studio 2008, they included a security fix in ATL that required a new runtime. Now all of a sudden all applications build with the Visual Studio that included that SP required different runtime dll's to be deployed. Double the fun when you have a bunch of dll's that require the 'old' version (no SP) and some that require the new one. You can't ship them in the same directory because they have the same name. For anyone trying to debug building problems, see this stack overflow question which shows, judging by the upvotes, that many people were bitten by this (people still upvote the answers there, still getting hit by this, after 2 years): http://stackoverflow.com/questions/59635/app-does-not-run-with-vs-2008-sp1-dlls-previous-version-works-with-rtm-versions/ http://stackoverflow.com/questions/59635/app-does-not-run-wi... .
- dmethvin 16y ago
  So sad, because that could have been the DLL's finest hour. A lot of code uses ATL. Imagine if Microsoft had been able to update a few ATL DLLs and close the hole for every app written to dynamically link to ATL. Instead, the dynalinkers were no better off than the static linkers. No wait, they were worse off.
  - roel_v 16y ago
    To elaborate, the rub here was that ATL heavily relies on templates, code that is instantiated by the client at build time. The fix (IIRC but I don't see any other explanations either) required a change in one of those compiled-at-build-time parts (i.e, in the header), and was of such nature that it changed the ABI. If the part that had the bug had been able to be included in a dll (but it couldn't be because of the way templates work), swapping out the dll would have worked. So arguably it was exactly the static linking (forced by the language) that was at fault here. People who linked dynamically were forced to make some painful changes, but those who linked statically potentially had the security bug left in their programs. I'm not sure who of the two would be worse off here. I guess at Microsoft they considered the security implications severe enough to warrant a patch as disruptive as this one was.
jimbobimbo 16y ago
Ah, lovely manifest problems! It's truly a new hell today for C++ applications.
Symmetry 16y ago
Reading that mostly made me appreciate how much my Linux Distro does for me.
hippich 16y ago
One page - http://www.drdobbs.com/article/printableArticle.jhtml;jsessionid=ZTPGFQCDLF2CFQE1GHPCKH4ATMY32JVN?articleId=227300037&dept_url=/windows/ http://www.drdobbs.com/article/printableArticle.jhtml;jsessi...
hippich 16y ago
After reading this.. It's all about community. If community can agree on API you can create something shareable like DLLs and it will work for all at benefit. If community doesn't exists at all - this is not gonna work and it's better to enclose everything in your app. I believe, community thing, in sense like it exists with open source projects, will not exists for any private software, so when you create something based on MS stack, or something which do not have open sourced codes - stick to "keep everything with app" way. But, of course, I would like to see more opensourced software and propose opening code for the whole stack to avoid this kind of problems =)) Ah.. dreams =)
- roel_v 16y ago
  I don't understand how a 'community' would help here. When the ABI of a dll changes (which it will at every major release, and it's a major pain to keep it compatible even between small releases - I don't know of any projects that even try), you need some way to communicate this change to any applications that use it.
  - djacobs 16y ago
    I definitely get what hippich is saying. The idea is that when you have a community built around a platform, and that community is, well, communal (cooperative in some ways), solutions emerge to avoid conflicts and problems like this. (RubyGems for Ruby, CPAN for Python, aptitude for Debian, RPM, etc., etc.) According to hippich (and I agree), that kind of community and spirit only ever emerges when we are sharing code.
    - hippich 16y ago
      roel_v, just like he said =) (except, CPAN is for Perl =)). Probably, API is incorrectly referenced here. But idea is - if something really shared, community using this something shared will be able to find a way to avoid conflicts.
      - Someone 16y ago
loup-vaillant 16y ago
Microsoft can recompile the relevant programs when the new DLLs aren't binary compatible, then push those new binaries along with the core update… Well… they could if only they had aptitude… and a repository for 3rd party's source code. I guess there's no easy way out, then.
- mikemol 16y ago
  Windows SxS was supposed to ensure that programs would get linked to the latest version of a DLL that the program would be compatible with. There are even conditional forwarders set up for it. You could have a program that would only link against the oldest version, if it needed to. They were very, very close to a working versioned symbols system.
mkramlich 16y ago
Hopfully enough folks have swung around to the Batteries Included approach (bundling dependencies inside the app's disk footprint) that DLL Hell will eventually die out. Plus both disk and memory space has gotten much less dear over the last 30 years, so conserving and treating every byte as sacred is no longer as important as not having one's software break in mysterious and painful ways.

53 points by bdfh42 16y ago

Actually, as the article itself points out, there's a very easy way to avoid DLL hell - don't share DLLs with other apps. All of the programs we produce here include all of their DLLs in their own folder. There's nothing shared, so no other program can cause a problem for them.

mapleoin 16y ago
Doesn't that defeat the purpose of using DDLs in the first place?
- cstuder 16y ago
  Two main arguments for using DLLs were saving disk space and saving RAM. When was the last time you ran out of disk space because of too many DLLs? And nowadays, our RAM is bigger than the disks back then, so that shouldn't be a problem either.
  - wzdd 16y ago
    The other main argument is so that bug fixes can be deployed to all affected applications by replacing a shared DLL. The importance of this aspect is also going away now that pretty much everything automatically updates itself via the Internet.
    - ciupicri 16y ago
      Why should I waste bandwidth because DLLs aren't shared anymore?
      - roel_v 16y ago
        It's a trade off between using bandwidth (+ the extra hassles for the auto update infrastructure etc., but that is carried by the vendor) and reducing the risk of deployment issues that may not be caught by QA because they are machine-specific. Plus, application dll's are in the magnitude of 100's of kilobytes - I 'waste' more bandwidth with the POP3 polling that my phone does each week than that. I would say that in this context the amount 'wasted' is so small that it can hardly be called a true 'waste'. (I rinse my cup before pouring in new coffee. Is that 'wasted' water? Technically yes, since it goes into the dishwasher at the end of the day anyway and reusing the cup during one day doesn't pose a health risk. That 'waste' won't prevent me from doing so though).
        ciupicri 16y ago
        Of course it's a trade off, I didn't deny that. I see it just like (micro) optimizations: in some cases it's worth doing it, in others it's not. Regarding the size of DLLs, some of them are indeed small, but others have a couple of megabytes, e.g. XUL (from Mozilla Firefox), Qt, GTK+. Multiply this with the number of applications and it can be come an issue if you have a slow Internet connection.
      - tomjen3 16y ago
        Because bandwidth is cheap. Programmers and system administers are not. And neither is your time when you have to fix the broken system.
    - viraptor 16y ago
      > pretty much everything automatically updates itself via the Internet But will the vendor care about every update from the system the program is running on? What happens if someone discovers a bug, your system is updated, but vendor a) doesn't care (or doesn't exist anymore) b) linked statically to some library? You're left with no protection against a known vulnerability, even though the patches are available and published.
  - wazoox 16y ago
    This may not seem a problem on the desktop, but RAM still is a major limitation on servers, and don't even talk of virtual machines.
    - JoachimSchipper 16y ago
      Libraries are still shared among multiple instances of the same process, and just how big are libraries anyway? Most servers run only a handful of (very busy) processes...
    - djacobs 16y ago
      That may be true, but RAM is not so limited (even on VMs/servers) that DLLs are worth the extreme hassle. Case in point: Linux (for me) runs much better on a minimal VM than any version of Windows.
      - wazoox 16y ago
        Does not compute. Linux too uses dynamic shared libraries, and you can also link programs to a specific version of a .so, or bind it with your binary distribution (see Firefox, Chrome), such binding posing problems to distributions : Debian just dropped chromium from Squeeze exactly because of that.
  - rubyrescue 16y ago
    also from the article: It is worth noting that DLLs no longer save disk space. In fact, the opposite is the case. If you have a look into the side-by-side repository (found in Windows' WinSXS directory), you will find, that its 5 GB on Windows 7 and even 7 GB on Windows Server 2008, containing 40.000 files with numerous versions of roughly all Microsoft Libraries. That's about 50% of the disk space used by Windows in total. We were needing a solution, but we created a monster
    - roel_v 16y ago
      I think this argument is wrong. Those 5GB are of separate versions of each DLL. If program A needs version 1 of a library and programs B and C need version 2, you only have to have 2 files - those of 1 and 2. B and C can share their copy. So they do save space. Now I don't know if it saves as much as the 5GB that the different versions take. This 5 GB is because there are so many versions. You could argue that they shouldn't be all installed, and that applications should install them on-demand, but that would require major deployment issues at vendors (especially those so small that they don't have specialized deployment engineers). I think the trade off they (=Microsoft) made here is quite reasonable (i.e. use more disk space to prevent deployment issues).
      - malkia 16y ago
        That 5gb is 20gb on my vista 64 bit machine. I have tons of different 64bit, debug and release dlls. Could be from the fact that I have 3 different versions of studio installed? Anyway...it does not serve it's purpose, and only wastes user money - 20gb cost a lot on a latest SSD drive
        contextfree 16y ago
        It's probably not really 20GB. Most of what's in WinSxS is links to files in other directories, often multiple links to the same file, and the Windows shell "Properties" dialog will include every "copy" when calculating total size.
        16y ago
- xentronium 16y ago
  You don't always have access to DLL source code. Actually, most of the time you don't.
- AndrewDucker 16y ago
  We can still use a DLL with multiple applications - they're just shared between the different apps, and live with them. So if I have two apps Foo and Bar that both use PrettyTreeView.DLL, then I just put a copy of it in both projects, and save it in the same repository. Means I can keep the functionality nicely carved off in one place, but I don't have to worry about updating one app and breaking the other. Also, the DLL is only loaded in as needed - which saves on memory.
- billpg 16y ago
  Keeping things in their own DLLs are handy to update modules without having to update the entire app.
elblanco 16y ago
Considering how cheap storage is today (~$65/TB) is there really a reason to use shared DLLs in this day and age on the desktop?
- qjz 16y ago
  Ideally, when a bugfix or security update is applied to the shared DLL, your application immediately benefits and you won't have to roll out your own update. I would imagine that some 3rd party enhancements (plugins, for example) also rely on shared DLLs.
  - jpr 16y ago
    Aren't dynamic loading (for plugins etc.) and dynamic linking two different things?
    - roel_v 16y ago
      Depends on the exact definitions. 'dynamic loading' (whic I would define as with ::LoadLibrary() ), is a subset of 'dynamic linking' because dynamic linking can be done explicitly (with ::LoadLibrary()) or implicitly by the application's startup code, which is generated/inserted into the executable by the linker. Then again I wouldn't have a problem with someone defining the terms differently so that they are the same or overlap in a different way. I wouldn't say though that they are very different - they are related no matter how you slice it.
- roel_v 16y ago
  Yes, so that one component can be upgraded and be used by other programs. Recall the zlib bug of a few years ago. Everybody who linked zlib statically had to provide a new build (or include the fix in their next release). There have to be thousands of applications out there who weren't upgraded (old version, vendor out of business, whatever) and who are vulnerable to a years-old security flaw. That said, I ship my own software with all required dll's in one directory, too.
  - 16y ago
jimbobimbo 16y ago
Unfortunately, it's a problem which is surprisingly hard to solve for code that uses CRT, MFC or ATL. And it is especially hard, if you link to the code which in turn links to some other version of these libs.
- malkia 16y ago
  Also if for some reason you would like to deploy debug version along with release. Normally you don't have to, but sometimes to inspect a problem on coworker's machine (say he's an artist, and the dll is some plugin for some editor) and then you end up with installing studio on his machine :(

roel_v 16y ago

He didn't even mention some of the fun tricks that Microsoft pulled with the versioning of the C and MFC runtime dll's. For example in the summer of 2008 with the release of SP1 for Visual Studio 2008, they included a security fix in ATL that required a new runtime. Now all of a sudden all applications build with the Visual Studio that included that SP required different runtime dll's to be deployed. Double the fun when you have a bunch of dll's that require the 'old' version (no SP) and some that require the new one. You can't ship them in the same directory because they have the same name. For anyone trying to debug building problems, see this stack overflow question which shows, judging by the upvotes, that many people were bitten by this (people still upvote the answers there, still getting hit by this, after 2 years): http://stackoverflow.com/questions/59635/app-does-not-run-with-vs-2008-sp1-dlls-previous-version-works-with-rtm-versions/ http://stackoverflow.com/questions/59635/app-does-not-run-wi... .

dmethvin 16y ago
So sad, because that could have been the DLL's finest hour. A lot of code uses ATL. Imagine if Microsoft had been able to update a few ATL DLLs and close the hole for every app written to dynamically link to ATL. Instead, the dynalinkers were no better off than the static linkers. No wait, they were worse off.
- roel_v 16y ago
  To elaborate, the rub here was that ATL heavily relies on templates, code that is instantiated by the client at build time. The fix (IIRC but I don't see any other explanations either) required a change in one of those compiled-at-build-time parts (i.e, in the header), and was of such nature that it changed the ABI. If the part that had the bug had been able to be included in a dll (but it couldn't be because of the way templates work), swapping out the dll would have worked. So arguably it was exactly the static linking (forced by the language) that was at fault here. People who linked dynamically were forced to make some painful changes, but those who linked statically potentially had the security bug left in their programs. I'm not sure who of the two would be worse off here. I guess at Microsoft they considered the security implications severe enough to warrant a patch as disruptive as this one was.

jimbobimbo 16y ago

Ah, lovely manifest problems! It's truly a new hell today for C++ applications.

Symmetry 16y ago

Reading that mostly made me appreciate how much my Linux Distro does for me.

hippich 16y ago

One page - http://www.drdobbs.com/article/printableArticle.jhtml;jsessionid=ZTPGFQCDLF2CFQE1GHPCKH4ATMY32JVN?articleId=227300037&dept_url=/windows/ http://www.drdobbs.com/article/printableArticle.jhtml;jsessi...

hippich 16y ago

After reading this.. It's all about community. If community can agree on API you can create something shareable like DLLs and it will work for all at benefit. If community doesn't exists at all - this is not gonna work and it's better to enclose everything in your app. I believe, community thing, in sense like it exists with open source projects, will not exists for any private software, so when you create something based on MS stack, or something which do not have open sourced codes - stick to "keep everything with app" way. But, of course, I would like to see more opensourced software and propose opening code for the whole stack to avoid this kind of problems =)) Ah.. dreams =)

roel_v 16y ago
I don't understand how a 'community' would help here. When the ABI of a dll changes (which it will at every major release, and it's a major pain to keep it compatible even between small releases - I don't know of any projects that even try), you need some way to communicate this change to any applications that use it.
- djacobs 16y ago
  I definitely get what hippich is saying. The idea is that when you have a community built around a platform, and that community is, well, communal (cooperative in some ways), solutions emerge to avoid conflicts and problems like this. (RubyGems for Ruby, CPAN for Python, aptitude for Debian, RPM, etc., etc.) According to hippich (and I agree), that kind of community and spirit only ever emerges when we are sharing code.
  - hippich 16y ago
    roel_v, just like he said =) (except, CPAN is for Perl =)). Probably, API is incorrectly referenced here. But idea is - if something really shared, community using this something shared will be able to find a way to avoid conflicts.
    - Someone 16y ago

loup-vaillant 16y ago

Microsoft can recompile the relevant programs when the new DLLs aren't binary compatible, then push those new binaries along with the core update… Well… they could if only they had aptitude… and a repository for 3rd party's source code. I guess there's no easy way out, then.

mikemol 16y ago
Windows SxS was supposed to ensure that programs would get linked to the latest version of a DLL that the program would be compatible with. There are even conditional forwarders set up for it. You could have a program that would only link against the oldest version, if it needed to. They were very, very close to a working versioned symbols system.

mkramlich 16y ago

Hopfully enough folks have swung around to the Batteries Included approach (bundling dependencies inside the app's disk footprint) that DLL Hell will eventually die out. Plus both disk and memory space has gotten much less dear over the last 30 years, so conserving and treating every byte as sacred is no longer as important as not having one's software break in mysterious and painful ways.