14 ms·
32-bit x86 Position Independent Code – It's That Bad
- bandrami 12y agoThe more I play around with musl (the author's C library) the more I'm convinced dynamic linking was not worth the trouble.
- pilif 12y agoAs a programmer who unfortunately is suckered into doing devops at times, the more I get to just update the openssl library instead of the whole OS whenever a security flaw in openssl is announced, the more I'm convinced that dynamic linking was very well worth the trouble. Funny how perceptions change depending on the angle you're looking at a problem.
- edgan 12y agoYou are right. Lack of it will be the next wave of security problems
- bandrami 12y agoOddly enough I was thinking about openssl updates specifically. Did you remember to restart all your long-running clients? (I'm a sysadmin who occasionally gets roped into doing development...) Dynamic linking makes security more difficult to reason about, which is a Bad Thing. It has many pluses, but also many minuses. And with symbol versioning it gets very problematic to actually figure out what your real code path is to begin with.
- yuriks 12y agoAnd when you update all your clients with static linked OpenSSL, did you also remember to restart every single one of them? These seem like equivalent problems to me.
- deathanatos 12y agoNot quite, I think. With a .so, I can ask `lsof` which services on the machine require restart: the ones that haven't are linked to a deleted so. And you wouldn't need to restart the clients with static OpenSSL — you need to recompile them. And with static linking, I'm not sure how you would easily determine the linked version out to say, the minor or the micro. (Perhaps, if this is your OS's thing like nix, the package manager keeps track…)
- danieldk 12y agoTip for Debian/Ubuntu users: checkrestart from the debian-goodies package is a nice wrapper around lsof that lists running binaries that rely on outdated solibs: http://manpages.debian.org/cgi-bin/man.cgi?query=checkrestart http://manpages.debian.org/cgi-bin/man.cgi?query=checkrestar... https://gehrcke.de/2014/06/good-to-know-checkrestart-from-debian-goodies/ https://gehrcke.de/2014/06/good-to-know-checkrestart-from-de... It not only shows processes that run older solibs, but in the case of services, it will also give you the commands to restart them :).
- tankenmate 12y agoIt also shows programs that are still using the old binary that has been deleted (unlinked from the filesystem) but not refcount 0 freed. This can also be used to uncover some forms of hiding that hack attempts use.
- pmahoney 12y agoI don't think that nix would be able to tell you which version of OpenSSL some binary linked statically, as there would no longer be a reference from that binary to /nix/store/xxx-openssl-1.0/lib/libssl.so. Nix generally uses dynamic linking, though the links are to absolute paths to a specific library version, so upgrading OpenSSL requires a recompile much like with static linking. With "normal" dynamic linking, upgrading OpenSSL and restarting some service means you're now running untested code, hoping that the new version of the dynamic lib really doesn't change an interface on which your code depends. But to be fair, this is usually a good assumption.
- nimrody 12y agoShould be relatively easy to make a tool that uses `losf` or similar to get a list of all running processes that depend on a shared library.
- _pmf_ 12y agoIt seems you are being downvoted by Go monkeys who drank the static linkage kool-aid.
- zzzcpan 12y agoThere are no problems with updates anymore. You don't have to download everything to update a bunch of static binaries. There are things, like bsdiff, that allow you to download and patch only tiny differences related to a security flaw. There is no need for dynamic linking for updates. Really.
- _yosefk 12y agoWhat if you have binaries compiled from proprietary code bases from umpteen vendors using different compilers and compiler options and what-not, and they all use the Internet for self-updating and stuff and they all need to be patched? How are you going to do that? Also - weakly related - what if you want to use a plugin from vendor A inside the program of vendor B, and they live in the same address space - how are you doing it without dynamic loading?
- zzzcpan 12y agoI don't see a problem with proprietary code bases.
- wtbob 12y ago> What if you have binaries compiled from proprietary code bases… Then don't do that. Seriously, RMS, ESR and others have written a plethora of well-reasoned essays indicating why proprietary software is a poor choice.
- GFK_of_xmaspast 12y agoESR advocating for something is reason enough to seriously consider the alternative.
- eropple 12y agoSo you're admitting that this isn't a solution and you need to change the rules of the game to make it work. Well, I'm convinced.
- 12y ago
- derekp7 12y agoI personally would like to see a distribution have a mixed policy -- dynamic linking against core libraries that are part of the default / basic OS install, and static linking against low usage libraries. That way, you keep the "update one library package to fix a bunch of programs", yet you also reduce cross dependency issues by limiting the number of library packages a given program is dependant on.
- bandrami 12y agoIIRC Snowflake[0] does that (but also has a very experimental per-process view of /usr -- pretty cool stuff, really, but takes some getting used to). [0]: https://github.com/GregorR/snowflake https://github.com/GregorR/snowflake
- deweerdt 12y agoOne of the main reasons for dynamic linking has become irrelevant, I believe: the availability of disk and memory space has grown faster than the size of the binary objects.
- pilif 12y agoIMHO, static linking, be it using a mechanism alike what npm is doing or be it plain-old static linking of binaries also means "you link it, you own it". For every package, you link statically, you as the parent package owner become responsible for all security flaws of all the packages you link statically. And by "responsible" I mean: Every dependent security announcement of a dependent package also becomes your security announcement. If you're AWESOMY 1.1 and you link statically against openssl and openssl announces a security flaw, then you better and quickly release AWESOMY 1.1.1 with an accompanying security announcement too. Are you willing to do this? Do you trust the chain of dependencies all the way down to also be willing to do the same? As a responsible developer, I'd much rather delegate that responsibility away to a packager or even the user, especially with well-known libraries like openssl. If I'm owning AWESOMY 1.1 and link dynamically against openssl, then I don't have to do anything when openssl releases a security announcement. I can, if I want to, inform my users to maybe update openssl, but with some likelihood they are already doing this anyways for some other package. For me as a developer, this is considerably more convenient. Yes. Static linking has huge advantages for me as a developer too, but it also comes with a great many additional responsibilities I'm personally not willing to take on, also because I don't trust my dependencies to be as diligent about their dependencies.
- deweerdt 12y agoI believe that what you describe is the remaining argument in favor of dynamic linking. That said, I'd like to point out that it is convenient for the developer - since customers are ultimately interested to know if your software is vulnerable, and you'll have to explain how the vulnerability affects it -. It's also a double edged sword: openssl has a good ascending compatibility record, but that can't be said about all user space libraries. IOW, it's tricky to guarantee that your software will work flawlessly across all the incarnations of CentOS 6.x, if you have a lot of external dependencies, for example.
- q3k 12y agoThere is however, an important security benefit from PIC: ASLR. It's not a silver bullet, but can stop a range of vulnerabilities from being reliably exploitable usable unless an memory leak is also available. Even if you so far as control the instruction pointer as an attacker, you might just not know where to jump to in the target address space...
- justincormack 12y agoOpenBSD now has ASLR for static executables. There was some discussion of it on #musl so it might happen.
- hyc_symas 12y agoASLR is a pretty hackish solution to the main problem, which is that the function call stack and parameter stack are combined. The problems that ASLR mitigates don't exist in languages like Forth, where there are two separate stacks - a call stack and a separate parameter stack.
- StillBored 12y agoThis is really an ABI issue, more than a language one. I'm not sure anything in C requires a combined stack. Plus, processor arches like ARM and POWER use link registers rather than a stack to store the return address. Its the ABI that then chooses to put the LR on the parameter stack. Even on x86 SP could be dedicated as a function stack and separated from BP (or pick another x86_64 register) as the parameter stack. Of course being an ABI issue, probably makes it really hard to solve, and hence hacky solutions abound. (actually its probably pretty easy to solve given LLVM, just impossible to get anyone to accept).
- hyc_symas 12y agoHm, you may be right that nothing in C requires it. But things like POSIX pthreads do - e.g. pthread_attr_setstack() only lets you set a single stack address and size. So yes, you'd have to create a new runtime from the ground up, not just a new ABI for an existing API.
- simias 12y agoCan you give more details? Isn't this particular issue pretty much x86 specific anyway? I think it's not a problem anymore on x86_64.
- atesti 12y agoI always wondered why Linux can't just use the same thing as windows for DLLs: Instead of PIC, a big tables of places to patch with the final position. It looks really terrible to lose one precious register and have all this PIC overhead. I don't use mono's ahead of time compilation because it also creates PIC. The JIT has one register more available. But I haven't measured it yet
- JoshTriplett 12y ago> Instead of PIC, a big tables of places to patch with the final position. Among other things, because that means the text section has to be writeable, and can't have a shared mapping across processes unless it's mapped in the same place in every process, which breaks address-space layout randomization (ASLR).
- atesti 12y agoOn Windows, one can decide a default position for each DLL and if one is lucky, there is space available in that area of the virtual address space. Otherwise the loader needs to do the relocation work and modify pages and make them writeable temporarily. If multiple processes load the same DLL and the default location happens to be chosen well and there is virtual address space in every process, then the linker can simply share the DLL. That's how I think it was before ASLR came around. I think Windows removes the writeable-flag again after relocating. For the area I described (Mono's JIT vs. Mono's ahead of time compilation as PIC .so-file) I personally don't care that much for ASLR, because it's a managed language and because of specific circumstances of that project.
- JoshTriplett 12y agoLinux had a similar mechanism, "prelink", which also completely broke with ASLR.
- userbinator 12y agoWindows has ASLR too (since Vista) and it works with existing DLLs. Only pages which contain relocations can't be shared, which in practice means only code that accesses global data (including calls through the import table to other modules.) All the other pages can be. The whole process image is writable when the loader is running - else how could it load the code into memory? :-) Section attributes are applied after loading.
- ndesaulniers 12y agoFor a JIT I'm working on, I've been trying to call into libc from the emitted code, and I'm running into this.
- dottrap 12y agoSo what's the story on 64-bit x64 or other processors?
- pmalynin 12y agoPosition independent code on x64 is quite trivial, it has a new instruction addressing mode called RIP-relative addressing so that the program can access everything relative to where it is currently executing.
- michaelg73 12y agoThere are some pretty pictures here which help to explain how x86_64 handles relocations in a much less cumbersome way: http://www.mindfruit.co.uk/2012/06/relocations-relocations.html http://www.mindfruit.co.uk/2012/06/relocations-relocations.h...
- gwu78 12y agoThere was a time when shared libraries were a necessity. Not to mention supporting myriad architectures. That was a different era. Conditions have changed. Of course, the designs and implementations have not. They are still in heavy use, 20 years later. What "just works" is never questioned. I am glad to see some people questioning the old assumptions. Unrelated question: Why does Linux pass arguments in registers instead of using the stack?
- deafeningblow 12y agoWhat makes you say conditions have changed? Why should I have to wait for everything that consumes OpenSSL downstream to be updated/recompiled to fully patch my system?
- icebraining 12y agoPlaying Devil's advocate, why do you need to wait for others to recompile the software?
- gwu78 12y agoAre you suggesting that the original purpose of shared libraries relates to patching software? Others seem to be suggesting that the emergence of dynamic linking was a response to general limitations in secondary storage and memory. Limitations that no longer exist.
- icebraining 12y agoDoes it have to be the original purpose to be a useful feature?
- gwu78 12y agoIs this supposed to be an answer? I like marssaxman's comment. I'm not sure I would call this a "feature" because I think of "features" as being intentional and if marssaxman is correct, this justification for shared library use was an accidental side effect. But I have limited knowledge of the history behind dynamic linking. Hence my questions. I am just curious.
- mappu 12y ago"I know on darwin, we kinda hate those sorts of relocations because they are a performance sap. This type of performance sap is nasty as it is pervasive and invisible and hard to ever get back." -- Mike Stump (gcc-patches ML, 2012) I think the Solaris linker took some interesting approach with shared libraries at runtime as well.
- amelius 12y ago> Obviously what we'd like to see is: "foo: jmp bar" Shouldn't that be "foo: call bar"?
- JasuM 12y agoI think it's JMP due to tail-call optimization. If there's a JMP, the RET of the inner function (bar) returns from the outer one (foo) also, and one unnecessary jump is eliminated.
- bla2 12y agoIs 32-bit Linux still worth worrying about? It seems that 64-bit CPUs and kernels have been around for a really long time now. (Honest question.)
- creshal 12y agoIntel is still selling (well, trying to…) 32-bit only Atom/Quark CPUs for smartphones and IoT devices. Coincidentally, Android defaults to PIC in newer SDK versions. So there is still a pressure to optimize for IA32+PIC.
- davidgerard 12y agoWe still have embedded 8-bit. Expect embedded 32-bit approximately forever.
- ori_b 12y agoEmbedded 32 bit x86, with the SysV ABI and support for register calling conventions, though? This is specific to i386, as far as I'm aware. And even then, the worst parts of it are only specific to i386 using the same SysV calling conventions -- ARM has PC relative addressing, as do most other processors commonly used in embedded systems. If you have PC-relative addressing, the entire problem goes away.
- wtallis 12y agoYeah, but we all know the embedded 32-bit stuff that's going to be around forever is ARM, not x86.
- creshal 12y agoWe hope it's not going to be x86… I dunno. I still see too many embedded 486/686 clones around for my liking, and Intel is now trying to get into the whole IoT business with x86-32 Quark CPUs.
- SamReidHughes 12y agoSay you want to build flying robots 1/64 of an inch in diameter. Do you want 64-bit CPUs?