6 ms·

Shocked that it takes a fringe project to be promoting static linking and cleaning up the filesystem. Would love to hear the linux grandfathers chime in...

by qume 10y ago

copperx 10y ago
Static linking is a virtue now? I don't understand the advantages.
- qznc 10y ago
  Google Go and Rust promote it. However, deploying web services is different than Linux distro.
  - rkangel 10y ago
    I don't know about Go, but Rust doesn't promote it. Rust compiles a whole binary at once (for full program optimisation), but that binary can be a dynamic library or executable. How you then build the operating system on top of that is up to you.
    - qznc 10y ago
      Recently, ripgrep was on the frontpage. It says "Linux binaries are static executables" [0]. If I run ldd on cargo, it spits out the usually glibc dependencies, but Rust libraries are statically linked. It seems to be the default behavior of cargo? I would describe that as "promote static linking". Personally, I don't judge this as good or bad. There is no simple answer. Static linking has clear disadvantage wrt security patches. On the other hand, it makes little sense to dynamically link tiny libraries, e.g. a queue data structure. [0] https://github.com/BurntSushi/ripgrep https://github.com/BurntSushi/ripgrep
      - burntsushi 10y ago
        > It says "Linux binaries are static executables" [0]. They are. Running `ldd` on cargo doesn't confirm or deny this. You need to run `ldd` on the binary distributed: $ curl -sLO 'https://github.com/BurntSushi/ripgrep/releases/download/0.2.1/ripgrep-0.2.1-x86_64-unknown-linux-musl.tar.gz' $ tar xf ripgrep-0.2.1-x86_64-unknown-linux-musl.tar.gz $ ldd ./ripgrep-0.2.1-x86_64-unknown-linux-musl/rg not a dynamic executable This is not the default output on Linux. You need to compile with musl instead of glibc: https://news.ycombinator.com/item?id=12565268 https://news.ycombinator.com/item?id=12565268
      - steveklabnik 10y ago
        To elaborate on what burntsushi said: By default, Rust statically links all Rust code. However, glibc isn't usually statically linked, so Rust doesn't either. You can use musl instead of glibc to get 100% statically linked binaries.
  - JdeBP 10y ago
    The problem here is that the choice is one extreme or the other, never the happy medium. The extremes are statically linked everything and dynamically linked everything. Bryan Cantrill deplores this part of the design of Go. He decries the fact that rather than sitting on top of a HLL-function-call binary interface, with the portability layer being HLL function calls like read(), write(), close(), socket(), execve(), Go produces executables that hardwire one specific instruction set's kernel system call traps. See https://news.ycombinator.com/item?id=11392119 https://news.ycombinator.com/item?id=11392119 for his own words. GCC allows one to selectively statically link various libraries in an otherwise dynamically linked executable. (clang had yet to achieve this functionality, last that I checked a year or so ago.)
    - justin66 10y ago
      Bryan would have to elaborate an awful lot on the evilness of using system calls to make that a more general argument. (we're not supposed to use the kernel now for some reason?) It looks like it was just kind of a pain in the ass for the Joyent guys because of what they were doing specifically, which was pretty specialized. (there's a mean and almost clinically insane blog post about Go by one of the other Joyent guys, who retired a week or two after writing it, which also doesn't shed much light but is funnier. You've got to wonder what their deal is with the Go guys, exactly...)
      - 4ad 10y ago
        Yeah, on Linux, the system call interface is the supported interface for interacting with the kernel. Of course we are going to use it since 1) it's supported, 2) it's stable, more stable than e.g. glibc has ever been, not to mention we're not locked to a particular libc vendor, it just works on embedded systems which don't use glibc, 3) it allows us to make system calls without going through all the cgo machinery and without having to switch stacks. On Solaris, the system call interface is not a supported public interface, so I made the Solaris port use libc through cgo. Solaris made the decision that the supported public interface is through libc. That's a fine decision to make if you are the sole vendor of libc. But, that doesn't mean there's anything wrong with different operating systems making different decisions. On Linux we use the Linux rules, and on Solaris we use the Solaris rules. I will tell you this though, having done numerous Go ports and having written many Go compiler targets several times, it's much, much simpler to port Go if your target is something that allows static executables and making system calls. Having done linux/arm64 and solaris/sparc64, I know exactly how much harder is one versus the other.
- icebraining 10y ago
  https://twitter.com/id_aa_carmack/status/26555760682 https://twitter.com/id_aa_carmack/status/26555760682
- notalaser 10y ago
  A lot -- maybe not a majority, but a sizeable number of people -- think dynamic linking is potentially exploitable due to its complexity, has real-life deployment problems (e.g. due to applications depending on a specific version of a library, or simply due to complacency in building and distribution) and that the advantages it offered twenty years ago are offset by larger hard drives, better and faster network connectivity, and better updating systems. In other words, that its cost is no longer as easy to justify as it was back in the 1990s. I have not studied the problem enough to be able to comment on it in its entirety, but there is at least some merit to a few of these claims: * Static linking was a huge nuisance back when Unices introduced dynamic linking because keeping a system up to date was a very different affair. There was no apt-get update, apt-get upgrade. The OS vendor (often the hardware manufacturer) would keep his system up to date. For third-party software, you often depended on building programs from source (oh, yes: no autotools/cmake/whatever to deal with junk Makefiles, either, although this was partly offset by the fact that a lot of developers still knew how to write a makefile). An update in a single library could mean a bunch of manual rebuilds, sometimes followed by manual deployment. This is no longer the case, really. It is a little schizophrenic that we do daily builds for continuous integration, but insist that there's no frickin' way we're going to be able to rebuild packages that depend on a library and distribute them on time. The problem is certainly tractable, albeit at higher resource expense (imagine what an update in glibc would entail). * On the other hand, it's not like dynamic libraries have fulfilled the promise of never using an out-of-date version ever again. It's not at all uncommon for programs to bundle their own version of shared libraries. A while back, when I first came across material discussing this problem, it turned out that a lot of programs of my system did it -- OpenOffice is the one I most distinctly remember, but there were others, too. As for other operating systems where package managers are less common (cough Windows cough), this situation is pretty much the norm when it comes to any library that Windows Update doesn't take care of. * The linking process is extremely complex, and it has been found to be vulnerable. The vulnerabilities were patched, however. There is always a degree of uncertainty in affirming that vulnerability is inherent to complexity. Plus, if it is, then we have a lot of really bigger things to worry about, like that huge pile of code in the kernel which is orders of magnitude more complex than a dynamic loader. Edit: I guess the best way to sum up my (current) understanding of the matter is that the case for static linking isn't as weak as it was a long time ago, but I don't think the case against dynamic linking is spectacular enough to be worth a full migration of everything. That its usefulness is diminishing, at least in some fields, is sufficiently proven by e.g. its adoption in Go. But I doubt that going back to static linking is the universal solution that it is sometimes advertised to be.
- swift 10y ago
  One advantage is that it improves the compiler's ability to optimize. Obviously that matters for some applications more than others.
veeti 10y ago
> cleaning up the filesystem. 4 years ago: https://fedoraproject.org/wiki/Features/UsrMove https://fedoraproject.org/wiki/Features/UsrMove
qwertyuiop924 10y ago
Arch already pretty much does the same thing this does. Of course, Arch defaults to using systemd, although it's easy to change...
JdeBP 10y ago
You should rather be shocked that the Linux world still believes that it is anywhere near this century when it comes to modernizing the filesystem hierarchy. * NeXTSTEP had ~/Apps, /LocalLibrary, /LocalApps, and so forth back in the early 1990s. * The "/usr merge" first happened in AT&T Unix System 5 Release 4. SunOS 5 (a.k.a. Solaris 2) introduced it a few years after NeXTSTEP introduced its directory hierarchy. AIX gained it with version 3.2 in 1992. * Daniel J. Bernstein's /package hierarchy (for package management without the need for conflict resolution) has been around since the turn of the century. Their major problem was the idea that one had to register things with what was to most of the world just some bloke on another continent. But they had concepts like a hierarchical package naming scheme ("admin/", "mail/", &c.), self-contained build trees with versioned names (allowing for side-by-side installation of multiple versions of a package amongst other things), and an "index" directory full of symbolic links. * * http://cr.yp.to/slashpackage.html http://cr.yp.to/slashpackage.html * * http://cr.yp.to/slashcommand.html http://cr.yp.to/slashcommand.html Previous Hacker News discussions with more: https://news.ycombinator.com/item?id=10356933 https://news.ycombinator.com/item?id=10356933 https://news.ycombinator.com/item?id=11647304 https://news.ycombinator.com/item?id=11647304
- ufo 10y ago
  Gobolinux is another Linux distro that ditches the FHS: http://gobolinux.org/ http://gobolinux.org/

Shocked that it takes a fringe project to be promoting static linking and cleaning up the filesystem. Would love to hear the linux grandfathers chime in...

by qume 10y ago

Shocked that it takes a fringe project to be promoting static linking and cleaning up the filesystem. Would love to hear the linux grandfathers chime in...

copperx 10y ago

Static linking is a virtue now? I don't understand the advantages.

qznc 10y ago
Google Go and Rust promote it. However, deploying web services is different than Linux distro.
- rkangel 10y ago
  I don't know about Go, but Rust doesn't promote it. Rust compiles a whole binary at once (for full program optimisation), but that binary can be a dynamic library or executable. How you then build the operating system on top of that is up to you.
  - qznc 10y ago
    Recently, ripgrep was on the frontpage. It says "Linux binaries are static executables" [0]. If I run ldd on cargo, it spits out the usually glibc dependencies, but Rust libraries are statically linked. It seems to be the default behavior of cargo? I would describe that as "promote static linking". Personally, I don't judge this as good or bad. There is no simple answer. Static linking has clear disadvantage wrt security patches. On the other hand, it makes little sense to dynamically link tiny libraries, e.g. a queue data structure. [0] https://github.com/BurntSushi/ripgrep https://github.com/BurntSushi/ripgrep
    - burntsushi 10y ago
      > It says "Linux binaries are static executables" [0]. They are. Running `ldd` on cargo doesn't confirm or deny this. You need to run `ldd` on the binary distributed: $ curl -sLO 'https://github.com/BurntSushi/ripgrep/releases/download/0.2.1/ripgrep-0.2.1-x86_64-unknown-linux-musl.tar.gz' $ tar xf ripgrep-0.2.1-x86_64-unknown-linux-musl.tar.gz $ ldd ./ripgrep-0.2.1-x86_64-unknown-linux-musl/rg not a dynamic executable This is not the default output on Linux. You need to compile with musl instead of glibc: https://news.ycombinator.com/item?id=12565268 https://news.ycombinator.com/item?id=12565268
    - steveklabnik 10y ago
      To elaborate on what burntsushi said: By default, Rust statically links all Rust code. However, glibc isn't usually statically linked, so Rust doesn't either. You can use musl instead of glibc to get 100% statically linked binaries.
- JdeBP 10y ago
  The problem here is that the choice is one extreme or the other, never the happy medium. The extremes are statically linked everything and dynamically linked everything. Bryan Cantrill deplores this part of the design of Go. He decries the fact that rather than sitting on top of a HLL-function-call binary interface, with the portability layer being HLL function calls like read(), write(), close(), socket(), execve(), Go produces executables that hardwire one specific instruction set's kernel system call traps. See https://news.ycombinator.com/item?id=11392119 https://news.ycombinator.com/item?id=11392119 for his own words. GCC allows one to selectively statically link various libraries in an otherwise dynamically linked executable. (clang had yet to achieve this functionality, last that I checked a year or so ago.)
  - justin66 10y ago
    Bryan would have to elaborate an awful lot on the evilness of using system calls to make that a more general argument. (we're not supposed to use the kernel now for some reason?) It looks like it was just kind of a pain in the ass for the Joyent guys because of what they were doing specifically, which was pretty specialized. (there's a mean and almost clinically insane blog post about Go by one of the other Joyent guys, who retired a week or two after writing it, which also doesn't shed much light but is funnier. You've got to wonder what their deal is with the Go guys, exactly...)
    - 4ad 10y ago
      Yeah, on Linux, the system call interface is the supported interface for interacting with the kernel. Of course we are going to use it since 1) it's supported, 2) it's stable, more stable than e.g. glibc has ever been, not to mention we're not locked to a particular libc vendor, it just works on embedded systems which don't use glibc, 3) it allows us to make system calls without going through all the cgo machinery and without having to switch stacks. On Solaris, the system call interface is not a supported public interface, so I made the Solaris port use libc through cgo. Solaris made the decision that the supported public interface is through libc. That's a fine decision to make if you are the sole vendor of libc. But, that doesn't mean there's anything wrong with different operating systems making different decisions. On Linux we use the Linux rules, and on Solaris we use the Solaris rules. I will tell you this though, having done numerous Go ports and having written many Go compiler targets several times, it's much, much simpler to port Go if your target is something that allows static executables and making system calls. Having done linux/arm64 and solaris/sparc64, I know exactly how much harder is one versus the other.
icebraining 10y ago
https://twitter.com/id_aa_carmack/status/26555760682 https://twitter.com/id_aa_carmack/status/26555760682
notalaser 10y ago
A lot -- maybe not a majority, but a sizeable number of people -- think dynamic linking is potentially exploitable due to its complexity, has real-life deployment problems (e.g. due to applications depending on a specific version of a library, or simply due to complacency in building and distribution) and that the advantages it offered twenty years ago are offset by larger hard drives, better and faster network connectivity, and better updating systems. In other words, that its cost is no longer as easy to justify as it was back in the 1990s. I have not studied the problem enough to be able to comment on it in its entirety, but there is at least some merit to a few of these claims: * Static linking was a huge nuisance back when Unices introduced dynamic linking because keeping a system up to date was a very different affair. There was no apt-get update, apt-get upgrade. The OS vendor (often the hardware manufacturer) would keep his system up to date. For third-party software, you often depended on building programs from source (oh, yes: no autotools/cmake/whatever to deal with junk Makefiles, either, although this was partly offset by the fact that a lot of developers still knew how to write a makefile). An update in a single library could mean a bunch of manual rebuilds, sometimes followed by manual deployment. This is no longer the case, really. It is a little schizophrenic that we do daily builds for continuous integration, but insist that there's no frickin' way we're going to be able to rebuild packages that depend on a library and distribute them on time. The problem is certainly tractable, albeit at higher resource expense (imagine what an update in glibc would entail). * On the other hand, it's not like dynamic libraries have fulfilled the promise of never using an out-of-date version ever again. It's not at all uncommon for programs to bundle their own version of shared libraries. A while back, when I first came across material discussing this problem, it turned out that a lot of programs of my system did it -- OpenOffice is the one I most distinctly remember, but there were others, too. As for other operating systems where package managers are less common (cough Windows cough), this situation is pretty much the norm when it comes to any library that Windows Update doesn't take care of. * The linking process is extremely complex, and it has been found to be vulnerable. The vulnerabilities were patched, however. There is always a degree of uncertainty in affirming that vulnerability is inherent to complexity. Plus, if it is, then we have a lot of really bigger things to worry about, like that huge pile of code in the kernel which is orders of magnitude more complex than a dynamic loader. Edit: I guess the best way to sum up my (current) understanding of the matter is that the case for static linking isn't as weak as it was a long time ago, but I don't think the case against dynamic linking is spectacular enough to be worth a full migration of everything. That its usefulness is diminishing, at least in some fields, is sufficiently proven by e.g. its adoption in Go. But I doubt that going back to static linking is the universal solution that it is sometimes advertised to be.
swift 10y ago
One advantage is that it improves the compiler's ability to optimize. Obviously that matters for some applications more than others.

veeti 10y ago

> cleaning up the filesystem. 4 years ago: https://fedoraproject.org/wiki/Features/UsrMove https://fedoraproject.org/wiki/Features/UsrMove

qwertyuiop924 10y ago

Arch already pretty much does the same thing this does. Of course, Arch defaults to using systemd, although it's easy to change...

JdeBP 10y ago

You should rather be shocked that the Linux world still believes that it is anywhere near this century when it comes to modernizing the filesystem hierarchy. * NeXTSTEP had ~/Apps, /LocalLibrary, /LocalApps, and so forth back in the early 1990s. * The "/usr merge" first happened in AT&T Unix System 5 Release 4. SunOS 5 (a.k.a. Solaris 2) introduced it a few years after NeXTSTEP introduced its directory hierarchy. AIX gained it with version 3.2 in 1992. * Daniel J. Bernstein's /package hierarchy (for package management without the need for conflict resolution) has been around since the turn of the century. Their major problem was the idea that one had to register things with what was to most of the world just some bloke on another continent. But they had concepts like a hierarchical package naming scheme ("admin/", "mail/", &c.), self-contained build trees with versioned names (allowing for side-by-side installation of multiple versions of a package amongst other things), and an "index" directory full of symbolic links. * * http://cr.yp.to/slashpackage.html http://cr.yp.to/slashpackage.html * * http://cr.yp.to/slashcommand.html http://cr.yp.to/slashcommand.html Previous Hacker News discussions with more: https://news.ycombinator.com/item?id=10356933 https://news.ycombinator.com/item?id=10356933 https://news.ycombinator.com/item?id=11647304 https://news.ycombinator.com/item?id=11647304

ufo 10y ago
Gobolinux is another Linux distro that ditches the FHS: http://gobolinux.org/ http://gobolinux.org/