6 ms·

> As for a system devoid of it, I have grand plans, but for now: people do it with Docker every day! That's what Docker is. No, Docker really isn't what you th

> As for a system devoid of it, I have grand plans, but for now: people do it with Docker every day! That's what Docker is. No, Docker really isn't what you think it is. Docker isolates processes, and provides a convenient platform independent way to deliver stand alone applications, complete with software defined networking and file system abstraction, at the expense of shipping the whole tech stack, including everything plus the kitchen sink like full blown web servers. Thus a .DEB package which otherwise would be 10MB ends up being a Docker image often weighing over 100MB. And pray tell what's Docker's solution to software updates? Well, it's periodic builds that force downstream dependencies and users to re-download payloads of well over 100MB a pop. And you advocate this as an alternative to shared libraries? Who in their ri

benreesman 4y ago
I advocate `musl`/etc. and LLVM as an alternative to `fuck.this.so.0.0.4 -> ...`. I switched from Docker to Nix some time ago (because of many of the pain points you mention). But in a world where native applications worked on any kernel >= the one it was built on, you'd rapidly find out that bin-packing the webserver and webcrawler into a finite list of SKUs (which is where all this container mania originated) via namespaces and cgroups are really not problems that most Docker users have, and there are much simpler ways to get two versions of `node.js` looking at two different sets of libraries. Because you can't reliably know what devilry just stomped all over your `/usr/lib/x86_64/...` or whatever, you kind of have to eat that 100MB image if you want software that worked on Tuesday to still work on Friday. It's a sad state of affairs, but This Too Will Pass. Getting the word out that the reason Docker saves your ass on reproducibility is emulation of static linking behind the FSF's back will accelerate that process.
- akiselev 4y ago
  > the reason Docker saves your ass on reproducibility is emulation of static linking behind the FSF's back Can you expand on this? It’s an interesting perspective I haven’t heard before. I get the gist but even without a Dockerfile, a container image is much less opaque and much more deterministic than a binary with a statically linked library where the compiler might inline entire functions and perform optimizations across module boundaries with LTO. It’s not as simple as setting LD_LIBRARY_PATH but it’s not like trying to reverse the information lost during compilation.
  - matheusmoreira 4y ago
    > Can you expand on this? Bundling dependecies is essentially static linking that doesn't violate the GPL. Static linking means all dependencies are compiled into the binary. Modern technology like docker, appimages, snaps, etc. all emulate that by creating a static image with all the dependencies and then dynamically linking to those specific libraries. So now we have a secondary linker and loader on top of the operating system's.
    - benreesman 4y ago
      And it' such a shame because GNU/FSF was like the friggin Rebel Alliance fighting the Empire for the longest time: we'd never be where we are today without that stuff. Those folks were my heroes growing up. But power corrupts, and once you've got everyone linking to your `libc`, the temptation to get some combination of lazy around standards compliance at best, and seduced by what amounts to proprietary vendor API lock-in to push your politics in at least some cases seemingly becomes overwhelming. Add in LLVM kicking your ass from here to FreeBSD with Stallman freaking the fuck out about `gcc`'s plunging relevance and there's understandable but still not acceptable instinct to go further into incompatible-by-design. People on here routinely lose their shit because NVIDIA or whoever makes it a nightmare to interoperate, and I don't like it either, but vendor lock-in is vendor lock-in. `emacs` talks to the LSP servers via JSON for Christ's sake, which only makes it about 100x laggier, but you can see why they did it, because just like Docker even RMS can't convince anyone that What We Went With is "linking". God bless the VSCode people for accepting the performance penalty so that the rest of us can come to the party, they could have muscled LSP out of the way IMO and didn't. So `rust-analyzer` gets to shovel megabytes of JSON around and Docker gets to (figuratively) tarball a bunch of `.so` files so that `ldd` can "dynamically" link them. The thing is, this way lies irrelevance. Even the GNU userland is pretty optional these days. Is Alpine, as I've taken to calling it, "GNU plus Linux"? Is NixOS? What are we putting up with this shit for? Bash?
      - matheusmoreira 4y ago
        The history of free software is the history of the GPL and the power struggles around it. Free software developers neglecting binary interfaces in favor of source code, refusing to support plugins because it enables proprietary applications, new compilers arising to challenge GCC as a result, hardware makers using cryptography to prevent users from running their own code... And now we have this "hybrid linking" thing: static image containing dynamically linked libraries.
- matheusmoreira 4y ago
  > in a world where native applications worked on any kernel >= the one it was built on Isn't that the world we live? Linux kernel has a stable binary interface. Problems almost always arise in user space.
  - benreesman 4y ago
    Ah, I phrased that clumsily. Linux/Linus are fanatical about not breaking userspace (God bless them). Your stuff never runs on a different system because some asshat clobbered your `/usr/lib/...`.
    - matheusmoreira 4y ago
      > Your stuff never runs on a different system because some asshat clobbered your `/usr/lib/...`. Yeah. I've had some success just dropping all that user space stuff and targeting Linux directly with system calls. I boot up Linux straight into my program. Not viable for every use case but it's amazing when the system is small.
      - benreesman 4y ago
        That's friggin rad. Are you at liberty to share a little about the domain that you're targeting with it? You've got to want the size/performance/etc. pretty bad to ditch every library that wants `libc` right?
        matheusmoreira 4y ago
        I'm not using this stuff professionally, it's just my own home lab's virtual machines with little services implemented as freestanding C programs. Not doing anything fancy right now, much of it was just to see if I could do it. I've seen other people commenting here on HN saying they're using the same approach in production so it's defenitely not my invention. Hopefully they'll see this thread and reply. I published some of my work in the form of a liblinux that I use to make system calls: https://github.com/matheusmoreira/liblinux https://github.com/matheusmoreira/liblinux I'm not developing it anymore though because I found out the kernel itself has a superior nolibc library: https://github.com/torvalds/linux/tree/master/tools/include/nolibc https://github.com/torvalds/linux/tree/master/tools/include/... It used to be a single header but it looks like they've recently organized it into a proper project! I wonder if it will become some kind of official kernel library at some point. I asked Greg Kroah-Hartman about this and he mentioned there was once a klibc: https://old.reddit.com/r/linux/comments/fx5e4v/im_greg_kroahhartman_linux_kernel_developer_ama/fn3rpht/ https://old.reddit.com/r/linux/comments/fx5e4v/im_greg_kroah...
diego_sandoval 4y ago
If running a webserver was as easy as ./nginx.bin and you could rest assured that there would be no library errors, then Docker wouldn't be as prevalent as it is. People go to ridiculous lengths just to avoid the shared library mess, and it only makes the argument against shared libraries stronger.

benreesman 4y ago

I advocate `musl`/etc. and LLVM as an alternative to `fuck.this.so.0.0.4 -> ...`. I switched from Docker to Nix some time ago (because of many of the pain points you mention). But in a world where native applications worked on any kernel >= the one it was built on, you'd rapidly find out that bin-packing the webserver and webcrawler into a finite list of SKUs (which is where all this container mania originated) via namespaces and cgroups are really not problems that most Docker users have, and there are much simpler ways to get two versions of `node.js` looking at two different sets of libraries. Because you can't reliably know what devilry just stomped all over your `/usr/lib/x86_64/...` or whatever, you kind of have to eat that 100MB image if you want software that worked on Tuesday to still work on Friday. It's a sad state of affairs, but This Too Will Pass. Getting the word out that the reason Docker saves your ass on reproducibility is emulation of static linking behind the FSF's back will accelerate that process.

akiselev 4y ago
> the reason Docker saves your ass on reproducibility is emulation of static linking behind the FSF's back Can you expand on this? It’s an interesting perspective I haven’t heard before. I get the gist but even without a Dockerfile, a container image is much less opaque and much more deterministic than a binary with a statically linked library where the compiler might inline entire functions and perform optimizations across module boundaries with LTO. It’s not as simple as setting LD_LIBRARY_PATH but it’s not like trying to reverse the information lost during compilation.
- matheusmoreira 4y ago
  > Can you expand on this? Bundling dependecies is essentially static linking that doesn't violate the GPL. Static linking means all dependencies are compiled into the binary. Modern technology like docker, appimages, snaps, etc. all emulate that by creating a static image with all the dependencies and then dynamically linking to those specific libraries. So now we have a secondary linker and loader on top of the operating system's.
  - benreesman 4y ago
    And it' such a shame because GNU/FSF was like the friggin Rebel Alliance fighting the Empire for the longest time: we'd never be where we are today without that stuff. Those folks were my heroes growing up. But power corrupts, and once you've got everyone linking to your `libc`, the temptation to get some combination of lazy around standards compliance at best, and seduced by what amounts to proprietary vendor API lock-in to push your politics in at least some cases seemingly becomes overwhelming. Add in LLVM kicking your ass from here to FreeBSD with Stallman freaking the fuck out about `gcc`'s plunging relevance and there's understandable but still not acceptable instinct to go further into incompatible-by-design. People on here routinely lose their shit because NVIDIA or whoever makes it a nightmare to interoperate, and I don't like it either, but vendor lock-in is vendor lock-in. `emacs` talks to the LSP servers via JSON for Christ's sake, which only makes it about 100x laggier, but you can see why they did it, because just like Docker even RMS can't convince anyone that What We Went With is "linking". God bless the VSCode people for accepting the performance penalty so that the rest of us can come to the party, they could have muscled LSP out of the way IMO and didn't. So `rust-analyzer` gets to shovel megabytes of JSON around and Docker gets to (figuratively) tarball a bunch of `.so` files so that `ldd` can "dynamically" link them. The thing is, this way lies irrelevance. Even the GNU userland is pretty optional these days. Is Alpine, as I've taken to calling it, "GNU plus Linux"? Is NixOS? What are we putting up with this shit for? Bash?
    - matheusmoreira 4y ago
      The history of free software is the history of the GPL and the power struggles around it. Free software developers neglecting binary interfaces in favor of source code, refusing to support plugins because it enables proprietary applications, new compilers arising to challenge GCC as a result, hardware makers using cryptography to prevent users from running their own code... And now we have this "hybrid linking" thing: static image containing dynamically linked libraries.
matheusmoreira 4y ago
> in a world where native applications worked on any kernel >= the one it was built on Isn't that the world we live? Linux kernel has a stable binary interface. Problems almost always arise in user space.
- benreesman 4y ago
  Ah, I phrased that clumsily. Linux/Linus are fanatical about not breaking userspace (God bless them). Your stuff never runs on a different system because some asshat clobbered your `/usr/lib/...`.
  - matheusmoreira 4y ago
    > Your stuff never runs on a different system because some asshat clobbered your `/usr/lib/...`. Yeah. I've had some success just dropping all that user space stuff and targeting Linux directly with system calls. I boot up Linux straight into my program. Not viable for every use case but it's amazing when the system is small.
    - benreesman 4y ago
      That's friggin rad. Are you at liberty to share a little about the domain that you're targeting with it? You've got to want the size/performance/etc. pretty bad to ditch every library that wants `libc` right?
      - matheusmoreira 4y ago
        I'm not using this stuff professionally, it's just my own home lab's virtual machines with little services implemented as freestanding C programs. Not doing anything fancy right now, much of it was just to see if I could do it. I've seen other people commenting here on HN saying they're using the same approach in production so it's defenitely not my invention. Hopefully they'll see this thread and reply. I published some of my work in the form of a liblinux that I use to make system calls: https://github.com/matheusmoreira/liblinux https://github.com/matheusmoreira/liblinux I'm not developing it anymore though because I found out the kernel itself has a superior nolibc library: https://github.com/torvalds/linux/tree/master/tools/include/nolibc https://github.com/torvalds/linux/tree/master/tools/include/... It used to be a single header but it looks like they've recently organized it into a proper project! I wonder if it will become some kind of official kernel library at some point. I asked Greg Kroah-Hartman about this and he mentioned there was once a klibc: https://old.reddit.com/r/linux/comments/fx5e4v/im_greg_kroahhartman_linux_kernel_developer_ama/fn3rpht/ https://old.reddit.com/r/linux/comments/fx5e4v/im_greg_kroah...

diego_sandoval 4y ago

If running a webserver was as easy as ./nginx.bin and you could rest assured that there would be no library errors, then Docker wouldn't be as prevalent as it is. People go to ridiculous lengths just to avoid the shared library mess, and it only makes the argument against shared libraries stronger.