5 ms·

Whack: Compile and run path-independent Linux programs

35 points by mwilliamson 13y ago

X4 13y ago
Isn't that just statically linking binaries? Couldn't I do that manually too very easily? Serious question. Otherwise, if it's really relocating binaries, then it's awesomse, BUT I definately want to know MORE about the 'caching' part. That comes short in the docs.
- mwilliamson 13y ago
  In my experience, a lot of Linux programs aren't straightforward to compile statically. Having said that, that might just be my ineptitude! Which bit of the caching isn't clear? If you're happy that whack creates relocatable versions, then the caching works just by copying the output directory into ~/.cache, and then copying that directory to the target on subsequent installations. The exact details of how programs are made relocatable are under "How does Whack work?" in the README.
  - X4 13y ago
    Thank you, that answers my question No you're right, not all binaries can be statically linked easily, it's very tedious in these cases. Oh, no I hoped that something performance related is happening when you said cache, like tuning the binary, predictive read-ahead streaming of the binary to memory, or something like that. I didn't know that it's a copy operation.
  - justincormack 13y ago
    Some programs rely on dynamic libraries for modules so you can't, but the main issue is glibc is not designed to support static linking. Use eg Musl as your libc and it is fine.
- 0x0 13y ago
  I'm guessing this will also help with hardcoded paths for config files and so on.
  - mwilliamson 13y ago
    Yes, so long as the hard-coded path for the the config file is within the output directory, the program will find the right config file for that particular installation. You still won't be able to put the config file in an arbitrary location, but at least you can have isolated instances of the program with isolated config files.
    - justincormack 13y ago
      You could bind mount new copies of config files in other locations, but you have to know where they are...
jbangert 13y ago
However, on a technical note, 'relocatable program' usually refers to a completely different concept, namely that the programs binary has relocation entries so it can be 'relocated' and executed at any address range (thing Address Space Layout randomization for binaries or libraries that will be linked).
- mwilliamson 13y ago
  You're absolutely right, but "relocatable" was the best word I could find that's used for the concept that was relatively accurate. The other usual adjective is "portable", but I felt that was misleading since you can't just stick the binaries on a USB stick, since the target computer needs to have whack-run installed, which requires root access. Suggestions for alternatives are extremely welcome!
  - palebluedot 13y ago
    I had the same concern - I too was confused when I read 'relocatable'. Some suggestions: * Movable * Location-Independent * Path-Independent * Path Neutral
  - innguest 13y ago
    We've been calling them "portable" much before usb sticks were around: http://en.wikipedia.org/wiki/Portable_Executable http://en.wikipedia.org/wiki/Portable_Executable I suggest you reconsider. :) I've been waiting to see a .dmg equivalent for Linux.
    - asveikau 13y ago
      > I've been waiting to see a .dmg equivalent for Linux. In a very literal technical sense I have no idea what this means. Linux can mount a file as if it is a disk with -o loop. I suspect you're really talking about something else. App bundles perhaps?
      - lmm 13y ago
ibisum 13y ago
Reminds me of gobolinux .. http://www.gobolinux.org/ http://www.gobolinux.org/ Another wild scheme to get around fixed paths in Linux ..
mratzloff 13y ago
So does it require root to install applications the first time?
- mwilliamson 13y ago
  You just need to be root to install whack-run, since whack-run requires the setuid flag. Once you've installed whack-run, you can install the applications as an ordinary user.
  - mh- 13y ago
    (yes)
  - asveikau 13y ago
    How complex is whack-run? A setuid binary should be sure to minimize its attack surface. It's just a quick look but when I see your shit-ton of .py files and then you say there is setuid involved it does not fill me with lots of confidence.
    - mwilliamson 13y ago
      whack-run is a separate C binary for this very reason: https://github.com/mwilliamson/whack-run https://github.com/mwilliamson/whack-run Whack itself is run with normal privileges.
      - deleted 13y ago
        [deleted]
        mwilliamson 13y ago
        I'm no Linux security expert, so any advice on that front is hugely appreciated. whack-run is supposed to drop any privileges it holds when it runs "setgid(getgid())" and "setuid(getuid())", and it exits early if either of those calls fail. Is there any reason why that wouldn't work?
Ixiaus 13y ago
This is a cool project but isn't that kind of what containerization is trying solve? (ex: docker or FreeBSD's zfs+ezjails)
- mwilliamson 13y ago
  Containerization solves this problem, but it also solves many more. Whack is intentionally narrow in scope, which means that there's relatively little to learn as a user. Whack certainly isn't a replacement for Docker and the like.
idupree 13y ago
It's possible to do similar things without setuid/root, by modifying the absolute paths in the binaries and config files. To modify binaries, http://nixos.org/patchelf.html http://nixos.org/patchelf.html . I'm not aware of an automated tool to do this at present.
- mwilliamson 13y ago
  Originally, I tried using similar tools (I think I used chrpath, I can't remember whether I tried patchelf) to implement whack, but I found it quite time-consuming to get things working, and had limited success. I chose the current implementation since it was simpler, quicker and more reliable, and the setuid requirement wasn't a problem for my particular use case. Having said that, it would be great to have an implementation that didn't require setuid/root.
rdw 13y ago
How can you not be calling them "whackages"? You're sitting on a gold mine!

35 points by mwilliamson 13y ago

X4 13y ago

Isn't that just statically linking binaries? Couldn't I do that manually too very easily? Serious question. Otherwise, if it's really relocating binaries, then it's awesomse, BUT I definately want to know MORE about the 'caching' part. That comes short in the docs.

mwilliamson 13y ago
In my experience, a lot of Linux programs aren't straightforward to compile statically. Having said that, that might just be my ineptitude! Which bit of the caching isn't clear? If you're happy that whack creates relocatable versions, then the caching works just by copying the output directory into ~/.cache, and then copying that directory to the target on subsequent installations. The exact details of how programs are made relocatable are under "How does Whack work?" in the README.
- X4 13y ago
  Thank you, that answers my question No you're right, not all binaries can be statically linked easily, it's very tedious in these cases. Oh, no I hoped that something performance related is happening when you said cache, like tuning the binary, predictive read-ahead streaming of the binary to memory, or something like that. I didn't know that it's a copy operation.
- justincormack 13y ago
  Some programs rely on dynamic libraries for modules so you can't, but the main issue is glibc is not designed to support static linking. Use eg Musl as your libc and it is fine.
0x0 13y ago
I'm guessing this will also help with hardcoded paths for config files and so on.
- mwilliamson 13y ago
  Yes, so long as the hard-coded path for the the config file is within the output directory, the program will find the right config file for that particular installation. You still won't be able to put the config file in an arbitrary location, but at least you can have isolated instances of the program with isolated config files.
  - justincormack 13y ago
    You could bind mount new copies of config files in other locations, but you have to know where they are...

jbangert 13y ago

However, on a technical note, 'relocatable program' usually refers to a completely different concept, namely that the programs binary has relocation entries so it can be 'relocated' and executed at any address range (thing Address Space Layout randomization for binaries or libraries that will be linked).

mwilliamson 13y ago
You're absolutely right, but "relocatable" was the best word I could find that's used for the concept that was relatively accurate. The other usual adjective is "portable", but I felt that was misleading since you can't just stick the binaries on a USB stick, since the target computer needs to have whack-run installed, which requires root access. Suggestions for alternatives are extremely welcome!
- palebluedot 13y ago
  I had the same concern - I too was confused when I read 'relocatable'. Some suggestions: * Movable * Location-Independent * Path-Independent * Path Neutral
- innguest 13y ago
  We've been calling them "portable" much before usb sticks were around: http://en.wikipedia.org/wiki/Portable_Executable http://en.wikipedia.org/wiki/Portable_Executable I suggest you reconsider. :) I've been waiting to see a .dmg equivalent for Linux.
  - asveikau 13y ago
    > I've been waiting to see a .dmg equivalent for Linux. In a very literal technical sense I have no idea what this means. Linux can mount a file as if it is a disk with -o loop. I suspect you're really talking about something else. App bundles perhaps?
    - lmm 13y ago

ibisum 13y ago

Reminds me of gobolinux .. http://www.gobolinux.org/ http://www.gobolinux.org/ Another wild scheme to get around fixed paths in Linux ..

mratzloff 13y ago

So does it require root to install applications the first time?

mwilliamson 13y ago
You just need to be root to install whack-run, since whack-run requires the setuid flag. Once you've installed whack-run, you can install the applications as an ordinary user.
- mh- 13y ago
  (yes)
- asveikau 13y ago
  How complex is whack-run? A setuid binary should be sure to minimize its attack surface. It's just a quick look but when I see your shit-ton of .py files and then you say there is setuid involved it does not fill me with lots of confidence.
  - mwilliamson 13y ago
    whack-run is a separate C binary for this very reason: https://github.com/mwilliamson/whack-run https://github.com/mwilliamson/whack-run Whack itself is run with normal privileges.
    - deleted 13y ago
      [deleted]
      - mwilliamson 13y ago
        I'm no Linux security expert, so any advice on that front is hugely appreciated. whack-run is supposed to drop any privileges it holds when it runs "setgid(getgid())" and "setuid(getuid())", and it exits early if either of those calls fail. Is there any reason why that wouldn't work?

Ixiaus 13y ago

This is a cool project but isn't that kind of what containerization is trying solve? (ex: docker or FreeBSD's zfs+ezjails)

mwilliamson 13y ago
Containerization solves this problem, but it also solves many more. Whack is intentionally narrow in scope, which means that there's relatively little to learn as a user. Whack certainly isn't a replacement for Docker and the like.

idupree 13y ago

It's possible to do similar things without setuid/root, by modifying the absolute paths in the binaries and config files. To modify binaries, http://nixos.org/patchelf.html http://nixos.org/patchelf.html . I'm not aware of an automated tool to do this at present.

mwilliamson 13y ago
Originally, I tried using similar tools (I think I used chrpath, I can't remember whether I tried patchelf) to implement whack, but I found it quite time-consuming to get things working, and had limited success. I chose the current implementation since it was simpler, quicker and more reliable, and the setuid requirement wasn't a problem for my particular use case. Having said that, it would be great to have an implementation that didn't require setuid/root.

rdw 13y ago

How can you not be calling them "whackages"? You're sitting on a gold mine!