5 ms·

Hysp – An independent package manager

42 points by weinzierl 3y ago

NewJazz 3y ago
danger_accept_invalid_certs(true) // allow http as well I don't know what I expected from a github user named "pwnwriter".
- deleted 3y ago
  [deleted]
- ragnarel 3y ago
  In the "Why?" section he says that he is a CTF player. He should be very security-concernet, he shouldn't?
  - 8organicbits 3y ago
    I've found that security skill in one area doesn't translate to others. Someone good at breaking things isn't necessarily good at building secure things.
- PhilipRoman 3y ago
  While I haven't looked at the code and can't tell you if it's the case here, most package managers I've used use unencrypted http for the traffic. Security is achieved by verifying the signature of the downloaded file. An additional benefit is that you can easily put a http proxy in front of it.
- azathothas 3y ago
  Hi, I am the package maintainer for https://github.com/metis-os/hysp-pkgs https://github.com/metis-os/hysp-pkgs (The default source that's shipped with hysp) To address your concerns: 1. Hysp allows making request to http-only sites, because hysp doesn't care where you host your binaries. As long as you have sha256sum/blake3sum of the binaries, and they match, hysp will work. 2. The default pkg source that ships with hysp, is hosted on github itself, where's there's no need to use http. HTTP is simply a fallback, and meant for maximum compatibility. 3. As installing random binaries from random sources is not advisable due to security concerns, the pkg-source can be self-hosted by anyone and hysp can be configured to use that instead of the default source.
  - NewJazz 3y ago
    Good to know that checksums are being used... But are they signed at all? How do I get the checksum in the first place? Sorry don't mean to heckle, just was skimming through the code and saw that line.
    - azathothas 3y ago
      It's up to the pkg maintainer that's being used as the source. The default PKGs, located at: https://github.com/metis-os/hysp-pkgs https://github.com/metis-os/hysp-pkgs, are automated via GitHub actions. You can look at the workflow codes. As to answer the signing question: $BIN.TOML must contain either blake3sum or sha256sum of the binary specified as source. And once hysp downloads them, it checks if they match exactly as specified in the $BIN.TOML We prevent MITM or any other shenanigans by trusting whoever is hosting the pkg, isn't tampering both the binary and the checksums. So yes, not completely foolproof (an attacker could, in theory, tamper both the binary and the checksum), but that's why we have the self-hosting option for people who want to trust only themselves and their servers. The config allows you to specify different URL/Host for the TOML files and the actual binaries. So unless, both the TOML files and source binaries are being hosted on an HTTP site, MITM is not possible. And as already specified, the default source that ships with hysp, doesn't use HTTP anywhere.
PufPufPuf 3y ago
Any advantages over established "independent" package managers for Linux, like Homebrew?
- tronicdude 3y ago
  i..i've never seen a homebrew for linux user in the wild
  - akdor1154 3y ago
    I've started using it as a goto for vendor tooling (go, node, etc), great experience so far and will likely continue.
    - airtonix 3y ago
      [dead]
    - pzmarzly 3y ago
      I've been using Linuxbrew for few months last year, it was always hit-or-miss whether some tool will work or if it will be macOS only recipe. Then I started using aarch64 CPUs, and Linuxbrew still doesn't work on those. Bummer. So while I'd love to see a package manager that spans all Linux distros, Homebrew is not it, at least for now. Nix may be it, but it has its own interesting issues at times. I recently used distrobox to run Arch Linux container on Fedora host and it was a good experience. I did not try exposing the CLIs from there to the host OS but it sounds possible https://github.com/89luca89/distrobox/blob/main/docs/usage/distrobox-export.md https://github.com/89luca89/distrobox/blob/main/docs/usage/d... . I don't want to daily-drive Arch again, but it's a great project to install fresh versions of various CLI tools from.
  - deleted 3y ago
    [deleted]
  - zaps 3y ago
    I use Linuxbrew extensively across multiple WSL distros every day. As the “weird Windows person” on a team full of Mac users the experience is pretty uniform—the same brew commands for standing up a macOS dev environment mostly “just work” on WSL/Linux.
ericyd 3y ago
Even with the "Why??" section of the readme, I was left wondering "Why?"
- xpe 3y ago
  /me seeing the icon selection: eyebrows raised, head scratched
  - croes 3y ago
    For me it's a red flag.
- giancarlostoro 3y ago
  Sounds like he wants a uniform way to install anything on any Linux OS, default package manager limitations be damned.
  - fodkodrasz 3y ago
    But this is already solved by nix or pkgsrc.
    - xpe 3y ago
      I hope we never find a time where a person (whether it be a researcher, tinkerer, dreamer, visionary, misfit, whatever) hears "X is already solved" and says "that's good enough for me".
- deleted 3y ago
  [deleted]
- azathothas 3y ago
  Hi, I am the package maintainer for https://github.com/metis-os/hysp-pkgs https://github.com/metis-os/hysp-pkgs (The default source that's shipped with hysp) To address your concerns: 1. Yes, the readme could be better and have better documentation explaining the why. We will update this and clearly list the reasons. Thanks for pointing it out. As to the actual why: 1. Be completely agnostic: Do not depend on or care about what OS, Architecture & dependencies are installed. Just parse config.toml, based on that, install what's needed. 2. The default pkg source that ships with hysp, contains only statically compiled/linked binaries that will work on anything. 3. As installing random binaries from random sources is not advisable due to security concerns, the pkg-source can be self-hosted by anyone and hysp can be configured to use that instead of the default source.
yjftsjthsd-h 3y ago
> bin="/home/user/.local/share/hysp/bin/" Why not ~/.local/bin?
- saghm 3y ago
  One advantage I can think of is that it makes it easier to remove everything at once without needing to rely on any behavior of the package manager itself. I imagine that this could be useful if you want to test changes "live" rather than in a sandboxed environment.
- spyridonas 3y ago
  The /home/user/.local one doesn't require the user to be root.
  - RealStickman_ 3y ago
    You missed the ~ The full path would be /home/user/.local/bin
- azathothas 3y ago
  Hi, I am the package maintainer for https://github.com/metis-os/hysp-pkgs https://github.com/metis-os/hysp-pkgs (The default source that's shipped with hysp) To address your concern: 1. $HOME/.local/share/hysp/bin is what's shipped as default location for reasons that others have already mentioned in this thread. 2. You can edit the config.toml to use whatever directory pleases you.
from-nibly 3y ago
What about nix? Or if it's just for developer tools, the 100 tool managers that exist, like asdf and rtx. Sometimes a package manager doesn't have my package doesnt get solved by adding one more package manager.
- airtonix 3y ago
  [dead]
- yjftsjthsd-h 3y ago
  A supporting note - my first thought was "well the author wants to work in CTF/pentesting situations where root access might not be available" but it tuns out that https://nixos.wiki/wiki/Nix_Installation_Guide#Installing_without_root_permissions https://nixos.wiki/wiki/Nix_Installation_Guide#Installing_wi... is pretty well covered these days. At that point the only thing is that nix is complex enough that it might be easier to invent your own package manager than to master nix[0]. (I don't know enough about your other suggestions to comment.) [0] Don't take this criticism too harshly; I'm writing this comment on a nixos laptop;)
  - YoshiRulz 3y ago
    And if you're not able to install Nix (running something on a target machine?), you can use `nix bundle` on your own machine. More relevant for the author may be that a lot of pentesting tools aren't in Nixpkgs yet: https://github.com/NixOS/nixpkgs/issues/81418 https://github.com/NixOS/nixpkgs/issues/81418
  - matrss 3y ago
    There is also nix-portable (https://github.com/DavHau/nix-portable https://github.com/DavHau/nix-portable), which is basically a drop-in replacement for normal nix that does everything required for no-root operation by itself when needed. Just put the single binary in your PATH and it's ready.
  - azathothas 3y ago
    As my comment above stated, hysp relies on single binaries. The default source for PKGs that's shipped with hysp, contains only statically compiled binaries. This allows you to run a single binary anywhere on any system. And since you can host your own binaries for whatever architecture, hysp is agnostic. Further, since only binaries are installed, there's no need for toolchains/cache or any kind of other dependencies.
puppetmaster 3y ago
The aquilles heel of package managers isn´t consistency, but lack of packages. I appreciate the benefit of distro-agnostic package management, but unless you rely on a portable toolchain, or distribute statically linked binaries exclusively, we fall short from the panacea of portability (many other options available, yes, but packages are needed!) I have personally leveraged pkgin[1] for consistency. I can simply drop pkgsrc[2] in my home directory and use the same tools in *bsds, smartos, linux, macos and cheese. [1]https://pkgin.net/ https://pkgin.net/ [2]https://www.pkgsrc.org/ https://www.pkgsrc.org/ edit: being more specific
- NewJazz 3y ago
  I wouldn't balk at a pure static binary distro. It could have its niche.
  - puppetmaster 3y ago
    Some people would argue that there are few differences between a container and a static binary... And of course, there is also https://sta.li/ https://sta.li/
- deleted 3y ago
  [deleted]
- azathothas 3y ago
  Hi, I am the package maintainer for https://github.com/metis-os/hysp-pkgs https://github.com/metis-os/hysp-pkgs (The default source that's shipped with hysp) To address your concerns: 1. Yes, no portable toolchains or any other kinds of dependencies are required. 2. Each & every single binary is statically compiled. This is the core at philosophy of Hysp. A single binary that runs anywhere. 3. Currently, there's about 200 pkgs for x86_64 & arm64 each. The upstream source (https://github.com/Azathothas/Toolpacks https://github.com/Azathothas/Toolpacks) has over 400 for x86_64 & 300+ for arm64, which will slowly be added to Hysp-Pkgs. 4. As installing random binaries from random sources is not advisable due to security concerns, the pkg-source can be self-hosted by anyone and hysp can be configured to use that instead of the default source.
spyridonas 3y ago
Wouldn't it be easier to make a `snap install x x x x ` Or `flatpak install x x x x` script ? Isn't this type of issue what those two project's aim to solve ?
- azathothas 3y ago
  Hi, I am the package maintainer for https://github.com/metis-os/hysp-pkgs https://github.com/metis-os/hysp-pkgs (The default source that's shipped with hysp) To address your concerns: 1. Hysp is a single binary (Statically compiled), download & run anywhere kind of package manager. 2. Hysp, doesn't care about dependencies, and neither does it attempt to install anything that's not specifically defined in the config. 3. Due to 1 & 2, you do not need go or rust or anything else installed to install something. No dependencies whatsoever. This saves enormous space, storage & time. 4. As installing random binaries from random sources is not advisable due to security concerns, the pkg-source can be self-hosted by anyone and hysp can be configured to use that instead of the default source.

42 points by weinzierl 3y ago

danger_accept_invalid_certs(true) // allow http as well I don't know what I expected from a github user named "pwnwriter".

deleted 3y ago
[deleted]
ragnarel 3y ago
In the "Why?" section he says that he is a CTF player. He should be very security-concernet, he shouldn't?
- 8organicbits 3y ago
  I've found that security skill in one area doesn't translate to others. Someone good at breaking things isn't necessarily good at building secure things.
PhilipRoman 3y ago
While I haven't looked at the code and can't tell you if it's the case here, most package managers I've used use unencrypted http for the traffic. Security is achieved by verifying the signature of the downloaded file. An additional benefit is that you can easily put a http proxy in front of it.
azathothas 3y ago
Hi, I am the package maintainer for https://github.com/metis-os/hysp-pkgs https://github.com/metis-os/hysp-pkgs (The default source that's shipped with hysp) To address your concerns: 1. Hysp allows making request to http-only sites, because hysp doesn't care where you host your binaries. As long as you have sha256sum/blake3sum of the binaries, and they match, hysp will work. 2. The default pkg source that ships with hysp, is hosted on github itself, where's there's no need to use http. HTTP is simply a fallback, and meant for maximum compatibility. 3. As installing random binaries from random sources is not advisable due to security concerns, the pkg-source can be self-hosted by anyone and hysp can be configured to use that instead of the default source.
- NewJazz 3y ago
  Good to know that checksums are being used... But are they signed at all? How do I get the checksum in the first place? Sorry don't mean to heckle, just was skimming through the code and saw that line.
  - azathothas 3y ago
    It's up to the pkg maintainer that's being used as the source. The default PKGs, located at: https://github.com/metis-os/hysp-pkgs https://github.com/metis-os/hysp-pkgs, are automated via GitHub actions. You can look at the workflow codes. As to answer the signing question: $BIN.TOML must contain either blake3sum or sha256sum of the binary specified as source. And once hysp downloads them, it checks if they match exactly as specified in the $BIN.TOML We prevent MITM or any other shenanigans by trusting whoever is hosting the pkg, isn't tampering both the binary and the checksums. So yes, not completely foolproof (an attacker could, in theory, tamper both the binary and the checksum), but that's why we have the self-hosting option for people who want to trust only themselves and their servers. The config allows you to specify different URL/Host for the TOML files and the actual binaries. So unless, both the TOML files and source binaries are being hosted on an HTTP site, MITM is not possible. And as already specified, the default source that ships with hysp, doesn't use HTTP anywhere.

PufPufPuf 3y ago

Any advantages over established "independent" package managers for Linux, like Homebrew?

tronicdude 3y ago
i..i've never seen a homebrew for linux user in the wild
- akdor1154 3y ago
  I've started using it as a goto for vendor tooling (go, node, etc), great experience so far and will likely continue.
  - airtonix 3y ago
    [dead]
  - pzmarzly 3y ago
    I've been using Linuxbrew for few months last year, it was always hit-or-miss whether some tool will work or if it will be macOS only recipe. Then I started using aarch64 CPUs, and Linuxbrew still doesn't work on those. Bummer. So while I'd love to see a package manager that spans all Linux distros, Homebrew is not it, at least for now. Nix may be it, but it has its own interesting issues at times. I recently used distrobox to run Arch Linux container on Fedora host and it was a good experience. I did not try exposing the CLIs from there to the host OS but it sounds possible https://github.com/89luca89/distrobox/blob/main/docs/usage/distrobox-export.md https://github.com/89luca89/distrobox/blob/main/docs/usage/d... . I don't want to daily-drive Arch again, but it's a great project to install fresh versions of various CLI tools from.
- deleted 3y ago
  [deleted]
- zaps 3y ago
  I use Linuxbrew extensively across multiple WSL distros every day. As the “weird Windows person” on a team full of Mac users the experience is pretty uniform—the same brew commands for standing up a macOS dev environment mostly “just work” on WSL/Linux.

ericyd 3y ago

Even with the "Why??" section of the readme, I was left wondering "Why?"

xpe 3y ago
/me seeing the icon selection: eyebrows raised, head scratched
- croes 3y ago
  For me it's a red flag.
giancarlostoro 3y ago
Sounds like he wants a uniform way to install anything on any Linux OS, default package manager limitations be damned.
- fodkodrasz 3y ago
  But this is already solved by nix or pkgsrc.
  - xpe 3y ago
    I hope we never find a time where a person (whether it be a researcher, tinkerer, dreamer, visionary, misfit, whatever) hears "X is already solved" and says "that's good enough for me".
deleted 3y ago
[deleted]
azathothas 3y ago
Hi, I am the package maintainer for https://github.com/metis-os/hysp-pkgs https://github.com/metis-os/hysp-pkgs (The default source that's shipped with hysp) To address your concerns: 1. Yes, the readme could be better and have better documentation explaining the why. We will update this and clearly list the reasons. Thanks for pointing it out. As to the actual why: 1. Be completely agnostic: Do not depend on or care about what OS, Architecture & dependencies are installed. Just parse config.toml, based on that, install what's needed. 2. The default pkg source that ships with hysp, contains only statically compiled/linked binaries that will work on anything. 3. As installing random binaries from random sources is not advisable due to security concerns, the pkg-source can be self-hosted by anyone and hysp can be configured to use that instead of the default source.

yjftsjthsd-h 3y ago

> bin="/home/user/.local/share/hysp/bin/" Why not ~/.local/bin?

saghm 3y ago
One advantage I can think of is that it makes it easier to remove everything at once without needing to rely on any behavior of the package manager itself. I imagine that this could be useful if you want to test changes "live" rather than in a sandboxed environment.
spyridonas 3y ago
The /home/user/.local one doesn't require the user to be root.
- RealStickman_ 3y ago
  You missed the ~ The full path would be /home/user/.local/bin
azathothas 3y ago
Hi, I am the package maintainer for https://github.com/metis-os/hysp-pkgs https://github.com/metis-os/hysp-pkgs (The default source that's shipped with hysp) To address your concern: 1. $HOME/.local/share/hysp/bin is what's shipped as default location for reasons that others have already mentioned in this thread. 2. You can edit the config.toml to use whatever directory pleases you.

from-nibly 3y ago

What about nix? Or if it's just for developer tools, the 100 tool managers that exist, like asdf and rtx. Sometimes a package manager doesn't have my package doesnt get solved by adding one more package manager.

airtonix 3y ago
[dead]
yjftsjthsd-h 3y ago
A supporting note - my first thought was "well the author wants to work in CTF/pentesting situations where root access might not be available" but it tuns out that https://nixos.wiki/wiki/Nix_Installation_Guide#Installing_without_root_permissions https://nixos.wiki/wiki/Nix_Installation_Guide#Installing_wi... is pretty well covered these days. At that point the only thing is that nix is complex enough that it might be easier to invent your own package manager than to master nix[0]. (I don't know enough about your other suggestions to comment.) [0] Don't take this criticism too harshly; I'm writing this comment on a nixos laptop;)
- YoshiRulz 3y ago
  And if you're not able to install Nix (running something on a target machine?), you can use `nix bundle` on your own machine. More relevant for the author may be that a lot of pentesting tools aren't in Nixpkgs yet: https://github.com/NixOS/nixpkgs/issues/81418 https://github.com/NixOS/nixpkgs/issues/81418
- matrss 3y ago
  There is also nix-portable (https://github.com/DavHau/nix-portable https://github.com/DavHau/nix-portable), which is basically a drop-in replacement for normal nix that does everything required for no-root operation by itself when needed. Just put the single binary in your PATH and it's ready.
- azathothas 3y ago
  As my comment above stated, hysp relies on single binaries. The default source for PKGs that's shipped with hysp, contains only statically compiled binaries. This allows you to run a single binary anywhere on any system. And since you can host your own binaries for whatever architecture, hysp is agnostic. Further, since only binaries are installed, there's no need for toolchains/cache or any kind of other dependencies.

puppetmaster 3y ago

The aquilles heel of package managers isn´t consistency, but lack of packages. I appreciate the benefit of distro-agnostic package management, but unless you rely on a portable toolchain, or distribute statically linked binaries exclusively, we fall short from the panacea of portability (many other options available, yes, but packages are needed!) I have personally leveraged pkgin[1] for consistency. I can simply drop pkgsrc[2] in my home directory and use the same tools in *bsds, smartos, linux, macos and cheese. [1]https://pkgin.net/ https://pkgin.net/ [2]https://www.pkgsrc.org/ https://www.pkgsrc.org/ edit: being more specific

NewJazz 3y ago
I wouldn't balk at a pure static binary distro. It could have its niche.
- puppetmaster 3y ago
  Some people would argue that there are few differences between a container and a static binary... And of course, there is also https://sta.li/ https://sta.li/
deleted 3y ago
[deleted]
azathothas 3y ago
Hi, I am the package maintainer for https://github.com/metis-os/hysp-pkgs https://github.com/metis-os/hysp-pkgs (The default source that's shipped with hysp) To address your concerns: 1. Yes, no portable toolchains or any other kinds of dependencies are required. 2. Each & every single binary is statically compiled. This is the core at philosophy of Hysp. A single binary that runs anywhere. 3. Currently, there's about 200 pkgs for x86_64 & arm64 each. The upstream source (https://github.com/Azathothas/Toolpacks https://github.com/Azathothas/Toolpacks) has over 400 for x86_64 & 300+ for arm64, which will slowly be added to Hysp-Pkgs. 4. As installing random binaries from random sources is not advisable due to security concerns, the pkg-source can be self-hosted by anyone and hysp can be configured to use that instead of the default source.

spyridonas 3y ago

Wouldn't it be easier to make a `snap install x x x x ` Or `flatpak install x x x x` script ? Isn't this type of issue what those two project's aim to solve ?

azathothas 3y ago
Hi, I am the package maintainer for https://github.com/metis-os/hysp-pkgs https://github.com/metis-os/hysp-pkgs (The default source that's shipped with hysp) To address your concerns: 1. Hysp is a single binary (Statically compiled), download & run anywhere kind of package manager. 2. Hysp, doesn't care about dependencies, and neither does it attempt to install anything that's not specifically defined in the config. 3. Due to 1 & 2, you do not need go or rust or anything else installed to install something. No dependencies whatsoever. This saves enormous space, storage & time. 4. As installing random binaries from random sources is not advisable due to security concerns, the pkg-source can be self-hosted by anyone and hysp can be configured to use that instead of the default source.