9 ms·

A clever trick with a funny example, but I’m also fine with curl|bash — as fine as I am with “pip3 install thing” or installing “thing.dmg”. I almost always de

• kevin_thibedeau 4y ago
  The problem with Pypi is the transitive dependencies. You may trust a high profile package but it could pull in any number of projects that are exploited by bad actors.
  • freeqaz 4y ago
    This is a real problem that I believe will become increasingly commonly used by attackers to break into companies. Unfortunately, it's not an easy problem to automate the detection. There are some heuristics, but you're basically writing something like Norton Antivirus for Python packages lol. A sufficiently skilled attacker can make their code look totally innocent[0]. :) Source: Former security engineer at Uber/Snap that's now working on tooling in this space[1]. 0: "Underhanded C Competition" http://www.underhanded-c.org/ http://www.underhanded-c.org/ 1: Open Source "Supply Chain Security" platform https://github.com/lunasec-io/lunasec https://github.com/lunasec-io/lunasec
    • jollofricepeas 4y ago
      It’s already extremely common in the Python, Ruby and Node ecosystem (see sources below). The only company that has a real chance at making a dent in the issue is GitHub and to a degree they already have. SOURCES: - https://www.mandiant.com/resources/blog/supply-chain-node-js https://www.mandiant.com/resources/blog/supply-chain-node-js - https://www.zdnet.com/article/backdoor-found-in-ruby-library-for-checking-for-strong-passwords/ https://www.zdnet.com/article/backdoor-found-in-ruby-library... - https://www.reversinglabs.com/blog/mining-for-malicious-ruby-gems https://www.reversinglabs.com/blog/mining-for-malicious-ruby... - https://news.sophos.com/en-us/2021/10/24/node-poisoning-hijacked-package-delivers-coin-miner-and-credential-stealing-backdoor/amp/ https://news.sophos.com/en-us/2021/10/24/node-poisoning-hija... - https://www.bleepingcomputer.com/news/security/dev-corrupts-npm-libs-colors-and-faker-breaking-thousands-of-apps/amp/ https://www.bleepingcomputer.com/news/security/dev-corrupts-... - https://portswigger.net/daily-swig/amp/malicious-python-library-ctx-removed-from-pypi-repo https://portswigger.net/daily-swig/amp/malicious-python-libr...
  • raverbashing 4y ago
    And a problem which is some 5x worse in npm
    • marcosdumay 4y ago
      5x worse on each level, maybe. That composes quite quickly. Npm packages have really crazy dependency graphs. It's usually on the 1000s of times larger than what you get on most languages, 100s of times larger than python or ruby.
      • pitsnatch 4y ago
        Why is this the case?
        • raverbashing 4y ago
          Because JS had a very poor stdlib and attracted people that had little to no experience in best-practices in older languages Hence left-pad. And this pet-peeve of mine https://nodejs.org/docs/latest/api/process.html#processargv https://nodejs.org/docs/latest/api/process.html#processargv (who cares about the node fullpath in argv[0]?)
          • SpeedilyDamage 4y ago
            Why oh why do people cite left-pad like this as an example of package bloat? It’s functionality was so valuable it was included in ECMAScript 2017! The problem with left pad was a security one, not a package bloat one.
            • deleted 4y ago
              [deleted]
            • gfxgirl 4y ago
              No, the problem is bad developers pulling in dependencies for trivial functionally. If there was a `for-loop` npm package bad devs would be pulling it in instead of writing their own for loops. Padding on the left is something if it doesn't exist you write it in a few lines of code yourself. You don't add a package for such trivial functionality.
  • pydry 4y ago
    I find that this is more of an npm problem - thanks to the lack of a standard library and a cultural affectation towards small packages with a narrow focus. A high level python package is more likely to depend upon 3-30 well known packages.
    • danenania 4y ago
      I think the biggest factor is just a much larger overall number of packages in the ecosystem. If there are, on average, 20 popular packages for solving a given problem instead of 3, you end up with exponentially more transitive dependencies.
  • wil421 4y ago
    The company I work for hosts their own PyPi mirror internally and will attempt to ban packages or dependencies before PyPi does.
    • nerdponx 4y ago
      I think more companies should do this.
  • jlarocco 4y ago
    But that's how trust works. If you don't trust them to pull in safe dependencies, then you don't really trust them.
    • warkdarrior 4y ago
      There are various degrees of trust. One may trust a package maintainer to write correct, well-designed code and to maintain it. But should one trust that package maintainer to have solid security standards in their code repo, their CI/CD, and their distribution mechanism?
  • Fatnino 4y ago
    So, actually you don't really trust them...
• freeqaz 4y ago
  The apt repo thing is actually why I switched to Arch a while back. In Arch you have most packages being up-to-date by default (because you're on a "rolling release"). There is also the "Arch User Repository" which includes a bunch of useful packages that are easy to install, but with full transparency about where the contents are coming from (you can inspect the script). They have even released an easy installed for Arch now that you can use, or you can just use Manjaro Linux. Regardless though, I really appreciate this distro for development after many years of using Ubuntu!
  • heyoni 4y ago
    Where this easy arch install at?
    • freeqaz 4y ago
      https://wiki.archlinux.org/title/Archinstall https://wiki.archlinux.org/title/Archinstall It's called archinstall and it's a helper script (it looks like)
      • heyoni 4y ago
        Oh thanks, I'll look into it!
• fathyb 4y ago
  The point of TFA is preventing piping curl to bash, not preventing from running bash scripts. The problem with piping is that bash will read line by line, keeping the connection open while it runs. If the connection fails for any reason, the script will stop, potentially breaking or corrupting things. This can be prevented by first downloading the file, and then running it.
  • codetrotter 4y ago
    > If the connection fails for any reason, the script will stop, potentially breaking or corrupting things. If the script is meant to be piped from curl, and is well written, it will be written so that it first defines everything as functions and then at the very end makes a single function call. This ensures that the script will only do anything if it has been completely downloaded. For example, the script that rustup.rs tells you to pipe from curl is written in that way.
    • vbezhenar 4y ago
      You can't be sure that developer did that and it's better to stay on the safe side IMO.
    • Arnavion 4y ago
      >it will be written so that it first defines everything as functions and then at the very end makes a single function call. This ensures that the script will only do anything if it has been completely downloaded. This is not sufficient. More care is required. lsp_init() { ... } lsp_init If the last line gets truncated between the `s` and the `p`, then `ls` gets executed. Of course `ls` is harmless, but I'm sure you can imagine how it could be worse. In other words, not only do you have to wrap your script in functions, but you have to ensure that any top-level function / command invocations are named such that they do not become different commands if truncated. This is unsolvable in general, because the user can have any arbitrary names in their $PATH , such as custom personal utils in ~/.local/bin which can have any unforeseeable name. It's much easier to just wrap the script in `()` to make it run in a subshell. bash will not run anything until it sees the closing `)` so truncated scripts are not a problem, and it also doesn't have the name collision problem.
• Ferret7446 4y ago
  Reputability won't help you if someone intercepts your connection and replaces the script. TLS helps, insofar as the version of curl you're using and its configuration is correct, your machine has the right certs installed, and the remote domain and/or host and/or repo hasn't been hijacked.
  • maccard 4y ago
    If eithet you or the host have been compromised, it doesn't matter where you curl into bash, run pip install or download an exe/dmg/deb file - you're hosed.
    • Ferret7446 4y ago
      Packages are signed, unless you're using a niche, insecure distro or package manager. Of course, that signing key could also be compromised, but those are usually more tightly guarded than web servers, which are compromised rather more frequently (any reputable dev should have their key in a hardware enclave like yubikey or similar).
      • ziml77 4y ago
        Then maybe a solution is script signing. PowerShell scripts can contain a signature at the end, and the shell will check that signature. It can either be mandated all the time or only if the script originally comes from a remote source (or disabled completely if you don’t care). Since it’s just a comment block, it’s fully backwards compatible to add too
        • maccard 4y ago
          This solves the part of downloading partial scripts, but doesn't solve the problem of malicious scripts. If you download a script from a compromised host, they'll just re-sign the modified script with their own signature. The solution is that you have to trust the script source, and transmit over secure channels.
• andreareina 4y ago
  My problem with curl|bash is that it tends to not take into account my preferences wrt where binaries, config files, etc go. If you’re lucky it’ll take an envar. If you’re really lucky they’ll tell you what it is in the docs. I recently had to install the latest curl from source and it was as easy as ./configure --prefix ….
• vbezhenar 4y ago
  An interesting thing is that it should be the other way around. Repository packages usually are signed with GPG. You can even use HTTP - it does not matter. And GPG could be used to sign packages on another machine, may be even on offline machine with HSM. Things like pip or curl rely only on HTTPS. There are many agents between developer who built a deliverable artifact and your machine. There's WWW-server. There's some CDN with HTTPS termination. There's NSA which can issue fake certificate and MITM attack you (mostly imaginably threat for sure, but still). And hacked WWW servers already happened. Transmission website was hacked and their binary was infected. Now I don't claim it to be a big issue. Malevolent developer is much more dangerous and much more likely to happen. But if we want to compare those two methods - I think that repositories with proper signatures are safer.
  • deleted 4y ago
    [deleted]
  • KyeRussell 4y ago
    At the risk of pointing out what I see as the obvious, but you’re focusing on different attacks. It’s a lot more nuance than “it should be the other way around”. A very valid argument can be made that it’s more likely that you’re installing a dodgy package rather than being subject to some sort of MITM attack.
  • deleted 4y ago
    [deleted]
  • gorgoiler 4y ago
    An apt repository has to be trusted forever: every apt upgrade brings a risk that something new and nefarious will slip in. With curl|bash it’s a one time risk.
• raverbashing 4y ago
  And in the end you could always do curl -o file.sh bash file.sh No detection
  • jrochkind1 4y ago
    Right, the point of the thought experiment is to inject malicious code only in cases where someone is piping directly to bash without reading it, and not in cases where they might have the opportunity to read it. So in that case you would not inject malicious code. That is correct for the exersize.
  • prmoustache 4y ago
    I hate it when I do that, look at the script and realize that the script itself curl | bash other scripts. Unless there are other sane ways to install the project, I just put in on my blacklist and ignore it.
• dheera 4y ago
  curl|bash does't have an obvious UNinstall mechanism. You can pip3 uninstall, apt-get remove, but what do you do for curl? It could give a shit all over your system, /usr, /etc and you wouldn't know what to delete.
  • ziml77 4y ago
    But I have the same problem building from source and using make install. There’s often not an easy make uninstall command. And pip uninstall only uninstalls individual packages, it doesn’t clean up auto-installed dependencies while also ignoring dependencies that I had at any point requested to be installed directly
• gfxgirl 4y ago
  curl | bash doesn't bother me because I do it like twice a year from sites I trust. On the otherhand, the node crowd uses "npx command" all the time where npx will download and execute code from the net. Unlike "curl | bash", "npx command" is something you're expected to do 10s or 100s of times a day. Each one of those times is a chance for you to have a typo and execute code from some randon source like if you type "npm commmand" or "npx comman" or "npx coomand" or "npx comman dargument", etc...
• fulafel 4y ago
  You may trust them to produce legit software and have good intentions. But it's not really enough - It's hard to use reputation to divine someone's proficiency and diligence in operating a web server without getting it pwned and the published trojaned. Several dimensions of risk / trust are relevant here, and the distribution mechanism can address some of the opsec risks in a way that is transparent to the end user (you). A platform like PyPI also increases chances of something like that getting detected after it happens, giving you a chance to react in a timely way.