5 ms·

The root of the problem is allowing pip to execute arbitrary code when installing a package from PyPI (default package index), combined with the complete lack o

The root of the problem is allowing pip to execute arbitrary code when installing a package from PyPI (default package index), combined with the complete lack of vetting of PyPI packages. Even something as simple as mistyping a package name can cause malicious code to be executed on your computer. Installing packages (i.e. source code) for programming languages should not execute arbitrary code.

throwaway290 2y ago
Same is true about NPM and post-install scripts. Actually it is even worse because JS is more popular and average JS dependency tree is so much more massive. Any tiny forgotten transitive dependency of a dependency (or dev dependency) 15 layers deep can pull this off. Total leftpadization is not a thing on pypi due to different culture
mrspuratic 2y ago
"curl ... | bash" makes me run away from a project. There is the reductive argument that basically everything you download is arbitrary code, but throwing away the code that is run seems uniquely silly. https://news.ycombinator.com/item?id=21490151 https://news.ycombinator.com/item?id=21490151
- avz 2y ago
  Agreed. Perhaps the most prominent example of this shameful behavior is this one: curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs https://sh.rustup.rs | sh Source: https://www.rust-lang.org/tools/install https://www.rust-lang.org/tools/install
  - hulitu 2y ago
    Haha. This is the most secure part. When you start compiling, rust will happily download random "crates" and include them in your program, because "dependencies".
- chowells 2y ago
  "curl | bash" is the exact same threat vector as "download this compiled installer". I wonder why that doesn't result in the same hyperbolic responses.
  - josephcsible 2y ago
    The difference is that "curl | bash" doesn't save the output and can be distinguished server-side from commands that do <https://news.ycombinator.com/item?id=34145799 https://news.ycombinator.com/item?id=34145799>, so it's an especially easy way for a malicious server to undetectably have you run something different from what you've looked over.
    - chowells 2y ago
      So you're saying it's exactly the same as binary installers, which frequently download additional components during installation?
      - josephcsible 2y ago
        When binary installers download additional components, they usually do so in a way indistinguishable server-side from downloading a file to analyze.
        chowells 2y ago
        Honest ones, sure. Why would a malicious one leave behind evidence of its malicious behavior? The threat model is identical.
        josephcsible 2y ago
        But that behavior would be noticeable in the original download, which would be evidence itself.
- blacksmith_tb 2y ago
  Pipe to bash install approaches at least can just be snagged with wget to inspect the script without running it, lots of package managers seem harder than that to inspect/predict? Not that there couldn't be sneaky things you'd miss, but at least obvious ones might be detectable...
- fragmede 2y ago
  alright, do curl | tee saved.sh | sh then
  - lxgr 2y ago
    Good idea, unless the attacker serves you just more code that fetches and executes a one-off script from memory (and the download link for that is dynamic and only valid once).
    - fragmede 2y ago
      we could go down a rabbit hole of various exploits and defenses, but at end of the day it all comes down to trust. If you don't trust the source of the code, it doesn't matter if it comes in via curl or a .deb signed with GPG, you're still trying to run untrusted code. If your threat model is such that you don't want to do that, don't do that. No one's forcing anybody to run curl | sudo bash at gunpoint.
yeputons 2y ago
Does installation matter at all? Once you've installed a package, you're very likely to immediately include it in the project, build, and run. Any malicious code can easily run during the package's initialization in your app. Seems like prohibiting arbitrary code in installation scripts would only help with issues like Bumblebee's `rm -rf`: https://github.com/MrMEEE/bumblebee-Old-and-abbandoned/issues/123 https://github.com/MrMEEE/bumblebee-Old-and-abbandoned/issue...
- bheadmaster 2y ago
  > Does installation matter at all? Once you've installed a package, you're very likely to immediately include it in the project, build, and run. Any malicious code can easily run during the package's initialization in your app. True, but in the case I've mentioned, if you've mistyped the name of the package, you can safely uninstall it without any issues. Furthermore, code is often ran in (sort of) sandboxed environments like Docker during development, in which case arbitrary code on runtime is less dangerous than arbitrary code on install time.
  - lxgr 2y ago
    Is that not the case here? I'm not sure if pip has a mechanism similar to deb's pre/post-install hooks. > Furthermore, code is often ran in (sort of) sandboxed environments like Docker during development, in which case arbitrary code on runtime is less dangerous than arbitrary code on install time. Wouldn't the package install then also happen in a Docker container anyway, negating the problem? Or how would you install a package to your host environment and then use it from within a container?
    - plonk 2y ago
      Sounds like a good reason to generalize dev containers. At least both dev and deployment environment will be somewhat isolated. And the worst you can do from the dev container is commit bad code, which can be seen in code review.
- kwertzzz 2y ago
Zhyl 2y ago
You need an analogue of Linux distributions (Python distribution? Making choosing a repo a separate and deliberate step?). That way you could pick a repo based on your risk appetite rather than needing to trust PyPI. Debian style python for the cautious and AUR style python for the bleeding edge and reckless.
- bheadmaster 2y ago
  Not really - just downloading files and storing them on the filesystem would be good enough. No need to run an arbitrary setup.py script. But setup.py is part of the Python's legacy, from before PyPI even existed.
- Snild 2y ago
  Isn't that just already my linux dist, if I want to? I can install python-matplotlib from my package manager, and it appears in my python path. I never bother with venvs. Maybe my needs aren't advanced (or exotic) enough.
mdhb 2y ago
This.. AND… dev machines should be isolated environments via something like GitHub Codespaces, local docker containers or Google’s IDX. You also shouldn’t be able to push code directly to anywhere other than a centralized repository. All the build stuff should happen in a dedicated and independent process. From a security point of view you need to accept that the idea of running into a malicious package as a direct or indirect dependency is not only not zero but fairly realistic and you should try to limit the blast radius as much as possible for when that does happen.
- nottorp 2y ago
  Just air gap the dev machines. But an attacker could infiltrate them by manipulating magnetic fields to generate keystrokes / mouse moves so also faraday cage them.
  - mdhb 2y ago
    I don’t know why you’re trying to make this some kind of gotcha. What I said was absolutely 100% standard security advice. You’re just making yourself look silly here.
appplication 2y ago
This is a valid and significant issue, though for our internal packages running arbitrary code on execution has been a lifesaver. Think: complex distributed systems where all nodes need to be synchronized with the version on the driver.
hulitu 2y ago
> Installing packages (i.e. source code) for programming languages should not execute arbitrary code. Tell that to JS, Python and Rust people. But hey, we are secure. /s
elric 2y ago
Other languages with dependency management frameworks (Java with Maven Central) go about this a little different: you can download as many evil dependencies as you like, but unless you actively execute them, nothing will happen. There are some caveats, especially with nasty frameworks that scan the entire classpath, but that's a different issue.
juliangmp 2y ago
Don't most package managers do that? I mean yes it can be exploited but I wouldn't blame pip specifically.

throwaway290 2y ago

Same is true about NPM and post-install scripts. Actually it is even worse because JS is more popular and average JS dependency tree is so much more massive. Any tiny forgotten transitive dependency of a dependency (or dev dependency) 15 layers deep can pull this off. Total leftpadization is not a thing on pypi due to different culture

mrspuratic 2y ago

"curl ... | bash" makes me run away from a project. There is the reductive argument that basically everything you download is arbitrary code, but throwing away the code that is run seems uniquely silly. https://news.ycombinator.com/item?id=21490151 https://news.ycombinator.com/item?id=21490151

avz 2y ago
Agreed. Perhaps the most prominent example of this shameful behavior is this one: curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs https://sh.rustup.rs | sh Source: https://www.rust-lang.org/tools/install https://www.rust-lang.org/tools/install
- hulitu 2y ago
  Haha. This is the most secure part. When you start compiling, rust will happily download random "crates" and include them in your program, because "dependencies".
chowells 2y ago
"curl | bash" is the exact same threat vector as "download this compiled installer". I wonder why that doesn't result in the same hyperbolic responses.
- josephcsible 2y ago
  The difference is that "curl | bash" doesn't save the output and can be distinguished server-side from commands that do <https://news.ycombinator.com/item?id=34145799 https://news.ycombinator.com/item?id=34145799>, so it's an especially easy way for a malicious server to undetectably have you run something different from what you've looked over.
  - chowells 2y ago
    So you're saying it's exactly the same as binary installers, which frequently download additional components during installation?
    - josephcsible 2y ago
      When binary installers download additional components, they usually do so in a way indistinguishable server-side from downloading a file to analyze.
      - chowells 2y ago
        Honest ones, sure. Why would a malicious one leave behind evidence of its malicious behavior? The threat model is identical.
        josephcsible 2y ago
        But that behavior would be noticeable in the original download, which would be evidence itself.
blacksmith_tb 2y ago
Pipe to bash install approaches at least can just be snagged with wget to inspect the script without running it, lots of package managers seem harder than that to inspect/predict? Not that there couldn't be sneaky things you'd miss, but at least obvious ones might be detectable...
fragmede 2y ago
alright, do curl | tee saved.sh | sh then
- lxgr 2y ago
  Good idea, unless the attacker serves you just more code that fetches and executes a one-off script from memory (and the download link for that is dynamic and only valid once).
  - fragmede 2y ago
    we could go down a rabbit hole of various exploits and defenses, but at end of the day it all comes down to trust. If you don't trust the source of the code, it doesn't matter if it comes in via curl or a .deb signed with GPG, you're still trying to run untrusted code. If your threat model is such that you don't want to do that, don't do that. No one's forcing anybody to run curl | sudo bash at gunpoint.

yeputons 2y ago

Does installation matter at all? Once you've installed a package, you're very likely to immediately include it in the project, build, and run. Any malicious code can easily run during the package's initialization in your app. Seems like prohibiting arbitrary code in installation scripts would only help with issues like Bumblebee's `rm -rf`: https://github.com/MrMEEE/bumblebee-Old-and-abbandoned/issues/123 https://github.com/MrMEEE/bumblebee-Old-and-abbandoned/issue...

bheadmaster 2y ago
> Does installation matter at all? Once you've installed a package, you're very likely to immediately include it in the project, build, and run. Any malicious code can easily run during the package's initialization in your app. True, but in the case I've mentioned, if you've mistyped the name of the package, you can safely uninstall it without any issues. Furthermore, code is often ran in (sort of) sandboxed environments like Docker during development, in which case arbitrary code on runtime is less dangerous than arbitrary code on install time.
- lxgr 2y ago
  Is that not the case here? I'm not sure if pip has a mechanism similar to deb's pre/post-install hooks. > Furthermore, code is often ran in (sort of) sandboxed environments like Docker during development, in which case arbitrary code on runtime is less dangerous than arbitrary code on install time. Wouldn't the package install then also happen in a Docker container anyway, negating the problem? Or how would you install a package to your host environment and then use it from within a container?
  - plonk 2y ago
    Sounds like a good reason to generalize dev containers. At least both dev and deployment environment will be somewhat isolated. And the worst you can do from the dev container is commit bad code, which can be seen in code review.
kwertzzz 2y ago

Zhyl 2y ago

You need an analogue of Linux distributions (Python distribution? Making choosing a repo a separate and deliberate step?). That way you could pick a repo based on your risk appetite rather than needing to trust PyPI. Debian style python for the cautious and AUR style python for the bleeding edge and reckless.

bheadmaster 2y ago
Not really - just downloading files and storing them on the filesystem would be good enough. No need to run an arbitrary setup.py script. But setup.py is part of the Python's legacy, from before PyPI even existed.
Snild 2y ago
Isn't that just already my linux dist, if I want to? I can install python-matplotlib from my package manager, and it appears in my python path. I never bother with venvs. Maybe my needs aren't advanced (or exotic) enough.

mdhb 2y ago

This.. AND… dev machines should be isolated environments via something like GitHub Codespaces, local docker containers or Google’s IDX. You also shouldn’t be able to push code directly to anywhere other than a centralized repository. All the build stuff should happen in a dedicated and independent process. From a security point of view you need to accept that the idea of running into a malicious package as a direct or indirect dependency is not only not zero but fairly realistic and you should try to limit the blast radius as much as possible for when that does happen.

nottorp 2y ago
Just air gap the dev machines. But an attacker could infiltrate them by manipulating magnetic fields to generate keystrokes / mouse moves so also faraday cage them.
- mdhb 2y ago
  I don’t know why you’re trying to make this some kind of gotcha. What I said was absolutely 100% standard security advice. You’re just making yourself look silly here.

appplication 2y ago

This is a valid and significant issue, though for our internal packages running arbitrary code on execution has been a lifesaver. Think: complex distributed systems where all nodes need to be synchronized with the version on the driver.

hulitu 2y ago

> Installing packages (i.e. source code) for programming languages should not execute arbitrary code. Tell that to JS, Python and Rust people. But hey, we are secure. /s

elric 2y ago

Other languages with dependency management frameworks (Java with Maven Central) go about this a little different: you can download as many evil dependencies as you like, but unless you actively execute them, nothing will happen. There are some caveats, especially with nasty frameworks that scan the entire classpath, but that's a different issue.

juliangmp 2y ago

Don't most package managers do that? I mean yes it can be exploited but I wouldn't blame pip specifically.