6 ms·

A study of malicious code in PyPI ecosystem

63 points by Dowwie 3y ago

siddheshgunjal 3y ago
So, is there a way to check if any malicious packages installed in my conda environment?
- ashishbijlani 3y ago
  I've been building Packj [1] to detect dummy, malicious, abandoned, typo-squatting, and other "risky" PyPI/NPM/Ruby/PHP/Maven/Rust packages. It carries out static/dynamic/metadata analysis and scans for 40+ attributes such as num funcs/files, spawning of shell, use of SSH keys, network communication, use of decode+eval, mismatch of GitHub code vs packaged code (provenance), change in APIs across versions, etc. to flag risky packages. 1. https://github.com/ossillate-inc/packj https://github.com/ossillate-inc/packj
  - siddheshgunjal 3y ago
    This is interesting! Implementing it today itself for my open source package development pipeline!
  - thenerdhead 3y ago
    Cool project. How do you feel about projects like OpenSSF scorecards or even the checks that socket.dev do today on these packages to help determine risk? https://github.com/ossillate-inc/packj/blob/main/.packj.yaml https://github.com/ossillate-inc/packj/blob/main/.packj.yaml Secondly, what about impersonation where attackers imitate a popular package and its respective metadata?
    - ashishbijlani 3y ago
      Thanks! We need more such efforts to improve supply-chain security of open-source software. Packj detects typo-squatting (impersonation) as well.
louislang 3y ago
One of the sources referenced in the paper is about the work the company I co-founded is doing (https://phylum.io https://phylum.io). We've been working closely with PyPI to not only report issues related to malware, but are also helping provide guidance around on-going work to help curb additional malware in the ecosystem. I think this is a step in the right direction, and one I hope other ecosystems will eventually take. Though, this is likely to be a cat and mouse game for the foreseeable future. Detection will get better, and attackers will change tactics. In the meantime, we've been open-sourcing some tooling to help protect developers from these sorts of attacks. Namely, a sandbox that locks down network/disk/env [1] and our CLI [2] that allows you to perform a `pip` install in the sandbox, after checking our API for behaviors/issues with the package. For example: phylum pip install <pkgName> Really glad to see software supply chain security getting some academic, rigorous study. Backstabbers Knife was one of the first I came across, and it's been a consistent stream of papers since. 1. https://github.com/phylum-dev/birdcage https://github.com/phylum-dev/birdcage 2. https://github.com/phylum-dev/cli https://github.com/phylum-dev/cli
- thenerdhead 3y ago
  > Though, this is likely to be a cat and mouse game for the foreseeable future. Detection will get better, and attackers will change tactics. How do you see detection ever getting better to the point where it isn't a game of cat and mouse? Isn't the whole point of these security threats that go undetected is that they are novel attacks? Or even popularized attacks with different parameters? In these trust but verify models of a central registry, it will require some heavy lifting like AI security detection to even keep up with. That seems like a new problem in itself if introduced.
angry_octet 3y ago
During mirroring recently I noticed a new pypi package with such a long name that it caused "file name too long" errors on artifactory and with bandersnatch saving to local risk. In this case the code handled the situation correctly (if python throwing an error con be considered correct handling) but it could well have been an exploit mechanism. Anything touching the internet where user provided content will be consumed should be hardened and preferably sandboxed in some way, stuck in an unprivileged DMZ, and monitored. As for the pypi modules themselves, a web of trust system might actually work. For scientific packages especially. There's always the risk of an author getting their machine compromised and someone pushing an update with either a subtle bug which can pass unnoticed or a loud and quickly detected attack. Making a change in account 2FA tokens visible would be a great start, as would integrating signing keys. (The long file name in this case was the digits of pi.)
thenerdhead 3y ago
Am I reading this correctly? There's only ~2k/470k packages containing malware on the PyPi registry? That seems exceptional. The registry maintainers must be working extremely hard on these problems.
spenrose 3y ago
"We conducted an empirical study to understand the characteristics and current state of the malicious code lifecycle in the PyPI ecosystem. We first built an automated data collection framework and collated a multi-source malicious code dataset containing 4,669 malicious package files. We preliminarily classified these malicious code into five categories based on malicious behaviour characteristics. Our research found that over 50% of malicious code exhibits multiple malicious behaviours, with information stealing and command execution being particularly prevalent. In addition, we observed several novel attack vectors and anti-detection techniques. Our analysis revealed that 74.81% of all malicious packages successfully entered end-user projects through source code installation, thereby increasing security risks. A real-world investigation showed that many reported malicious packages persist in PyPI mirror servers globally, with over 72% remaining for an extended period after being discovered. Finally, we sketched a portrait of the malicious code lifecycle in the PyPI ecosystem, effectively reflecting the characteristics of malicious code at different stages. We also present some suggested mitigations to improve the security of the Python open-source ecosystem."
simonw 3y ago
There's a website to accompany this which is more readable than the paper: https://sites.google.com/view/pypiempircal https://sites.google.com/view/pypiempircal (At least I think it's to accompany this paper, I found it in a footnote and it's not entirely clear if it's officially related or not.)
7373737373 3y ago
It's (partially) a fundamental problem with Python and (all?) other popular programming languages. The majority of libraries don't need more authority than doing (some) computation, yet any Python script can access anything and everything by default. The https://en.wikipedia.org/wiki/Principle_of_least_privilege https://en.wikipedia.org/wiki/Principle_of_least_privilege as constructed by https://en.wikipedia.org/wiki/Capability-based_security https://en.wikipedia.org/wiki/Capability-based_security is the solution for this, yet Python will probably never be capable of this kind of internal encapsulation, it's too much of a fundamental change - and even if some sort of sandboxing ability is accomplished, creating separate/recursive sandboxes (needed when importing more, separate libraries which in turn import libraries whose authority they want to limit) will probably require another interpreter instance for each sandbox (as with WebAssembly). I hope current and future language designers will take this into account, and construct their compilers, virtual machines and interpreters accordingly. Python was created before the internet as we know it now existed, so perhaps its lack of security mechanisms shouldn't be surprising. But it and any new developments that fail to consider this aspect of computation will be fundamentally flawed from the beginning. https://github.com/void4/notes/issues/41 https://github.com/void4/notes/issues/41
- simonw 3y ago
  This is why I'm so interested in running Python (and almost every other language) inside a WebAssembly sandbox. It's 2023. I never want to run untrusted code on any of my devices outside of a sandbox ever again. WebAssembly feels like it should be that sandbox. Every few months I check to see if it's easy enough to run regular Python/JavaScript/etc code inside a WebAssembly sandbox yet... we're not quite there but it feels really close. More notes here: https://til.simonwillison.net/webassembly/python-in-a-wasm-sandbox https://til.simonwillison.net/webassembly/python-in-a-wasm-s...
  - mhh__ 3y ago
    WebAssembly as a platform is probably as good as any other, but I think as an ISA and compilation target they flubbed a lot of the basics. It's slightly too clever, slightly too awkward to target seamlessly etc
woodruffw 3y ago
Two interesting takeaways from the paper itself[1]: Chinese companies and universities appear to make extensive use of PyPI mirrors (references 23-26), and those mirrors consistently fail to purge malicious packages in a timely manner. [1]: https://about.honywen.com/publication/2023ase/2023ASE.pdf https://about.honywen.com/publication/2023ase/2023ASE.pdf
- ashishbijlani 3y ago
  Interesting. I wonder if this is also true for available/existing commercial/open-source PyPI mirroring solutions [Edited].
  - blueflow 3y ago
    Why would the Chinese have their own pypi mirror software? I think they are running the same software as western companies.
    - pxeger1 3y ago
      Probably because accessing PyPI directly is slow, because of the Great Firewall.
      - woodruffw 3y ago
        I think the GP is asking why Chinese engineers would have their own software for mirroring, not why they'd have their own mirrors. (I believe most Chinese mirrors use bandersnatch[1], like most non-Chinese mirrors.) [1]: https://github.com/pypa/bandersnatch https://github.com/pypa/bandersnatch

63 points by Dowwie 3y ago

siddheshgunjal 3y ago

So, is there a way to check if any malicious packages installed in my conda environment?

ashishbijlani 3y ago
I've been building Packj [1] to detect dummy, malicious, abandoned, typo-squatting, and other "risky" PyPI/NPM/Ruby/PHP/Maven/Rust packages. It carries out static/dynamic/metadata analysis and scans for 40+ attributes such as num funcs/files, spawning of shell, use of SSH keys, network communication, use of decode+eval, mismatch of GitHub code vs packaged code (provenance), change in APIs across versions, etc. to flag risky packages. 1. https://github.com/ossillate-inc/packj https://github.com/ossillate-inc/packj
- siddheshgunjal 3y ago
  This is interesting! Implementing it today itself for my open source package development pipeline!
- thenerdhead 3y ago
  Cool project. How do you feel about projects like OpenSSF scorecards or even the checks that socket.dev do today on these packages to help determine risk? https://github.com/ossillate-inc/packj/blob/main/.packj.yaml https://github.com/ossillate-inc/packj/blob/main/.packj.yaml Secondly, what about impersonation where attackers imitate a popular package and its respective metadata?
  - ashishbijlani 3y ago
    Thanks! We need more such efforts to improve supply-chain security of open-source software. Packj detects typo-squatting (impersonation) as well.

louislang 3y ago

One of the sources referenced in the paper is about the work the company I co-founded is doing (https://phylum.io https://phylum.io). We've been working closely with PyPI to not only report issues related to malware, but are also helping provide guidance around on-going work to help curb additional malware in the ecosystem. I think this is a step in the right direction, and one I hope other ecosystems will eventually take. Though, this is likely to be a cat and mouse game for the foreseeable future. Detection will get better, and attackers will change tactics. In the meantime, we've been open-sourcing some tooling to help protect developers from these sorts of attacks. Namely, a sandbox that locks down network/disk/env [1] and our CLI [2] that allows you to perform a `pip` install in the sandbox, after checking our API for behaviors/issues with the package. For example: phylum pip install <pkgName> Really glad to see software supply chain security getting some academic, rigorous study. Backstabbers Knife was one of the first I came across, and it's been a consistent stream of papers since. 1. https://github.com/phylum-dev/birdcage https://github.com/phylum-dev/birdcage 2. https://github.com/phylum-dev/cli https://github.com/phylum-dev/cli

thenerdhead 3y ago
> Though, this is likely to be a cat and mouse game for the foreseeable future. Detection will get better, and attackers will change tactics. How do you see detection ever getting better to the point where it isn't a game of cat and mouse? Isn't the whole point of these security threats that go undetected is that they are novel attacks? Or even popularized attacks with different parameters? In these trust but verify models of a central registry, it will require some heavy lifting like AI security detection to even keep up with. That seems like a new problem in itself if introduced.

angry_octet 3y ago

During mirroring recently I noticed a new pypi package with such a long name that it caused "file name too long" errors on artifactory and with bandersnatch saving to local risk. In this case the code handled the situation correctly (if python throwing an error con be considered correct handling) but it could well have been an exploit mechanism. Anything touching the internet where user provided content will be consumed should be hardened and preferably sandboxed in some way, stuck in an unprivileged DMZ, and monitored. As for the pypi modules themselves, a web of trust system might actually work. For scientific packages especially. There's always the risk of an author getting their machine compromised and someone pushing an update with either a subtle bug which can pass unnoticed or a loud and quickly detected attack. Making a change in account 2FA tokens visible would be a great start, as would integrating signing keys. (The long file name in this case was the digits of pi.)

thenerdhead 3y ago

Am I reading this correctly? There's only ~2k/470k packages containing malware on the PyPi registry? That seems exceptional. The registry maintainers must be working extremely hard on these problems.

spenrose 3y ago

"We conducted an empirical study to understand the characteristics and current state of the malicious code lifecycle in the PyPI ecosystem. We first built an automated data collection framework and collated a multi-source malicious code dataset containing 4,669 malicious package files. We preliminarily classified these malicious code into five categories based on malicious behaviour characteristics. Our research found that over 50% of malicious code exhibits multiple malicious behaviours, with information stealing and command execution being particularly prevalent. In addition, we observed several novel attack vectors and anti-detection techniques. Our analysis revealed that 74.81% of all malicious packages successfully entered end-user projects through source code installation, thereby increasing security risks. A real-world investigation showed that many reported malicious packages persist in PyPI mirror servers globally, with over 72% remaining for an extended period after being discovered. Finally, we sketched a portrait of the malicious code lifecycle in the PyPI ecosystem, effectively reflecting the characteristics of malicious code at different stages. We also present some suggested mitigations to improve the security of the Python open-source ecosystem."

simonw 3y ago

There's a website to accompany this which is more readable than the paper: https://sites.google.com/view/pypiempircal https://sites.google.com/view/pypiempircal (At least I think it's to accompany this paper, I found it in a footnote and it's not entirely clear if it's officially related or not.)

7373737373 3y ago

It's (partially) a fundamental problem with Python and (all?) other popular programming languages. The majority of libraries don't need more authority than doing (some) computation, yet any Python script can access anything and everything by default. The https://en.wikipedia.org/wiki/Principle_of_least_privilege https://en.wikipedia.org/wiki/Principle_of_least_privilege as constructed by https://en.wikipedia.org/wiki/Capability-based_security https://en.wikipedia.org/wiki/Capability-based_security is the solution for this, yet Python will probably never be capable of this kind of internal encapsulation, it's too much of a fundamental change - and even if some sort of sandboxing ability is accomplished, creating separate/recursive sandboxes (needed when importing more, separate libraries which in turn import libraries whose authority they want to limit) will probably require another interpreter instance for each sandbox (as with WebAssembly). I hope current and future language designers will take this into account, and construct their compilers, virtual machines and interpreters accordingly. Python was created before the internet as we know it now existed, so perhaps its lack of security mechanisms shouldn't be surprising. But it and any new developments that fail to consider this aspect of computation will be fundamentally flawed from the beginning. https://github.com/void4/notes/issues/41 https://github.com/void4/notes/issues/41

simonw 3y ago
This is why I'm so interested in running Python (and almost every other language) inside a WebAssembly sandbox. It's 2023. I never want to run untrusted code on any of my devices outside of a sandbox ever again. WebAssembly feels like it should be that sandbox. Every few months I check to see if it's easy enough to run regular Python/JavaScript/etc code inside a WebAssembly sandbox yet... we're not quite there but it feels really close. More notes here: https://til.simonwillison.net/webassembly/python-in-a-wasm-sandbox https://til.simonwillison.net/webassembly/python-in-a-wasm-s...
- mhh__ 3y ago
  WebAssembly as a platform is probably as good as any other, but I think as an ISA and compilation target they flubbed a lot of the basics. It's slightly too clever, slightly too awkward to target seamlessly etc

woodruffw 3y ago

Two interesting takeaways from the paper itself[1]: Chinese companies and universities appear to make extensive use of PyPI mirrors (references 23-26), and those mirrors consistently fail to purge malicious packages in a timely manner. [1]: https://about.honywen.com/publication/2023ase/2023ASE.pdf https://about.honywen.com/publication/2023ase/2023ASE.pdf

ashishbijlani 3y ago
Interesting. I wonder if this is also true for available/existing commercial/open-source PyPI mirroring solutions [Edited].
- blueflow 3y ago
  Why would the Chinese have their own pypi mirror software? I think they are running the same software as western companies.
  - pxeger1 3y ago
    Probably because accessing PyPI directly is slow, because of the Great Firewall.
    - woodruffw 3y ago
      I think the GP is asking why Chinese engineers would have their own software for mirroring, not why they'd have their own mirrors. (I believe most Chinese mirrors use bandersnatch[1], like most non-Chinese mirrors.) [1]: https://github.com/pypa/bandersnatch https://github.com/pypa/bandersnatch