3 ms·

Adding Auditing to Pip

67 points by chmaynard 4y ago

westurner 4y ago
Looks like I'm a little bit late to this article about mailing list discussion about adding ~CVE search to pip. The broader trend here is to identify the names, versions, and hashes of all software installed packages in all languages and present an SBOM [1][2]. Does/would `pip audit` also lookup CVE vulns for extension modules written in other programming languages like C, Go, and Rust; or do existing tools that also already lookup vulns for Python packages [1] https://westurner.github.io/hnlog/ https://westurner.github.io/hnlog/ ; Ctrl-F "SBOM" [2] "Existing artifact vuln scanners, databases, and specs?" https://github.com/google/osv.dev/issues/55#issue-802542447 https://github.com/google/osv.dev/issues/55#issue-802542447 FWIW, from the OSV README https://github.com/google/osv.dev https://github.com/google/osv.dev: > This is an ongoing project. We encourage open source ecosystems to adopt the OpenSSF Vulnerability format to enable open source users to easily aggregate and consume vulnerabilities across all ecosystesm. See our blog post for more details. > The following ecosystems have vulnerabilities encoded in this format: > GitHub Advisory Database (CC-BY 4.0), PyPI Advisory Database (CC-BY 4.0), Go Vulnerability Database (CC-BY 4.0), Rust Advisory Database (CC0 1.0), Global Security Database (CC0 1.0) OSS-Fuzz (CC-BY 4.0) > Together, these include vulnerabilities from: > npm, Maven, Go, NuGet, PyPI, RubyGems, crates.io, Packagist, Linux, OSS-Fuzz
- woodruffw 4y ago
  > Does/would `pip audit` also lookup CVE vulns for extension modules written in other programming languages like C, Go, and Rust For the time being, the plan is to focus `pip-audit` on vulnerabilities in Python packages themselves, not native dependencies of packages that happen to contain binary extensions. The line is, however, blurry: if a Python package is written in C or Rust and has a vulnerability therein, that would be considered a Python ecosystem vulnerability for the purposes of the PYSEC DB. A vulnerability in libssl that happens to be exposed via a Python extension, however, would not be.
  - westurner 4y ago
    So something more comprehensive for the complete SBOM for all languages and extension modules is also advisable for "[Software] Component Inventory and Risk Assessment". From the article: > The PyPA maintains an advisory database that stores vulnerabilities affecting PyPI packages in YAML format. For example, pip-audit reported that the version of Babel on my system is vulnerable to PYSEC-2021-421, which is a local code-execution flaw. That PYSEC advisory refers to CVE-2021-42771, which is how the flaw is known to the wider world. > As it turns out, my system is actually not vulnerable to CVE-2021-42771, as the Ubuntu security entry shows. The pip-audit tool looks at the version numbers of the installed PyPI package to decide which are vulnerable, but Linux distributions regularly backport fixes into earlier versions so the PyPI package version number does not tell the whole story—at least for those who get their PyPI packages from distributions rather than via pip. pypa/advisory-database: https://github.com/pypa/advisory-database https://github.com/pypa/advisory-database
woodruffw 4y ago
Oh hello, that’s my project. Happy to answer questions!
- nighthawk454 4y ago
  Thanks for your work! Reading the article, seems like the main debate is on whether this should be included in pip. Fwiw, as a user experience, including it seems best. I think the experience npm (tries to) provide is a fine goal. However, whether it's a priority for pip and burdening maintainers is another matter. In any case, I'm more concerned with not repeating npm audit's mistakes. Abramov's blog post linked in the article [1] popped to my mind right away. Can you say any more about how pip-audit won't "cry wolf"? Especially since Python doesn't have great packaging standardization around tools, dev-dependencies, build dependencies, pip-tools vs pipenv vs poetry, etc. [1] https://overreacted.io/npm-audit-broken-by-design/ https://overreacted.io/npm-audit-broken-by-design/
  - woodruffw 4y ago
    Sorry for the late response! Yes, this is a great point: we absolutely do not want either the current instantiation of pip-audit or any potential integration to cause security fatigue. Our proposed integration plan was made with this anti-goal in mind: audits will never be “automatic” the way they are in npm, meaning that users will not be suddenly put on the spot to make security decisions about their dependencies. Instead, auditing will always be an intentional action: users will have to explicitly run `pip audit` to receive notices, which they can address synchronously. Separately, on the data side: the JS ecosystem suffers from a deluge of low-value vulnerabilities (things like ReDoS weaknesses), which in turn produce more fatigue. The Python ecosystem avoids those kinds of vulnerabilities mostly incidentally, since Python is not (generally) run on the client side of the browser in the way that JS is. But more formally, our hope is that the community vulnerability database will help us balance fatigue concerns with “completeness” and expose sufficient metadata to allow users to make policy-style decisions about which potential vulnerabilities matter to them.
    - nighthawk454 4y ago
rrdharan 4y ago
I wish they would bring back pip search.

67 points by chmaynard 4y ago

Looks like I'm a little bit late to this article about mailing list discussion about adding ~CVE search to pip. The broader trend here is to identify the names, versions, and hashes of all software installed packages in all languages and present an SBOM [1][2]. Does/would `pip audit` also lookup CVE vulns for extension modules written in other programming languages like C, Go, and Rust; or do existing tools that also already lookup vulns for Python packages [1] https://westurner.github.io/hnlog/ https://westurner.github.io/hnlog/ ; Ctrl-F "SBOM" [2] "Existing artifact vuln scanners, databases, and specs?" https://github.com/google/osv.dev/issues/55#issue-802542447 https://github.com/google/osv.dev/issues/55#issue-802542447 FWIW, from the OSV README https://github.com/google/osv.dev https://github.com/google/osv.dev: > This is an ongoing project. We encourage open source ecosystems to adopt the OpenSSF Vulnerability format to enable open source users to easily aggregate and consume vulnerabilities across all ecosystesm. See our blog post for more details. > The following ecosystems have vulnerabilities encoded in this format: > GitHub Advisory Database (CC-BY 4.0), PyPI Advisory Database (CC-BY 4.0), Go Vulnerability Database (CC-BY 4.0), Rust Advisory Database (CC0 1.0), Global Security Database (CC0 1.0) OSS-Fuzz (CC-BY 4.0) > Together, these include vulnerabilities from: > npm, Maven, Go, NuGet, PyPI, RubyGems, crates.io, Packagist, Linux, OSS-Fuzz

woodruffw 4y ago
> Does/would `pip audit` also lookup CVE vulns for extension modules written in other programming languages like C, Go, and Rust For the time being, the plan is to focus `pip-audit` on vulnerabilities in Python packages themselves, not native dependencies of packages that happen to contain binary extensions. The line is, however, blurry: if a Python package is written in C or Rust and has a vulnerability therein, that would be considered a Python ecosystem vulnerability for the purposes of the PYSEC DB. A vulnerability in libssl that happens to be exposed via a Python extension, however, would not be.
- westurner 4y ago
  So something more comprehensive for the complete SBOM for all languages and extension modules is also advisable for "[Software] Component Inventory and Risk Assessment". From the article: > The PyPA maintains an advisory database that stores vulnerabilities affecting PyPI packages in YAML format. For example, pip-audit reported that the version of Babel on my system is vulnerable to PYSEC-2021-421, which is a local code-execution flaw. That PYSEC advisory refers to CVE-2021-42771, which is how the flaw is known to the wider world. > As it turns out, my system is actually not vulnerable to CVE-2021-42771, as the Ubuntu security entry shows. The pip-audit tool looks at the version numbers of the installed PyPI package to decide which are vulnerable, but Linux distributions regularly backport fixes into earlier versions so the PyPI package version number does not tell the whole story—at least for those who get their PyPI packages from distributions rather than via pip. pypa/advisory-database: https://github.com/pypa/advisory-database https://github.com/pypa/advisory-database

woodruffw 4y ago

Oh hello, that’s my project. Happy to answer questions!

nighthawk454 4y ago
Thanks for your work! Reading the article, seems like the main debate is on whether this should be included in pip. Fwiw, as a user experience, including it seems best. I think the experience npm (tries to) provide is a fine goal. However, whether it's a priority for pip and burdening maintainers is another matter. In any case, I'm more concerned with not repeating npm audit's mistakes. Abramov's blog post linked in the article [1] popped to my mind right away. Can you say any more about how pip-audit won't "cry wolf"? Especially since Python doesn't have great packaging standardization around tools, dev-dependencies, build dependencies, pip-tools vs pipenv vs poetry, etc. [1] https://overreacted.io/npm-audit-broken-by-design/ https://overreacted.io/npm-audit-broken-by-design/
- woodruffw 4y ago
  Sorry for the late response! Yes, this is a great point: we absolutely do not want either the current instantiation of pip-audit or any potential integration to cause security fatigue. Our proposed integration plan was made with this anti-goal in mind: audits will never be “automatic” the way they are in npm, meaning that users will not be suddenly put on the spot to make security decisions about their dependencies. Instead, auditing will always be an intentional action: users will have to explicitly run `pip audit` to receive notices, which they can address synchronously. Separately, on the data side: the JS ecosystem suffers from a deluge of low-value vulnerabilities (things like ReDoS weaknesses), which in turn produce more fatigue. The Python ecosystem avoids those kinds of vulnerabilities mostly incidentally, since Python is not (generally) run on the client side of the browser in the way that JS is. But more formally, our hope is that the community vulnerability database will help us balance fatigue concerns with “completeness” and expose sufficient metadata to allow users to make policy-style decisions about which potential vulnerabilities matter to them.
  - nighthawk454 4y ago

rrdharan 4y ago

I wish they would bring back pip search.