3 ms·

This is a great writeup on a perennially misunderstood topic in Python packaging (and namespacing/module semantics)! A lot of (bad) security tools begin with th

This is a great writeup on a perennially misunderstood topic in Python packaging (and namespacing/module semantics)! A lot of (bad) security tools begin with the assumption that a top-level module name can always be reliably mapped back to its PyPI package name, and this post's data concretely dispels that assumption. It's a shame that there isn't (currently) a reliable way to perform this backwards link: the closest current things are `{dist}.dist-info/METADATA` (unreliable, entirely user controlled) and `direct_url.json` for URL-installed packages, which isn't present for packages resolved from indices. Edit: PEP 710[1] would accomplish the above, but it's still in draft. [1]: https://peps.python.org/pep-0710/ https://peps.python.org/pep-0710/

staticautomatic 2y ago
It took me what seemed like ages to figure out how to auth into Google cloud because the name of the module in their example code isn’t the name of the package. You shouldn’t have to be a detective to figure out what to pip install from looking at an import.
- woodruffw 2y ago
  I don't necessarily disagree, although note that this is true for just about every packaging ecosystem: Rust, Ruby, etc. are similar in making no guarantee that the index name is even remotely related to the importable/module name. Python gets the "worst" of it in the sense that it's big and has a large diversity of packages, but it's a general consequence of having a packaging ecosystem that's distinct from a given language's import machinery.
  - Timon3 2y ago
    This is one thing I really, really like about JavaScript - you explicitly import everything from packages using the same name you install them with. When viewing source code without a code editor, many modern languages have no way to know what comes from where. I don't understand why this seems to be the standard for new languages like Rust.
- deleted 2y ago
  [deleted]
ggm 2y ago
> This is a great writeup on a perennially misunderstood topic in Python packaging (and namespacing/module semantics)! A lot of (bad) security tools begin with the assumption that a top-level module name can always be reliably mapped back to its PyPI package name, and this post's data concretely dispels that assumption. The whole model of naming of apt install <thing> vs port install <thing> is a wargame all of it's own. Your general point is well made: how you get a distribution, and unpack and install it is quite distinct from how it names inside the language/system namespace it installs into. Even at the level of ssh vs sshd, there can be confusion. the daemon is configured from sshd_ files, but they live inside /etc/ssh alongside /etc/ssh/ssh_ files configuring the client side.

staticautomatic 2y ago

It took me what seemed like ages to figure out how to auth into Google cloud because the name of the module in their example code isn’t the name of the package. You shouldn’t have to be a detective to figure out what to pip install from looking at an import.

woodruffw 2y ago
I don't necessarily disagree, although note that this is true for just about every packaging ecosystem: Rust, Ruby, etc. are similar in making no guarantee that the index name is even remotely related to the importable/module name. Python gets the "worst" of it in the sense that it's big and has a large diversity of packages, but it's a general consequence of having a packaging ecosystem that's distinct from a given language's import machinery.
- Timon3 2y ago
  This is one thing I really, really like about JavaScript - you explicitly import everything from packages using the same name you install them with. When viewing source code without a code editor, many modern languages have no way to know what comes from where. I don't understand why this seems to be the standard for new languages like Rust.
deleted 2y ago
[deleted]

ggm 2y ago

> This is a great writeup on a perennially misunderstood topic in Python packaging (and namespacing/module semantics)! A lot of (bad) security tools begin with the assumption that a top-level module name can always be reliably mapped back to its PyPI package name, and this post's data concretely dispels that assumption. The whole model of naming of apt install <thing> vs port install <thing> is a wargame all of it's own. Your general point is well made: how you get a distribution, and unpack and install it is quite distinct from how it names inside the language/system namespace it installs into. Even at the level of ssh vs sshd, there can be confusion. the daemon is configured from sshd_ files, but they live inside /etc/ssh alongside /etc/ssh/ssh_ files configuring the client side.