7 ms·

One thing that simply HAS to happen in order to put a stop to typosquatting and related errors: FFS, package maintainers should be forced to have the name of th

One thing that simply HAS to happen in order to put a stop to typosquatting and related errors: FFS, package maintainers should be forced to have the name of the package on the repository be the same as the main import name. Quick, Python programmers, how do you find the current version of Beautiful Soup? How do you import it? How about Scikit-Learn?* Those are two of the most important packages on the ecosystem, and they make you guess as to what the actual name is. STUPID. STUPID. STUPID. * Answers: pip install beautifulsoup4, import bs4; pip install scikit-learn, import sklearn

3pt14159 6y ago
What we really need is a web-of-trust. I've been saying this for years. I know who I trust on GitHub. We even have our public keys there. It's perfectly possible to sign a release with an SSH key, even if GPG would be better. It won't solve all the issues[0], but it would solve many of these obvious, unskilled attacks and I'm getting kinda sick of the blasé attitude in OSS. [0] For example, an honest software developer can get local malware that could modify a file just before it is committed and signed.
- pjc50 6y ago
  cough Debian has enforced this for at least two decades.
  - 3pt14159 6y ago
    Yes! And there are some steps in the Ruby community to pull in a web-of-trust as well. It's just frustrating that it didn't happen from the get-go in language after language. That said, I applaud the Debian folks for putting in the work early. Not just in the web-of-trust, but in many aspects of software security.
- swsieber 6y ago
  You mean like crev? https://github.com/crev-dev/crev/ https://github.com/crev-dev/crev/ Basically a web of code reviews. There is a pup integration in the early stages per the README. I came across crev through the cargo/rust integration.
remram 6y ago
There is also nothing preventing you from installing multiple importable modules, or module names already used by somebody else, etc. It is really a behavior that most people would describe as "broken" if there wasn't a need to keep supporting the legacy of existing packages.
Constellarise 6y ago
Namespaces would solve this better and this wouldn't work in cases such as multiple loose files. It also hard breaks anything with a hyphen in the package name.
throw_m239339 6y ago
I like the go way: import a url. If you want to alias the namespace however you want afterward then that's your problem. It doesn't solve every single security issues, obviously. But at the very least the source of the code is more transparent. Second option: signed packages. A warning when a package is not digitally signed. Finally, a package repository curated by whatever language foundation. But then we are getting close to OS wide native package management... But at the very least namespace+package name should be the norm for every single package manager out there. instead of just a package name. So it should be "pip install code.launchpad.net/beautifulsoup", not just "pip install beautifulsoup4", if it's not possible already. And then import "code.launchpad.net/beautifulsoup" as beautifulsoup in the code.
- X6S1x6Okd1st 6y ago
  > Second option: signed packages. A warning when a package is not digitally signed. It's trivial to sign a malicious package with a signature. The problem is to check and make sure it's signed with the signature
  - Spivak 6y ago
    And it doesn't help typosquatting since baeutifulsoup will have a perfectly valid signature.
- TheDong 6y ago
  Sure, the go way: import ( "fmt" "github.com/dimuska139/go-email-normalizer" ) func main() { fmt.Println(emailnormalizer.NewNormalizer().Normalize("x+1@gmail.com")) } Wait, how did I know to do "emailnormalizer.NewNormalizer()"? Well, that's the package name of the package at that url. The package name is only the same as the last part of the URL in go by convention, and importing a package can put an arbitrary package identifier into your package scope.
  - LukeShu 6y ago
    FWIW, `goimports` would say "OK, technically that's valid, but I'm going to complain unless you explicitly import it as 'emailnormalizer'". import ( "fmt" emailnormalizer "github.com/dimuska139/go-email-normalizer" )
bigbubba 6y ago
It's not just Python that has this problem. In Debian the zlib package is called zlib1g. I'm not sure what the '1g' is for, but what really puzzles me is why Debian's package for the Rust bindings is called librust-libz-sys-dev with the description "libz library (also known as zlib)" I guess it's too late to fix this apparent typo? The project is called zlib, ''libz'' is simply incorrect.
- duskwuff 6y ago
  The "1g" in "zlib1g" is a version. Dynamic libraries can have multiple versions which can be installed in parallel. zlib hasn't changed much lately, so there's only really one version that matters, but this does come into play for other libraries. For example, LibUSB has libusb-0.1 and libusb-1.0, both of which are widely used. librust-libz-sys-dev is a Debian package for a Rust library that's literally called "libz-sys" [1]. Why the developers of this package decided to call it this and not "zlib" is a question for them, but Debian packages libraries based on the language's own name for the package, not its own reinterpretation of those names. [1]: https://crates.io/crates/libz-sys https://crates.io/crates/libz-sys
  - bigbubba 6y ago
    The mingw package (libz-mingw-w64) also has it backwards. As far as I can tell, that upstream gets it right ('mingw-w64-x86_64-zlib' I believe.) Also, putting version number suffixes on package names when the package system already has a concept of version numbers never sat right with me. It seems like a hack around tooling deficiencies.
    - duskwuff 6y ago
      The Debian packaging system only allows one version of a package to be installed at a time, as it assumes that a newer version of a package will contain substantially the same files as a previous version. Versioned libraries install different files, and can be installed alongside each other.
  - steveklabnik 6y ago
david2ndaccount 6y ago
Sometimes this is a feature though. PIL is abandoned but was widely used, so its replacement pillow can use the same import as it is a drop-in replacement.
- progval 6y ago
  Agreed. I maintain a fork of a package that loads plugins; and these plugins import from the original package's name. If I couldn't use the same name, users would need to edit every old plugin's code.
- RasmusLarsen 6y ago
  Wouldn't it be better to have some explicit method of aliasing your dependencies if this is actually something you want to do? It seems wrong to have that as a feature opaque like that.
  - david2ndaccount 6y ago
    It’d probably be better, but it is an accidental feature of the current system.
bobbylarrybobby 6y ago
What's especially odd is python would let you do `import beautifulsoup4 as bs4`, so there's no need to pick `bs4` for the module name. (For `scikit-learn`, I'd say restricting package names to valid python module names is fine.) Imagine if `numpy` had been named `np` or `pandas` `pd`.
- mantap 6y ago
  Considering I have written import numpy as np probably thousands of times I would be quite okay with import np.
thaumasiotes 6y ago
> Quick, Python programmers, how do you find the current version of Beautiful Soup? How do you import it? How about Scikit-Learn? > Those are two of the most important packages on the ecosystem, and they make you guess as to what the actual name is. STUPID. STUPID. STUPID. Well, my approach is to go to the project website and look for installation instructions, because searching pip is completely useless.

3pt14159 6y ago

What we really need is a web-of-trust. I've been saying this for years. I know who I trust on GitHub. We even have our public keys there. It's perfectly possible to sign a release with an SSH key, even if GPG would be better. It won't solve all the issues[0], but it would solve many of these obvious, unskilled attacks and I'm getting kinda sick of the blasé attitude in OSS. [0] For example, an honest software developer can get local malware that could modify a file just before it is committed and signed.

pjc50 6y ago
cough Debian has enforced this for at least two decades.
- 3pt14159 6y ago
  Yes! And there are some steps in the Ruby community to pull in a web-of-trust as well. It's just frustrating that it didn't happen from the get-go in language after language. That said, I applaud the Debian folks for putting in the work early. Not just in the web-of-trust, but in many aspects of software security.
swsieber 6y ago
You mean like crev? https://github.com/crev-dev/crev/ https://github.com/crev-dev/crev/ Basically a web of code reviews. There is a pup integration in the early stages per the README. I came across crev through the cargo/rust integration.

remram 6y ago

There is also nothing preventing you from installing multiple importable modules, or module names already used by somebody else, etc. It is really a behavior that most people would describe as "broken" if there wasn't a need to keep supporting the legacy of existing packages.

Constellarise 6y ago

Namespaces would solve this better and this wouldn't work in cases such as multiple loose files. It also hard breaks anything with a hyphen in the package name.

throw_m239339 6y ago

I like the go way: import a url. If you want to alias the namespace however you want afterward then that's your problem. It doesn't solve every single security issues, obviously. But at the very least the source of the code is more transparent. Second option: signed packages. A warning when a package is not digitally signed. Finally, a package repository curated by whatever language foundation. But then we are getting close to OS wide native package management... But at the very least namespace+package name should be the norm for every single package manager out there. instead of just a package name. So it should be "pip install code.launchpad.net/beautifulsoup", not just "pip install beautifulsoup4", if it's not possible already. And then import "code.launchpad.net/beautifulsoup" as beautifulsoup in the code.

X6S1x6Okd1st 6y ago
> Second option: signed packages. A warning when a package is not digitally signed. It's trivial to sign a malicious package with a signature. The problem is to check and make sure it's signed with the signature
- Spivak 6y ago
  And it doesn't help typosquatting since baeutifulsoup will have a perfectly valid signature.
TheDong 6y ago
Sure, the go way: import ( "fmt" "github.com/dimuska139/go-email-normalizer" ) func main() { fmt.Println(emailnormalizer.NewNormalizer().Normalize("x+1@gmail.com")) } Wait, how did I know to do "emailnormalizer.NewNormalizer()"? Well, that's the package name of the package at that url. The package name is only the same as the last part of the URL in go by convention, and importing a package can put an arbitrary package identifier into your package scope.
- LukeShu 6y ago
  FWIW, `goimports` would say "OK, technically that's valid, but I'm going to complain unless you explicitly import it as 'emailnormalizer'". import ( "fmt" emailnormalizer "github.com/dimuska139/go-email-normalizer" )

bigbubba 6y ago

It's not just Python that has this problem. In Debian the zlib package is called zlib1g. I'm not sure what the '1g' is for, but what really puzzles me is why Debian's package for the Rust bindings is called librust-libz-sys-dev with the description "libz library (also known as zlib)" I guess it's too late to fix this apparent typo? The project is called zlib, ''libz'' is simply incorrect.

duskwuff 6y ago
The "1g" in "zlib1g" is a version. Dynamic libraries can have multiple versions which can be installed in parallel. zlib hasn't changed much lately, so there's only really one version that matters, but this does come into play for other libraries. For example, LibUSB has libusb-0.1 and libusb-1.0, both of which are widely used. librust-libz-sys-dev is a Debian package for a Rust library that's literally called "libz-sys" [1]. Why the developers of this package decided to call it this and not "zlib" is a question for them, but Debian packages libraries based on the language's own name for the package, not its own reinterpretation of those names. [1]: https://crates.io/crates/libz-sys https://crates.io/crates/libz-sys
- bigbubba 6y ago
  The mingw package (libz-mingw-w64) also has it backwards. As far as I can tell, that upstream gets it right ('mingw-w64-x86_64-zlib' I believe.) Also, putting version number suffixes on package names when the package system already has a concept of version numbers never sat right with me. It seems like a hack around tooling deficiencies.
  - duskwuff 6y ago
    The Debian packaging system only allows one version of a package to be installed at a time, as it assumes that a newer version of a package will contain substantially the same files as a previous version. Versioned libraries install different files, and can be installed alongside each other.
- steveklabnik 6y ago

david2ndaccount 6y ago

Sometimes this is a feature though. PIL is abandoned but was widely used, so its replacement pillow can use the same import as it is a drop-in replacement.

progval 6y ago
Agreed. I maintain a fork of a package that loads plugins; and these plugins import from the original package's name. If I couldn't use the same name, users would need to edit every old plugin's code.
RasmusLarsen 6y ago
Wouldn't it be better to have some explicit method of aliasing your dependencies if this is actually something you want to do? It seems wrong to have that as a feature opaque like that.
- david2ndaccount 6y ago
  It’d probably be better, but it is an accidental feature of the current system.

bobbylarrybobby 6y ago

What's especially odd is python would let you do `import beautifulsoup4 as bs4`, so there's no need to pick `bs4` for the module name. (For `scikit-learn`, I'd say restricting package names to valid python module names is fine.) Imagine if `numpy` had been named `np` or `pandas` `pd`.

mantap 6y ago
Considering I have written import numpy as np probably thousands of times I would be quite okay with import np.

thaumasiotes 6y ago

> Quick, Python programmers, how do you find the current version of Beautiful Soup? How do you import it? How about Scikit-Learn? > Those are two of the most important packages on the ecosystem, and they make you guess as to what the actual name is. STUPID. STUPID. STUPID. Well, my approach is to go to the project website and look for installation instructions, because searching pip is completely useless.