7 ms·

I use “cargo … —-locked” to install things using the dependencies from the Cargo.lock file, which includes specific commit hashes for dependencies. Avoids thing

I use “cargo … —-locked” to install things using the dependencies from the Cargo.lock file, which includes specific commit hashes for dependencies. Avoids things like the 0.0.1 problem or even replaced crates. Need to be careful to watch for actual security updates, though. I really wish crates.io would have at least launched with a name spacing feature. This wouldn’t solve every spoofing or typosquatting issue, but it would go a long way toward improving the situation. There’s a separate issue of crates.io squatting. One person famously registered hundreds (or thousands? Tens of thousands) of common words as crate names on crates.io and has been squatting them ever since. Those names are effectively unavailable for use but also completely useless because they don’t contain anything. It’s also becoming a huge problem for abandoned crates. New forks have to choose a completely different, less intuitive name because they can’t just namespace their alternative. As old crates get abandoned, this leads to weird situations where the newest, best maintained crate has the least obvious crate name. It takes work to find the good crate some times because the best named crates might just be the oldest, most abandoned ones

Lifelarper 5y ago
https://users.rust-lang.org/t/name-squatting-on-the-crates-io/42415 https://users.rust-lang.org/t/name-squatting-on-the-crates-i... This a very old discussion if you keep digging through the links.
- ansible 5y ago
  I hope that the Rust Foundation (which finally exists) can put in some time / resources to help fix this. I totally get that maintaining crates.io is a time suck for all involved, and mediating disputes would increase that even more. But at some point, this is going to need to be addressed.
belter 5y ago
I think if somebody decides to act as a vandal I can too? https://crates.io/users/swmon https://crates.io/users/swmon Edit: Lots of love for this user in the thread.... https://github.com/swmon/Charles-Crack/pull/1 https://github.com/swmon/Charles-Crack/pull/1
- brabel 5y ago
  While I can agree it's a dickhead thing to do, it seems to be entirely within the rules of crates.io, so I think the hate should go towards them, not someone who's just playing by the rules. Domain squatting has been a thing for decades, it's not like no one expected this kind of thing would happen. Just use org/author namespaces for crying out loud.
  - kibwen 5y ago
    I am a mild proponent of namespaces. But adding them means you are now managing an identity layer, which is not something that a volunteer organization wants to deal with. In a flat namespace, users don't expect that a google-foo library is actually from Google. But with namespaces, people expect that everything under the google/ namespace actually is officially supported by Google. So when the initial rush to register names happens, who verifies that Google owns the google/ namespace? And not just for Google, but for every litigous trademark-defending organization under the sun? And since the package repository is supposed to be immutable, you don't want to wantonly delete crates from namespaces that they originally registered. Likewise, you can't go transferring ownership of a namespace from one entity to another, because that violates the security assumptions of your users who originally trusted the original owner of the namespace and not the new owner. Identity layers are hard. It's a problem worth solving, but it's not going to happen on a volunteer basis. It needs full-time paid employees to deal with this. Perhaps the Rust Foundation will make such a thing a priority.
    - brabel 5y ago
      These problems you raise have been solved problems for a long time. Maven Central uses verified reversed domain names, so to own `com.google` you need to prove you own the domain `google.com`. It's not hard to do it. Dart's pub does the same thing with verified publishers. crates.io already requires user accounts from publishers, so it already manages identity. The only reason they don't require namespaces is to make crate names look cooler and the barrier to entry non-existen, so they can amass large numbers of packages in a short time (then deal with the fallout only when it becomes an emergency), I can't see any other reason anywhere, including from the ones you've raised.
ziml77 5y ago
The lack of namespacing in crates.io has always been a strange decision to me. I hope at some point they decide that it's worth it to introduce namespaces. The could use some an aliasing mechanism to avoid breaking any existing package references.
- jamincan 5y ago
  From previous times this has come up, I was led to believe that it's not so much that the maintainers don't want namespaces as it's a thankless and overwhelming task just maintaining the status quo, and so they don't have the time and resources to implement something. Hopefully this is one of those issues that the Rust Foundation can direct some funding toward as the entire Rust community would be all the better for it.
  - jiggawatts 5y ago
    This is passing the buck in the worst possible way. Not implementing namespaces is fantastically lazy, to the point of outright professional negligence. If you can't be bothered to lift a finger to type in the few hundred lines of code it takes to implement namespaces, then by definition you're not responsible enough to run a public package repository. Do it properly or don't do it at all.
    - Shikadi 5y ago
      There's a lot more to it than a few lines of code, see: other comments on this post
      - jiggawatts 5y ago
        There really isn't, and everyone else is just making excuses. Whatever the effort level is, it is certainly much lower than the thousands of Rust users navigating a tangled mess of typo squatting, name conflicts, crate spam, and difficulty in identification of official or related crates. Again: If the crates.io team doesn't have even just this much time, then by definition they don't have the time to manage the site responsibly. They should shut it down, seek funding, or hand it over to someone else. Doing anything else is irresponsible.
moojd 5y ago
This one is frustrating because this is an issue that has been solved many times before and I hate seeing it repeated in every new package manager. A vendor name should always be required and the top level should be reserved for official/standard packages. I want all of the following from a package manager: 1. Required vendor/namespace for third party packages 2. No multiple package versions. If there is a version conflict between transitive dependencies of a package because of semver, you should not be able to install that package. 3. Lock file and a separate 'install' command for installing the locked versions and an 'upgrade' command for updating versions via semver 4. Upgrade command should support a --dry-run option that lists the packages and versions that are to be updated and a --diff that lets you preview the code changes.
- pcwalton 5y ago
  Why are multiple package versions a problem? In Rust they Just Work.
  - stubish 5y ago
    It can mess up when you are dealing with global resources. ie. two different versions of the same package both attempting to initialize global state (memory, filesystem, databases, whatever) will usually conflict with itself. Often in ways that will crash or even cause data loss.
    - pcwalton 5y ago
      I've never had that happen in Rust because multiple versions of a crate effectively live in completely separate worlds: they have their own copies of globals, for example. Can you name an example of multiple versions conflicting at runtime in Rust?
      - stubish 5y ago
        By globals, I mean global resources outside of the codes namespace. It may not even be a resource in the process, such as a log file or temporary directory or a database. If you have two versions of a crate in their completely separate worlds, and call both of their init_logging() functions to log to a file specified by an environment variable, things are likely to go pear shaped when they stomp over each others log file. I'm a Rust novice, but the example I tripped over in Go was https://github.com/golang/glog https://github.com/golang/glog. It has a module level init() initialization routine that makes calls to the stdlib flags package, manipulating the default command line flags (a global resource). If you ended up with multiple versions of glog via transient dependencies, your program would panic on startup as the second version's init() would make calls only allowed to be called once. Rust thankfully avoids this particular one by requiring initialization to be called by main() (apart from the hack described in the article).
rectang 5y ago
My understanding is that validating identity for package authors is a hard problem thus expensive to solve robustly, and the crates.io folks have hitherto deferred tackling it in earnest. That is arguably a responsible approach up to a point, in that they haven't committed prematurely to something half-baked. As described elsethread, there is prior art — Maven's identity verification is substantially better: https://news.ycombinator.com/item?id=29266591 https://news.ycombinator.com/item?id=29266591 Validating ownership of a namespace reliably enough that it is difficult to spoof is tough. It's possible for PGP creds to be stolen. But then at least the keys can be revoked, and old packages signed with a new key.
- miohtama 5y ago
  I believe Debian Maintainer process and its keyring is one, if not only, project that gets this as good as it can get. https://wiki.debian.org/DebianMaintainer https://wiki.debian.org/DebianMaintainer
- PragmaticPulp 5y ago
  They don’t need to validate anything, though. Just treat the namespace as something that can be claimed exactly the way that a crate name can be claimed. Only owners of the namespace can add crates to that namespace, in the same way that only people who own a crate name can publish to that crate.
  - kibwen 5y ago
    If you're not validating anything, then that violates the assumptions of users who see an amazon/ namespace that isn't officially associated with Amazon. We don't even need to presume malicious or squatting intent; the entire left-pad fiasco was precipitated by a person who innocently registered "kik", and then later a company named Kik demanded that NPM transfer ownership to them. This happened with a package, but it would just as easily happen with a namespace. Once you introduce an identity layer, people start to expect identity validation. It's a problem worth solving, but crates.io would need to move beyond being merely staffed by part-time volunteers to solve it.
mikepurvis 5y ago
I feel like "single shared namespace" works best when the ecosystem is managed collaboratively, with a clear leadership to make calls on who gets what name— for example, something like a Linux distro, where there are Replaces/Provides metadata specifically to facilitate these kinds of transitions and avoid being stuck forever with crappy legacy nonsense. But this doesn't work at all in a free-for-all environment like PyPI, NPM, or Crates, where anyone can just grab a name and then have it in perpetuity. IMO the Docker ecosystem got this right, with baking in a domain name as part of the container, and insisting that everyone on docker.io use a vendor/product convention. This meant that the toplevel namespace was reserved for them to offer (or delegate the offering of) specific blessed container images, much more in line with how distro packaging might work. # Get the "official" image, whatever that means (but you trust Docker Inc, so yay). docker pull nginx # Get an image supplied by a specific vendor. docker pull bitnami/nginx # Get an image from a different server altogether; maybe it's your company, or you don't trust Docker Inc after all? docker pull quay.io/jitesoft/nginx Maybe the big-flat-namespace thing is still a years-later reaction against huge and unnecessary hierarchies in Java land? I think the ideal is not to permit infinite depth, but perhaps to insist on 2-3 levels.
- dmurray 5y ago
  Github generally has two levels (organization and repository) which works pretty well.
  - hollerith 5y ago
    It works pretty well, but might not if making org2/repo55 a copy of org1/repo55 (i.e., forking) weren't free for the user (namely, org2) and maybe a venture-backed startup can afford to keep it free for the user, but a non-profit cannot.
    - xfer 5y ago
      Forking doesn't cost github anything it's a git branch of parent repo. That's why you could see people accessing youtube-dl via a commit hash of github dmca repo.
nine_k 5y ago
It's heartbreaking to see crates.io repeat nearly every mistake which npm has committed. I did not expect that from the Rust crowd which usually tends to design things carefully and with correctness and corner cases in mind.

Lifelarper 5y ago

https://users.rust-lang.org/t/name-squatting-on-the-crates-io/42415 https://users.rust-lang.org/t/name-squatting-on-the-crates-i... This a very old discussion if you keep digging through the links.

ansible 5y ago
I hope that the Rust Foundation (which finally exists) can put in some time / resources to help fix this. I totally get that maintaining crates.io is a time suck for all involved, and mediating disputes would increase that even more. But at some point, this is going to need to be addressed.

belter 5y ago

I think if somebody decides to act as a vandal I can too? https://crates.io/users/swmon https://crates.io/users/swmon Edit: Lots of love for this user in the thread.... https://github.com/swmon/Charles-Crack/pull/1 https://github.com/swmon/Charles-Crack/pull/1

brabel 5y ago
While I can agree it's a dickhead thing to do, it seems to be entirely within the rules of crates.io, so I think the hate should go towards them, not someone who's just playing by the rules. Domain squatting has been a thing for decades, it's not like no one expected this kind of thing would happen. Just use org/author namespaces for crying out loud.
- kibwen 5y ago
  I am a mild proponent of namespaces. But adding them means you are now managing an identity layer, which is not something that a volunteer organization wants to deal with. In a flat namespace, users don't expect that a google-foo library is actually from Google. But with namespaces, people expect that everything under the google/ namespace actually is officially supported by Google. So when the initial rush to register names happens, who verifies that Google owns the google/ namespace? And not just for Google, but for every litigous trademark-defending organization under the sun? And since the package repository is supposed to be immutable, you don't want to wantonly delete crates from namespaces that they originally registered. Likewise, you can't go transferring ownership of a namespace from one entity to another, because that violates the security assumptions of your users who originally trusted the original owner of the namespace and not the new owner. Identity layers are hard. It's a problem worth solving, but it's not going to happen on a volunteer basis. It needs full-time paid employees to deal with this. Perhaps the Rust Foundation will make such a thing a priority.
  - brabel 5y ago
    These problems you raise have been solved problems for a long time. Maven Central uses verified reversed domain names, so to own `com.google` you need to prove you own the domain `google.com`. It's not hard to do it. Dart's pub does the same thing with verified publishers. crates.io already requires user accounts from publishers, so it already manages identity. The only reason they don't require namespaces is to make crate names look cooler and the barrier to entry non-existen, so they can amass large numbers of packages in a short time (then deal with the fallout only when it becomes an emergency), I can't see any other reason anywhere, including from the ones you've raised.

ziml77 5y ago

The lack of namespacing in crates.io has always been a strange decision to me. I hope at some point they decide that it's worth it to introduce namespaces. The could use some an aliasing mechanism to avoid breaking any existing package references.

jamincan 5y ago
From previous times this has come up, I was led to believe that it's not so much that the maintainers don't want namespaces as it's a thankless and overwhelming task just maintaining the status quo, and so they don't have the time and resources to implement something. Hopefully this is one of those issues that the Rust Foundation can direct some funding toward as the entire Rust community would be all the better for it.
- jiggawatts 5y ago
  This is passing the buck in the worst possible way. Not implementing namespaces is fantastically lazy, to the point of outright professional negligence. If you can't be bothered to lift a finger to type in the few hundred lines of code it takes to implement namespaces, then by definition you're not responsible enough to run a public package repository. Do it properly or don't do it at all.
  - Shikadi 5y ago
    There's a lot more to it than a few lines of code, see: other comments on this post
    - jiggawatts 5y ago
      There really isn't, and everyone else is just making excuses. Whatever the effort level is, it is certainly much lower than the thousands of Rust users navigating a tangled mess of typo squatting, name conflicts, crate spam, and difficulty in identification of official or related crates. Again: If the crates.io team doesn't have even just this much time, then by definition they don't have the time to manage the site responsibly. They should shut it down, seek funding, or hand it over to someone else. Doing anything else is irresponsible.

moojd 5y ago

This one is frustrating because this is an issue that has been solved many times before and I hate seeing it repeated in every new package manager. A vendor name should always be required and the top level should be reserved for official/standard packages. I want all of the following from a package manager: 1. Required vendor/namespace for third party packages 2. No multiple package versions. If there is a version conflict between transitive dependencies of a package because of semver, you should not be able to install that package. 3. Lock file and a separate 'install' command for installing the locked versions and an 'upgrade' command for updating versions via semver 4. Upgrade command should support a --dry-run option that lists the packages and versions that are to be updated and a --diff that lets you preview the code changes.

pcwalton 5y ago
Why are multiple package versions a problem? In Rust they Just Work.
- stubish 5y ago
  It can mess up when you are dealing with global resources. ie. two different versions of the same package both attempting to initialize global state (memory, filesystem, databases, whatever) will usually conflict with itself. Often in ways that will crash or even cause data loss.
  - pcwalton 5y ago
    I've never had that happen in Rust because multiple versions of a crate effectively live in completely separate worlds: they have their own copies of globals, for example. Can you name an example of multiple versions conflicting at runtime in Rust?
    - stubish 5y ago
      By globals, I mean global resources outside of the codes namespace. It may not even be a resource in the process, such as a log file or temporary directory or a database. If you have two versions of a crate in their completely separate worlds, and call both of their init_logging() functions to log to a file specified by an environment variable, things are likely to go pear shaped when they stomp over each others log file. I'm a Rust novice, but the example I tripped over in Go was https://github.com/golang/glog https://github.com/golang/glog. It has a module level init() initialization routine that makes calls to the stdlib flags package, manipulating the default command line flags (a global resource). If you ended up with multiple versions of glog via transient dependencies, your program would panic on startup as the second version's init() would make calls only allowed to be called once. Rust thankfully avoids this particular one by requiring initialization to be called by main() (apart from the hack described in the article).

rectang 5y ago

My understanding is that validating identity for package authors is a hard problem thus expensive to solve robustly, and the crates.io folks have hitherto deferred tackling it in earnest. That is arguably a responsible approach up to a point, in that they haven't committed prematurely to something half-baked. As described elsethread, there is prior art — Maven's identity verification is substantially better: https://news.ycombinator.com/item?id=29266591 https://news.ycombinator.com/item?id=29266591 Validating ownership of a namespace reliably enough that it is difficult to spoof is tough. It's possible for PGP creds to be stolen. But then at least the keys can be revoked, and old packages signed with a new key.

miohtama 5y ago
I believe Debian Maintainer process and its keyring is one, if not only, project that gets this as good as it can get. https://wiki.debian.org/DebianMaintainer https://wiki.debian.org/DebianMaintainer
PragmaticPulp 5y ago
They don’t need to validate anything, though. Just treat the namespace as something that can be claimed exactly the way that a crate name can be claimed. Only owners of the namespace can add crates to that namespace, in the same way that only people who own a crate name can publish to that crate.
- kibwen 5y ago
  If you're not validating anything, then that violates the assumptions of users who see an amazon/ namespace that isn't officially associated with Amazon. We don't even need to presume malicious or squatting intent; the entire left-pad fiasco was precipitated by a person who innocently registered "kik", and then later a company named Kik demanded that NPM transfer ownership to them. This happened with a package, but it would just as easily happen with a namespace. Once you introduce an identity layer, people start to expect identity validation. It's a problem worth solving, but crates.io would need to move beyond being merely staffed by part-time volunteers to solve it.

mikepurvis 5y ago

I feel like "single shared namespace" works best when the ecosystem is managed collaboratively, with a clear leadership to make calls on who gets what name— for example, something like a Linux distro, where there are Replaces/Provides metadata specifically to facilitate these kinds of transitions and avoid being stuck forever with crappy legacy nonsense. But this doesn't work at all in a free-for-all environment like PyPI, NPM, or Crates, where anyone can just grab a name and then have it in perpetuity. IMO the Docker ecosystem got this right, with baking in a domain name as part of the container, and insisting that everyone on docker.io use a vendor/product convention. This meant that the toplevel namespace was reserved for them to offer (or delegate the offering of) specific blessed container images, much more in line with how distro packaging might work. # Get the "official" image, whatever that means (but you trust Docker Inc, so yay). docker pull nginx # Get an image supplied by a specific vendor. docker pull bitnami/nginx # Get an image from a different server altogether; maybe it's your company, or you don't trust Docker Inc after all? docker pull quay.io/jitesoft/nginx Maybe the big-flat-namespace thing is still a years-later reaction against huge and unnecessary hierarchies in Java land? I think the ideal is not to permit infinite depth, but perhaps to insist on 2-3 levels.

dmurray 5y ago
Github generally has two levels (organization and repository) which works pretty well.
- hollerith 5y ago
  It works pretty well, but might not if making org2/repo55 a copy of org1/repo55 (i.e., forking) weren't free for the user (namely, org2) and maybe a venture-backed startup can afford to keep it free for the user, but a non-profit cannot.
  - xfer 5y ago
    Forking doesn't cost github anything it's a git branch of parent repo. That's why you could see people accessing youtube-dl via a commit hash of github dmca repo.

nine_k 5y ago

It's heartbreaking to see crates.io repeat nearly every mistake which npm has committed. I did not expect that from the Rust crowd which usually tends to design things carefully and with correctness and corner cases in mind.