10 ms·

Rust crate rg typosquatting/redirect to ripgrep

87 points by super_linear 3y ago

oars 3y ago
Interesting read. Thanks for sharing! This was created a year ago and Crates.io haven't taken it down so I assume they're ok with it.
- goku12 3y ago
  They should probably provide an official way of doing this with manual review.
  - Sharlin 3y ago
    Indeed, but unfortunately the crates.io team is chronically understaffed.
- Arnavion 3y ago
  It's not surprising. crates.io's stance on squatting or namespacing is to put their fingers in their ears and hope you stop asking them. https://crates.io/policies#squatting https://crates.io/policies#squatting Eg https://crates.io/users/swmon https://crates.io/users/swmon has squatted 100 empty crates since 2017.
  - ChrisSD 3y ago
    No their stance is to create RFC#3463[0] setting out a policy for handling name squatting and other issues. What they won't do is act unilaterally without a policy and accountability. [0]: https://github.com/Turbo87/rust-rfcs/blob/crates-io-policy-update/text/3463-crates-io-policy-update.md https://github.com/Turbo87/rust-rfcs/blob/crates-io-policy-u...
    - Arnavion 3y ago
      I'll believe it when they actually do something, such as delete the garbage crates from my second link. These "policies" and "discussions" have been going on since 2014. Note that even your link is not an accepted RFC; it's still being reviewed ( https://github.com/rust-lang/rfcs/pull/3463 https://github.com/rust-lang/rfcs/pull/3463 )
      - burntsushi 3y ago
        You've moving the goalposts. The point is that a written RFC and one that is under review is more than "put their fingers in their ears and hope you stop asking them."
lapinot 3y ago
Note that this problem of squatting (like many others security problems) is mostly a consequence of unmanaged repositories where developers publish themselves (like crates.io here, npm, pypi or the various "app stores"). Well-tended community-organized repositories, like most linux distribution have, do separate the role of package maintainer. This makes a much needed buffer between users and the developers, which regularly have contradicting interests, security-, support- and integration-wise. See ddevault's two very clear explanations of this issue: https://drewdevault.com/2019/12/09/Developers-shouldnt-distribute.html https://drewdevault.com/2019/12/09/Developers-shouldnt-distr... and https://drewdevault.com/2021/09/27/Let-distros-do-their-job.html https://drewdevault.com/2021/09/27/Let-distros-do-their-job....
- dgb23 3y ago
  You seem to be right in large parts, however other econsystems and package managers don't have the same kinds of problems to that degree, since their package names are namespaced. I'm not sure if this would solve this particular issue, it actually might, but name squatting in general seems to be far less of an issue with other package repositories.
  - MrBuddyCasino 3y ago
    I expect to hear about the fallout of crate.io's decision to not use namespaces and have no procedures to claim names for several years to come. Its not like they weren't warned this would happen, or that there was prior experience with exactly this kind of issue (just ask the Maven team).
    - dgb23 3y ago
      Is the rationale documented somewhere? I have a hard time to understand why this decision was made, especially in the face of prior art. To me it seems like a non-tradeoff.
      - pornel 3y ago
        I don't think there's an answer in one place, but various reasons have been given in many namespacing megathreads: • Some members think squatting is not a problem, let people take all names, and then people will invent creative ones, like nokogiri in Ruby. • Adding of namespaces only moves the squatting problem from squatting crate names to squatting namespaces. People like nice namespaces too. What if someone grabs an official-looking namespace like "aws", and what if that's a legit project? • Using usernames for namespaces makes typosquatting even worse, because many usernames are odd and hard to remember correctly (would you remember digits in winapi's owner handle? Is it BurnSushi, BurntSushi, BurnedSushi?) • crates.io relies on GitHub for identities, but GitHub usernames are not permanent. Crate names must be permanent. Letting the two out of sync creates new problems. • Crate names map to Rust identifiers, and there's a bikeshed about separators and ambiguities. • There's already a ton of non-namespaced crates, which must be supported. Having both namespaces and non-namespaces creates another bunch of bikeshed problems. • Having anti-squatting policy is laborious to enforce, and handling of disputes was a terrible drain of resources for npm. • crates.io is understaffed and can't deal with this right now. • people also proposed different approaches, like UUIDs/hashes/git URLs. There's a current RFC to use existing crates as namespace prefixes for projects.
deleted 3y ago
[deleted]
Xymist 3y ago
I've found this useful several times, and wish that `fd-find` did the same thing. It's not an unreasonable thing to do, IMO, under the appropriate circumstances.
sixhobbits 3y ago
I've definitely found `https://pypi.org/project/bs4/ https://pypi.org/project/bs4/` useful - in Python if you want to use BeautifulSoup (a common package for parsing and manipulating HTML), you import it with `from bs4 import BeautifulSoup`, but you install it with `pip3 install beautifulsoup4`. In this case, the `bs4` package actually directly installs what you need, though I agree with the arguments in the article why this might not be ideal. It would be nice if the committees that deal with the language itself could also look after things like this as it's hard to say objectively (main package needs x installs/month?) when something is squatting and when it is useful, but I think a 'common sense' approach goes pretty far.
jbaber 3y ago
rg's a rusty ag. To install ag, you usually have to guess something like "ag-the-silver-searcher". Not easy.
- joe-user 3y ago
  Why guess when there are installation instructions for various platforms on the README at https://github.com/ggreer/the_silver_searcher#installing https://github.com/ggreer/the_silver_searcher#installing? Also, although it may not be easy to remember, is this really a problem in practice given the installation count in most contexts is one? If there's a context where it's installed regularly, that's a one-time addition to an install script, Dockerfile, etc. in my experience. Do you have a situation that isn't amenable to that?
_lvbh 3y ago
I prefer Go’s imports via Git
- goku12 3y ago
  What happens if someone takes down that git repo? Or modify a commit? Sounds like another left-pad waiting to happen. Crates.io doesn't take down packages - even the yanked ones.
  - _lvbh 3y ago
    You can pin the version/commit (default). Won’t cause anything to break unless you explicitly run go get -u. Left pad situation is a concern yes.
alphazard 3y ago
The Go modules ecosystem doesn't suffer from the squatting problem because they chose not to create a new vacant namespace, and the corresponding rush to fill it. They easily could have. pkg.go.dev could be like npm. It's not a question of cost, google is paying for the infrastructure. It seems that language creators generally get this false impression that if they are the one to create the new namespace, then it will be high quality, and the best packages will get the short de-facto names. Maybe a few of the packages they wrote themselves can get some of the first names. That's never what happens. The wise solution is just to use DNS. We already have names, people pay for them, there is infrastructure for selling them, there is an auditable certificate system. A new package namespace won't have any of that.
- predictabl3 3y ago
  This issue has been pointed out so many times, it's clear it just doesn't matter, really, to anyone on the Cargo team. Meanwhile, years after this criticism was first offered, the problem remains, only more entrenched.
  - predictabl3 3y ago
    Man, HN really just can't handle harsh truths. Do I need to go dig up 6 year old issues about this? Or the half dozen times it's come up on HN over the last 5 years? I love Rust, I'm a big fanboy, but we need to stop pretending there aren't glaring blind spots.
miki123211 3y ago
THe same strategy is employed by PyTorch. If you do "pip install PyTorch", like I've done many times, it just tells you to "pip install torch" instead. To be even more confusing, though, the Anaconda package is actually named "PyTorch".

87 points by super_linear 3y ago

oars 3y ago

Interesting read. Thanks for sharing! This was created a year ago and Crates.io haven't taken it down so I assume they're ok with it.

goku12 3y ago
They should probably provide an official way of doing this with manual review.
- Sharlin 3y ago
  Indeed, but unfortunately the crates.io team is chronically understaffed.
Arnavion 3y ago
It's not surprising. crates.io's stance on squatting or namespacing is to put their fingers in their ears and hope you stop asking them. https://crates.io/policies#squatting https://crates.io/policies#squatting Eg https://crates.io/users/swmon https://crates.io/users/swmon has squatted 100 empty crates since 2017.
- ChrisSD 3y ago
  No their stance is to create RFC#3463[0] setting out a policy for handling name squatting and other issues. What they won't do is act unilaterally without a policy and accountability. [0]: https://github.com/Turbo87/rust-rfcs/blob/crates-io-policy-update/text/3463-crates-io-policy-update.md https://github.com/Turbo87/rust-rfcs/blob/crates-io-policy-u...
  - Arnavion 3y ago
    I'll believe it when they actually do something, such as delete the garbage crates from my second link. These "policies" and "discussions" have been going on since 2014. Note that even your link is not an accepted RFC; it's still being reviewed ( https://github.com/rust-lang/rfcs/pull/3463 https://github.com/rust-lang/rfcs/pull/3463 )
    - burntsushi 3y ago
      You've moving the goalposts. The point is that a written RFC and one that is under review is more than "put their fingers in their ears and hope you stop asking them."

lapinot 3y ago

Note that this problem of squatting (like many others security problems) is mostly a consequence of unmanaged repositories where developers publish themselves (like crates.io here, npm, pypi or the various "app stores"). Well-tended community-organized repositories, like most linux distribution have, do separate the role of package maintainer. This makes a much needed buffer between users and the developers, which regularly have contradicting interests, security-, support- and integration-wise. See ddevault's two very clear explanations of this issue: https://drewdevault.com/2019/12/09/Developers-shouldnt-distribute.html https://drewdevault.com/2019/12/09/Developers-shouldnt-distr... and https://drewdevault.com/2021/09/27/Let-distros-do-their-job.html https://drewdevault.com/2021/09/27/Let-distros-do-their-job....

dgb23 3y ago
You seem to be right in large parts, however other econsystems and package managers don't have the same kinds of problems to that degree, since their package names are namespaced. I'm not sure if this would solve this particular issue, it actually might, but name squatting in general seems to be far less of an issue with other package repositories.
- MrBuddyCasino 3y ago
  I expect to hear about the fallout of crate.io's decision to not use namespaces and have no procedures to claim names for several years to come. Its not like they weren't warned this would happen, or that there was prior experience with exactly this kind of issue (just ask the Maven team).
  - dgb23 3y ago
    Is the rationale documented somewhere? I have a hard time to understand why this decision was made, especially in the face of prior art. To me it seems like a non-tradeoff.
    - pornel 3y ago
      I don't think there's an answer in one place, but various reasons have been given in many namespacing megathreads: • Some members think squatting is not a problem, let people take all names, and then people will invent creative ones, like nokogiri in Ruby. • Adding of namespaces only moves the squatting problem from squatting crate names to squatting namespaces. People like nice namespaces too. What if someone grabs an official-looking namespace like "aws", and what if that's a legit project? • Using usernames for namespaces makes typosquatting even worse, because many usernames are odd and hard to remember correctly (would you remember digits in winapi's owner handle? Is it BurnSushi, BurntSushi, BurnedSushi?) • crates.io relies on GitHub for identities, but GitHub usernames are not permanent. Crate names must be permanent. Letting the two out of sync creates new problems. • Crate names map to Rust identifiers, and there's a bikeshed about separators and ambiguities. • There's already a ton of non-namespaced crates, which must be supported. Having both namespaces and non-namespaces creates another bunch of bikeshed problems. • Having anti-squatting policy is laborious to enforce, and handling of disputes was a terrible drain of resources for npm. • crates.io is understaffed and can't deal with this right now. • people also proposed different approaches, like UUIDs/hashes/git URLs. There's a current RFC to use existing crates as namespace prefixes for projects.

deleted 3y ago

[deleted]

Xymist 3y ago

I've found this useful several times, and wish that `fd-find` did the same thing. It's not an unreasonable thing to do, IMO, under the appropriate circumstances.

sixhobbits 3y ago

I've definitely found `https://pypi.org/project/bs4/ https://pypi.org/project/bs4/` useful - in Python if you want to use BeautifulSoup (a common package for parsing and manipulating HTML), you import it with `from bs4 import BeautifulSoup`, but you install it with `pip3 install beautifulsoup4`. In this case, the `bs4` package actually directly installs what you need, though I agree with the arguments in the article why this might not be ideal. It would be nice if the committees that deal with the language itself could also look after things like this as it's hard to say objectively (main package needs x installs/month?) when something is squatting and when it is useful, but I think a 'common sense' approach goes pretty far.

jbaber 3y ago

rg's a rusty ag. To install ag, you usually have to guess something like "ag-the-silver-searcher". Not easy.

joe-user 3y ago
Why guess when there are installation instructions for various platforms on the README at https://github.com/ggreer/the_silver_searcher#installing https://github.com/ggreer/the_silver_searcher#installing? Also, although it may not be easy to remember, is this really a problem in practice given the installation count in most contexts is one? If there's a context where it's installed regularly, that's a one-time addition to an install script, Dockerfile, etc. in my experience. Do you have a situation that isn't amenable to that?

_lvbh 3y ago

I prefer Go’s imports via Git

goku12 3y ago
What happens if someone takes down that git repo? Or modify a commit? Sounds like another left-pad waiting to happen. Crates.io doesn't take down packages - even the yanked ones.
- _lvbh 3y ago
  You can pin the version/commit (default). Won’t cause anything to break unless you explicitly run go get -u. Left pad situation is a concern yes.

alphazard 3y ago

The Go modules ecosystem doesn't suffer from the squatting problem because they chose not to create a new vacant namespace, and the corresponding rush to fill it. They easily could have. pkg.go.dev could be like npm. It's not a question of cost, google is paying for the infrastructure. It seems that language creators generally get this false impression that if they are the one to create the new namespace, then it will be high quality, and the best packages will get the short de-facto names. Maybe a few of the packages they wrote themselves can get some of the first names. That's never what happens. The wise solution is just to use DNS. We already have names, people pay for them, there is infrastructure for selling them, there is an auditable certificate system. A new package namespace won't have any of that.

predictabl3 3y ago
This issue has been pointed out so many times, it's clear it just doesn't matter, really, to anyone on the Cargo team. Meanwhile, years after this criticism was first offered, the problem remains, only more entrenched.
- predictabl3 3y ago
  Man, HN really just can't handle harsh truths. Do I need to go dig up 6 year old issues about this? Or the half dozen times it's come up on HN over the last 5 years? I love Rust, I'm a big fanboy, but we need to stop pretending there aren't glaring blind spots.

miki123211 3y ago

THe same strategy is employed by PyTorch. If you do "pip install PyTorch", like I've done many times, it just tells you to "pip install torch" instead. To be even more confusing, though, the Anaconda package is actually named "PyTorch".