3 ms·

> crates.io matches the code on GitHub, There is no tight coupling between GitHub and cargo/crates.io (sure it uses GitHub internally but that is an implementa

> crates.io matches the code on GitHub, There is no tight coupling between GitHub and cargo/crates.io (sure it uses GitHub internally but that is an implementation detail). But not only has it no tight coupling with GitHub it also doesn't require you to use git, you can use whatever version control you want and at worse you don't get support for "detect dirty repository". Similar git tags are fundamentally unreliable as you can always "move" some to any arbitrary commit. So IMHO the problem here is relying on code you didn't got from github which might not even use git to match a arbitrary tag on something on github which might not even be from the same author (but e.g. a mirror on GitHub from whatever VC the author uses). But uploads to crates.io are immutable and are source code uploads, so you can just review them. In general (independent of cargo) do review the code you use not some code you got form somewhere else which you hope/believe is the same.

PragmaticPulp 5y ago
> There is no tight coupling between GitHub and cargo/crates.io (sure it uses GitHub internally but that is an implementation detail). Cargo had a “locked” option that uses the URL and commit hash from the Cargo.lock file. If the crate, repository, or commit has changed then it won’t build. This is what everyone uses for reproducible and secure builds, but it’s not as commonly used for casual use.
- jhugo 5y ago
  Most cargo deps aren't specified as Git deps, so there's no repository or commit hash to check. Cargo.lock has a hash of the crate itself; there's no requirement that it even came from a VCS repository. You can publish from a local folder. Checking the hash of the crate still verifies that the code hasn't changed, I just want to clarify that there isn't some mechanism by which non-git deps are automatically verified back to the VCS repo commit hash they came from.
  - rectang 5y ago
    It may be possible to publish from a local folder and offer the user no history or context. However, I think it makes sense to privilege packages which can be cryptographically verified as deriving from a specific commit within an open source commit history, in a public repository at a specific URL.
    - jhugo 5y ago
      I'd almost say that publishing from a nonpublic repository (or no repository) is a misfeature. At least when there is no good way to rapidly audit the source of crates on crates.io. Most people will go looking for a public git repo, but there's often no quick way to be certain that it's the same code as was published on crates.io.
      - dathinab 5y ago
        It's less about a non-public repository and more about a non-git repository. (And some big company/mono repo/history-less CI-auto clones.) So if you add a "trust"/"signing" system you probably want to make it version control system independent, which doesn't mean there couldn't also be something build on top which isn't vcs independent ;=) > people will go looking for a public git repo And worth the will likely read the code shown through the http-code preview of the repository hoster of whatever the default branch is :=( I have seen to many issue in issue trackers because of that. I hope you don't mean "using the code preview of the hoster" when you wrote "rapid audit".
        jhugo 5y ago
        My implication with "rapid audit" was that crates.io would provide a nice code preview, so that people can get the feature that they currently go to GitHub for, while actually viewing the code from the same source they're depending on the crate from rather than from GitHub where there is no guarantee it's the same code. Of course the best thing to do would be to look locally at the source of the crate that was actually downloaded by cargo. That's what I do. But you have to recognise that most people are going to weight convenience quite highly, and are going to be lazy when they should be thorough, and so on, so if you want to improve the security and trust in the ecosystem as a whole, you should try to make the safe path as low-friction as possible. The docs.rs code browser is currently the most "convenient" way to see the code for a specific crates.io release, so improving that would also be worthwhile. Right now I think there's no way to see the source of build.rs in that browser, and the formatting & navigation is nowhere as nice as on the likes of GitHub.
- jeltz 5y ago
  Why isn't that the default?
  - Diggsey 5y ago
    To be clear, even without `--locked`, cargo will use the dependencies from the lockfile, as long as the lockfile is not out of date compared to Cargo.toml. However, for development, you normally want cargo to update the lockfile after you change something in `Cargo.toml` (like adding a new dependency). The `--locked` option is particularly useful in CI though, where you want it to fail if the lockfile is out of date, rather than update the lockfile and continue.
    - steveklabnik 5y ago
      To spell it out in even more detail: I add foo = "1.0.0" to my Cargo.toml. The latest release of foo is 1.0.1. I invoke "cargo build." Cargo builds my project, using 1.0.1 (since "1.0.0" is short for "^1.0.0"), and records that in the lockfile. Now foo releases 1.1.0. I invoke "cargo build". Cargo uses the lockfile, and nothing changes. I still build with 1.0.1, because that's what's in the lockfile. Now, let's say I go in and change my Cargo.toml to use 1.1.0. This is where the behavior differs: 'cargo build' will say "oh, you've changed your Cargo.toml, let's perform resolution again" and will update your Cargo.lock to have 1.1.0. 'cargo build --locked' will say "error, you have changed your Cargo.toml but the lockfile was not updated." That's why --locked is useful in CI; it will make sure that the lock you've committed is up to date. It doesn't imply that Cargo ignores the lockfile by default, only that it will update your Cargo.lock when you change your Cargo.toml, because that implies you're asking for a change in your dependencies.
      - dathinab 5y ago
        > will update your Cargo.lock to have 1.1.0. 1.1.0 or later, as it's `^1.1.0`.
        steveklabnik 5y ago
rectang 5y ago
Treating PGP signed commits as privileged and only pointing at them as opposed to mutable tags seems like it would help.
- dathinab 5y ago
  I'm not sure it's actually possible, for varying reasons. Even if we ignore that cargo can be used with other version control systems I see some problems: - Validation must be done server side as everything the client does can be manipulated. - So the server would need to be able to make sure that some signature is valid for some commit. - But you don't upload commits, you uploads a sub-set of the checkout produced by the combination of the current commit and previous commits. - Uploading all of the diff of the current (or previous) commit is a no-go for various reasons (size, leaking internal code etc.). - Similar even if it would work, the repository on GitHub could still contains different code, the signature might not match, but who is checking the signature? The reviewer manually? What you maybe could do is e.g. signing the uploaded archive which in combination with e.g. a Yubikey would make it harder for attackers to upload malicious packages (if the authors aren't the attackers). You also could include some version id (e.g. git hash) in the signature, then review tools could check that the uploaded code matches the GitHub repository. But really the most important thing is to review the code you actually use (e.g. what is uploaded to crates.io) and not what you belive you probably should be using.
  - rectang 5y ago
    In the abstract, what I'm wishing for is definitely possible at least for some subset of packages. • Source code packages are cryptographically signed by at least one and ideally several identities which are verifiable via a web of trust. • Packages are associated with a public repository base URL known to the package manager and that base URL doesn't change without setting off alarm bells. • The source code of a package can be traced back and cryptographically connected to a public commit history at the public repository URL.

PragmaticPulp 5y ago

> There is no tight coupling between GitHub and cargo/crates.io (sure it uses GitHub internally but that is an implementation detail). Cargo had a “locked” option that uses the URL and commit hash from the Cargo.lock file. If the crate, repository, or commit has changed then it won’t build. This is what everyone uses for reproducible and secure builds, but it’s not as commonly used for casual use.

jhugo 5y ago
Most cargo deps aren't specified as Git deps, so there's no repository or commit hash to check. Cargo.lock has a hash of the crate itself; there's no requirement that it even came from a VCS repository. You can publish from a local folder. Checking the hash of the crate still verifies that the code hasn't changed, I just want to clarify that there isn't some mechanism by which non-git deps are automatically verified back to the VCS repo commit hash they came from.
- rectang 5y ago
  It may be possible to publish from a local folder and offer the user no history or context. However, I think it makes sense to privilege packages which can be cryptographically verified as deriving from a specific commit within an open source commit history, in a public repository at a specific URL.
  - jhugo 5y ago
    I'd almost say that publishing from a nonpublic repository (or no repository) is a misfeature. At least when there is no good way to rapidly audit the source of crates on crates.io. Most people will go looking for a public git repo, but there's often no quick way to be certain that it's the same code as was published on crates.io.
    - dathinab 5y ago
      It's less about a non-public repository and more about a non-git repository. (And some big company/mono repo/history-less CI-auto clones.) So if you add a "trust"/"signing" system you probably want to make it version control system independent, which doesn't mean there couldn't also be something build on top which isn't vcs independent ;=) > people will go looking for a public git repo And worth the will likely read the code shown through the http-code preview of the repository hoster of whatever the default branch is :=( I have seen to many issue in issue trackers because of that. I hope you don't mean "using the code preview of the hoster" when you wrote "rapid audit".
      - jhugo 5y ago
        My implication with "rapid audit" was that crates.io would provide a nice code preview, so that people can get the feature that they currently go to GitHub for, while actually viewing the code from the same source they're depending on the crate from rather than from GitHub where there is no guarantee it's the same code. Of course the best thing to do would be to look locally at the source of the crate that was actually downloaded by cargo. That's what I do. But you have to recognise that most people are going to weight convenience quite highly, and are going to be lazy when they should be thorough, and so on, so if you want to improve the security and trust in the ecosystem as a whole, you should try to make the safe path as low-friction as possible. The docs.rs code browser is currently the most "convenient" way to see the code for a specific crates.io release, so improving that would also be worthwhile. Right now I think there's no way to see the source of build.rs in that browser, and the formatting & navigation is nowhere as nice as on the likes of GitHub.
jeltz 5y ago
Why isn't that the default?
- Diggsey 5y ago
  To be clear, even without `--locked`, cargo will use the dependencies from the lockfile, as long as the lockfile is not out of date compared to Cargo.toml. However, for development, you normally want cargo to update the lockfile after you change something in `Cargo.toml` (like adding a new dependency). The `--locked` option is particularly useful in CI though, where you want it to fail if the lockfile is out of date, rather than update the lockfile and continue.
  - steveklabnik 5y ago
    To spell it out in even more detail: I add foo = "1.0.0" to my Cargo.toml. The latest release of foo is 1.0.1. I invoke "cargo build." Cargo builds my project, using 1.0.1 (since "1.0.0" is short for "^1.0.0"), and records that in the lockfile. Now foo releases 1.1.0. I invoke "cargo build". Cargo uses the lockfile, and nothing changes. I still build with 1.0.1, because that's what's in the lockfile. Now, let's say I go in and change my Cargo.toml to use 1.1.0. This is where the behavior differs: 'cargo build' will say "oh, you've changed your Cargo.toml, let's perform resolution again" and will update your Cargo.lock to have 1.1.0. 'cargo build --locked' will say "error, you have changed your Cargo.toml but the lockfile was not updated." That's why --locked is useful in CI; it will make sure that the lock you've committed is up to date. It doesn't imply that Cargo ignores the lockfile by default, only that it will update your Cargo.lock when you change your Cargo.toml, because that implies you're asking for a change in your dependencies.
    - dathinab 5y ago
      > will update your Cargo.lock to have 1.1.0. 1.1.0 or later, as it's `^1.1.0`.
      - steveklabnik 5y ago

rectang 5y ago

Treating PGP signed commits as privileged and only pointing at them as opposed to mutable tags seems like it would help.

dathinab 5y ago
I'm not sure it's actually possible, for varying reasons. Even if we ignore that cargo can be used with other version control systems I see some problems: - Validation must be done server side as everything the client does can be manipulated. - So the server would need to be able to make sure that some signature is valid for some commit. - But you don't upload commits, you uploads a sub-set of the checkout produced by the combination of the current commit and previous commits. - Uploading all of the diff of the current (or previous) commit is a no-go for various reasons (size, leaking internal code etc.). - Similar even if it would work, the repository on GitHub could still contains different code, the signature might not match, but who is checking the signature? The reviewer manually? What you maybe could do is e.g. signing the uploaded archive which in combination with e.g. a Yubikey would make it harder for attackers to upload malicious packages (if the authors aren't the attackers). You also could include some version id (e.g. git hash) in the signature, then review tools could check that the uploaded code matches the GitHub repository. But really the most important thing is to review the code you actually use (e.g. what is uploaded to crates.io) and not what you belive you probably should be using.
- rectang 5y ago
  In the abstract, what I'm wishing for is definitely possible at least for some subset of packages. • Source code packages are cryptographically signed by at least one and ideally several identities which are verifiable via a web of trust. • Packages are associated with a public repository base URL known to the package manager and that base URL doesn't change without setting off alarm bells. • The source code of a package can be traced back and cryptographically connected to a public commit history at the public repository URL.