4 ms·

> There is no tight coupling between GitHub and cargo/crates.io (sure it uses GitHub internally but that is an implementation detail). Cargo had a “locked” opt

> There is no tight coupling between GitHub and cargo/crates.io (sure it uses GitHub internally but that is an implementation detail). Cargo had a “locked” option that uses the URL and commit hash from the Cargo.lock file. If the crate, repository, or commit has changed then it won’t build. This is what everyone uses for reproducible and secure builds, but it’s not as commonly used for casual use.

jhugo 5y ago
Most cargo deps aren't specified as Git deps, so there's no repository or commit hash to check. Cargo.lock has a hash of the crate itself; there's no requirement that it even came from a VCS repository. You can publish from a local folder. Checking the hash of the crate still verifies that the code hasn't changed, I just want to clarify that there isn't some mechanism by which non-git deps are automatically verified back to the VCS repo commit hash they came from.
- rectang 5y ago
  It may be possible to publish from a local folder and offer the user no history or context. However, I think it makes sense to privilege packages which can be cryptographically verified as deriving from a specific commit within an open source commit history, in a public repository at a specific URL.
  - jhugo 5y ago
    I'd almost say that publishing from a nonpublic repository (or no repository) is a misfeature. At least when there is no good way to rapidly audit the source of crates on crates.io. Most people will go looking for a public git repo, but there's often no quick way to be certain that it's the same code as was published on crates.io.
    - dathinab 5y ago
      It's less about a non-public repository and more about a non-git repository. (And some big company/mono repo/history-less CI-auto clones.) So if you add a "trust"/"signing" system you probably want to make it version control system independent, which doesn't mean there couldn't also be something build on top which isn't vcs independent ;=) > people will go looking for a public git repo And worth the will likely read the code shown through the http-code preview of the repository hoster of whatever the default branch is :=( I have seen to many issue in issue trackers because of that. I hope you don't mean "using the code preview of the hoster" when you wrote "rapid audit".
      - jhugo 5y ago
        My implication with "rapid audit" was that crates.io would provide a nice code preview, so that people can get the feature that they currently go to GitHub for, while actually viewing the code from the same source they're depending on the crate from rather than from GitHub where there is no guarantee it's the same code. Of course the best thing to do would be to look locally at the source of the crate that was actually downloaded by cargo. That's what I do. But you have to recognise that most people are going to weight convenience quite highly, and are going to be lazy when they should be thorough, and so on, so if you want to improve the security and trust in the ecosystem as a whole, you should try to make the safe path as low-friction as possible. The docs.rs code browser is currently the most "convenient" way to see the code for a specific crates.io release, so improving that would also be worthwhile. Right now I think there's no way to see the source of build.rs in that browser, and the formatting & navigation is nowhere as nice as on the likes of GitHub.
  - dathinab 5y ago
    IMHO the idea that source code in any place outside of crates.io should in any way be relevant for reviewing code uploaded to crates.io is deeply flawed. Even if you make it technically work there are endless ways this could be abused in a social engineering context. Like even if you when uploading check the code is the same, how do you check it stays the same. How do you check it's the same for any way the site is accessed (e.g. git vs. html code preview). How do you make sure there is not other social engineering in place (misleading tags etc.). You can't, reliable, in a good way. Also don't forget, the client which uploads thinks can't be trusted. So you would need to push a full git (maybe sparse) branch to the server instead of just uploading a subset of the current checkout, that's pretty much a no-go for various reason. Through it's not hat there are no ways to reach something similar like e.g.: What you can do: - sign uploaded packages - automatically include the commit hash or other vcs history ids (but then it's in the client so they CAN be tempered with) - all some form of "external-signing" that attaches "labels" to crates - then you can e.g. have a GitHub service which verifies that the uploaded code matches a checked from a repository on GitHub and adds an appropriate label, similar services with all the service specific nobs could be made for various repository hosters. But this would not be a part of cargo/crates, but 3rd party services (e.g. provided by the hosters). - Similar a company doing reviews could add a "reviewed by us" label. - Or cargo-crev could attach a "there are reviews" label. How this "labels" then interact with the UI and if they are even stored on crates.io is a different matter altogether.
jeltz 5y ago
Why isn't that the default?
- Diggsey 5y ago
  To be clear, even without `--locked`, cargo will use the dependencies from the lockfile, as long as the lockfile is not out of date compared to Cargo.toml. However, for development, you normally want cargo to update the lockfile after you change something in `Cargo.toml` (like adding a new dependency). The `--locked` option is particularly useful in CI though, where you want it to fail if the lockfile is out of date, rather than update the lockfile and continue.
  - steveklabnik 5y ago
    To spell it out in even more detail: I add foo = "1.0.0" to my Cargo.toml. The latest release of foo is 1.0.1. I invoke "cargo build." Cargo builds my project, using 1.0.1 (since "1.0.0" is short for "^1.0.0"), and records that in the lockfile. Now foo releases 1.1.0. I invoke "cargo build". Cargo uses the lockfile, and nothing changes. I still build with 1.0.1, because that's what's in the lockfile. Now, let's say I go in and change my Cargo.toml to use 1.1.0. This is where the behavior differs: 'cargo build' will say "oh, you've changed your Cargo.toml, let's perform resolution again" and will update your Cargo.lock to have 1.1.0. 'cargo build --locked' will say "error, you have changed your Cargo.toml but the lockfile was not updated." That's why --locked is useful in CI; it will make sure that the lock you've committed is up to date. It doesn't imply that Cargo ignores the lockfile by default, only that it will update your Cargo.lock when you change your Cargo.toml, because that implies you're asking for a change in your dependencies.
    - dathinab 5y ago
      > will update your Cargo.lock to have 1.1.0. 1.1.0 or later, as it's `^1.1.0`.
      - steveklabnik 5y ago

jhugo 5y ago

Most cargo deps aren't specified as Git deps, so there's no repository or commit hash to check. Cargo.lock has a hash of the crate itself; there's no requirement that it even came from a VCS repository. You can publish from a local folder. Checking the hash of the crate still verifies that the code hasn't changed, I just want to clarify that there isn't some mechanism by which non-git deps are automatically verified back to the VCS repo commit hash they came from.

rectang 5y ago
It may be possible to publish from a local folder and offer the user no history or context. However, I think it makes sense to privilege packages which can be cryptographically verified as deriving from a specific commit within an open source commit history, in a public repository at a specific URL.
- jhugo 5y ago
  I'd almost say that publishing from a nonpublic repository (or no repository) is a misfeature. At least when there is no good way to rapidly audit the source of crates on crates.io. Most people will go looking for a public git repo, but there's often no quick way to be certain that it's the same code as was published on crates.io.
  - dathinab 5y ago
    It's less about a non-public repository and more about a non-git repository. (And some big company/mono repo/history-less CI-auto clones.) So if you add a "trust"/"signing" system you probably want to make it version control system independent, which doesn't mean there couldn't also be something build on top which isn't vcs independent ;=) > people will go looking for a public git repo And worth the will likely read the code shown through the http-code preview of the repository hoster of whatever the default branch is :=( I have seen to many issue in issue trackers because of that. I hope you don't mean "using the code preview of the hoster" when you wrote "rapid audit".
    - jhugo 5y ago
      My implication with "rapid audit" was that crates.io would provide a nice code preview, so that people can get the feature that they currently go to GitHub for, while actually viewing the code from the same source they're depending on the crate from rather than from GitHub where there is no guarantee it's the same code. Of course the best thing to do would be to look locally at the source of the crate that was actually downloaded by cargo. That's what I do. But you have to recognise that most people are going to weight convenience quite highly, and are going to be lazy when they should be thorough, and so on, so if you want to improve the security and trust in the ecosystem as a whole, you should try to make the safe path as low-friction as possible. The docs.rs code browser is currently the most "convenient" way to see the code for a specific crates.io release, so improving that would also be worthwhile. Right now I think there's no way to see the source of build.rs in that browser, and the formatting & navigation is nowhere as nice as on the likes of GitHub.
- dathinab 5y ago
  IMHO the idea that source code in any place outside of crates.io should in any way be relevant for reviewing code uploaded to crates.io is deeply flawed. Even if you make it technically work there are endless ways this could be abused in a social engineering context. Like even if you when uploading check the code is the same, how do you check it stays the same. How do you check it's the same for any way the site is accessed (e.g. git vs. html code preview). How do you make sure there is not other social engineering in place (misleading tags etc.). You can't, reliable, in a good way. Also don't forget, the client which uploads thinks can't be trusted. So you would need to push a full git (maybe sparse) branch to the server instead of just uploading a subset of the current checkout, that's pretty much a no-go for various reason. Through it's not hat there are no ways to reach something similar like e.g.: What you can do: - sign uploaded packages - automatically include the commit hash or other vcs history ids (but then it's in the client so they CAN be tempered with) - all some form of "external-signing" that attaches "labels" to crates - then you can e.g. have a GitHub service which verifies that the uploaded code matches a checked from a repository on GitHub and adds an appropriate label, similar services with all the service specific nobs could be made for various repository hosters. But this would not be a part of cargo/crates, but 3rd party services (e.g. provided by the hosters). - Similar a company doing reviews could add a "reviewed by us" label. - Or cargo-crev could attach a "there are reviews" label. How this "labels" then interact with the UI and if they are even stored on crates.io is a different matter altogether.

jeltz 5y ago

Why isn't that the default?

Diggsey 5y ago
To be clear, even without `--locked`, cargo will use the dependencies from the lockfile, as long as the lockfile is not out of date compared to Cargo.toml. However, for development, you normally want cargo to update the lockfile after you change something in `Cargo.toml` (like adding a new dependency). The `--locked` option is particularly useful in CI though, where you want it to fail if the lockfile is out of date, rather than update the lockfile and continue.
- steveklabnik 5y ago
  To spell it out in even more detail: I add foo = "1.0.0" to my Cargo.toml. The latest release of foo is 1.0.1. I invoke "cargo build." Cargo builds my project, using 1.0.1 (since "1.0.0" is short for "^1.0.0"), and records that in the lockfile. Now foo releases 1.1.0. I invoke "cargo build". Cargo uses the lockfile, and nothing changes. I still build with 1.0.1, because that's what's in the lockfile. Now, let's say I go in and change my Cargo.toml to use 1.1.0. This is where the behavior differs: 'cargo build' will say "oh, you've changed your Cargo.toml, let's perform resolution again" and will update your Cargo.lock to have 1.1.0. 'cargo build --locked' will say "error, you have changed your Cargo.toml but the lockfile was not updated." That's why --locked is useful in CI; it will make sure that the lock you've committed is up to date. It doesn't imply that Cargo ignores the lockfile by default, only that it will update your Cargo.lock when you change your Cargo.toml, because that implies you're asking for a change in your dependencies.
  - dathinab 5y ago
    > will update your Cargo.lock to have 1.1.0. 1.1.0 or later, as it's `^1.1.0`.
    - steveklabnik 5y ago