5 ms·

All crate sources are stored in the crates.io package archive, which never deletes packages. A dependency veering off in a direction you don't like is one of t

by roca 4y ago

All crate sources are stored in the crates.io package archive, which never deletes packages. A dependency veering off in a direction you don't like is one of the risks of using someone else's code instead of writing it yourself. Cargo makes it easy to use forked dependencies, and forking a dependency is almost always less work than if you'd never used it and written the code yourself from the beginning. (And to be clear this is only a problem for future evolution; a crate author cannot remove or modify an already-published version of their crate.)

humanrebar 4y ago
"Never" is a long time, just saying. It'll be impossible to beat the "availability" guarantees of a local mirror (like a thumb drive) of a kernel source tarball. What happens when a crate version has to be removed due to a critical CVE or court order (IP Law violation, perhaps)? There may come a day where crates.io becomes torn between not breaking Linux source and not hosting actively bad source code. Note that some of those concerns do apply to vendoring source as well, but the additional download step also removes options that the kernel maintainers have as long as they ship all the source for the kernel in one tarball. Like more control over the timing of inevitable decisions.
- 3836293648 4y ago
  Does crates.io actively host any code? I thought it was all just readmes and links to github and docs.rs
  - conradludgate 4y ago
    They do host it. It's registry info is mirrored on github though
- mcherm 4y ago
  What happens today when a kernel module has to be removed due to a critical CVE or court order? That's not just a rhetorical flourish, I'm actually curious what the answer is. As far as I know, (1) it almost never happens and (2) when it does, the change is made in upstream repos and as a practical matter, everyone downloads those changes and their up-to-date local copies lose that code.
  - humanrebar 4y ago
    Fixing it in the future isn't the point. Breaking previous releases is. The previous tarballs still work and contain the relevant code. Your build wouldn't rely on hosts complying with court orders in countries you might not live in. If the code isn't vendored, just referenced with URLs, the old tarballs stop working.
    - roca 4y ago
      This hypothetical court-order situation is quite far-fetched. If crates.io was ordered to take down some or all versions of a package, an alternative mirror could easily be created elsewhere and you could configure cargo to use it. But I think the kernel would vendor crate dependencies, partly so that people can build without accessing the network, simply because that's policy in many places.
- notriddle 4y ago
  > What happens when a crate version has to be removed due to a critical CVE or court order (IP Law violation, perhaps)? CVE = The Yank flag. Cargo will refuse to add new yanked packages to a lock file, but if a yanked package is already in the lock file, it will still build. The package is not actually deleted. https://doc.rust-lang.org/cargo/commands/cargo-yank.html https://doc.rust-lang.org/cargo/commands/cargo-yank.html Legal = Hard delete. Nobody will go to jail just to avoid breaking your build. Of course, since crates.io and kernel.org are in the same legal jurisdiction, is there any actual difference here?
marginalia_nu 4y ago
This is still fairly short sighted. Websites shut down, large websites with big storage demands are especially vulnerable to attrition. Who wants to pay the mounting bill for keeping decades of revisions of historical rust packages online? I can grab the kernel sources from 1997 and build them today. Will I be able to build rust code from 2022 in 2047, because the 1997 kernel will still build at that date.
- pdw 4y ago
  "I can grab the kernel sources from 1997 and build them today." Where would you be grabbing it from? ...From a website? "Websites shut down, large websites with big storage demands are especially vulnerable to attrition. Who wants to pay the mounting bill for keeping decades of revisions of historical Linux kernels online?"
  - humanrebar 4y ago
    You make a copy, store it on your medium if choice, and put it in a filing cabinet. I gather that certain organizations use magnetic tape backups for especially important data. For some organizations and individuals, kernel source code could be that important.
    - kibwen 4y ago
      You can easily do this with Rust and Cargo as well.
      - marginalia_nu 4y ago
        There is a fairly large difference between archiving your own project's history for as long as you feel like, and archiving the complete history of every significant piece of code ever written in a particular programming language forever.
        kibwen 4y ago

by roca 4y ago

humanrebar 4y ago

"Never" is a long time, just saying. It'll be impossible to beat the "availability" guarantees of a local mirror (like a thumb drive) of a kernel source tarball. What happens when a crate version has to be removed due to a critical CVE or court order (IP Law violation, perhaps)? There may come a day where crates.io becomes torn between not breaking Linux source and not hosting actively bad source code. Note that some of those concerns do apply to vendoring source as well, but the additional download step also removes options that the kernel maintainers have as long as they ship all the source for the kernel in one tarball. Like more control over the timing of inevitable decisions.

3836293648 4y ago
Does crates.io actively host any code? I thought it was all just readmes and links to github and docs.rs
- conradludgate 4y ago
  They do host it. It's registry info is mirrored on github though
mcherm 4y ago
What happens today when a kernel module has to be removed due to a critical CVE or court order? That's not just a rhetorical flourish, I'm actually curious what the answer is. As far as I know, (1) it almost never happens and (2) when it does, the change is made in upstream repos and as a practical matter, everyone downloads those changes and their up-to-date local copies lose that code.
- humanrebar 4y ago
  Fixing it in the future isn't the point. Breaking previous releases is. The previous tarballs still work and contain the relevant code. Your build wouldn't rely on hosts complying with court orders in countries you might not live in. If the code isn't vendored, just referenced with URLs, the old tarballs stop working.
  - roca 4y ago
    This hypothetical court-order situation is quite far-fetched. If crates.io was ordered to take down some or all versions of a package, an alternative mirror could easily be created elsewhere and you could configure cargo to use it. But I think the kernel would vendor crate dependencies, partly so that people can build without accessing the network, simply because that's policy in many places.
notriddle 4y ago
> What happens when a crate version has to be removed due to a critical CVE or court order (IP Law violation, perhaps)? CVE = The Yank flag. Cargo will refuse to add new yanked packages to a lock file, but if a yanked package is already in the lock file, it will still build. The package is not actually deleted. https://doc.rust-lang.org/cargo/commands/cargo-yank.html https://doc.rust-lang.org/cargo/commands/cargo-yank.html Legal = Hard delete. Nobody will go to jail just to avoid breaking your build. Of course, since crates.io and kernel.org are in the same legal jurisdiction, is there any actual difference here?

marginalia_nu 4y ago

This is still fairly short sighted. Websites shut down, large websites with big storage demands are especially vulnerable to attrition. Who wants to pay the mounting bill for keeping decades of revisions of historical rust packages online? I can grab the kernel sources from 1997 and build them today. Will I be able to build rust code from 2022 in 2047, because the 1997 kernel will still build at that date.

pdw 4y ago
"I can grab the kernel sources from 1997 and build them today." Where would you be grabbing it from? ...From a website? "Websites shut down, large websites with big storage demands are especially vulnerable to attrition. Who wants to pay the mounting bill for keeping decades of revisions of historical Linux kernels online?"
- humanrebar 4y ago
  You make a copy, store it on your medium if choice, and put it in a filing cabinet. I gather that certain organizations use magnetic tape backups for especially important data. For some organizations and individuals, kernel source code could be that important.
  - kibwen 4y ago
    You can easily do this with Rust and Cargo as well.
    - marginalia_nu 4y ago
      There is a fairly large difference between archiving your own project's history for as long as you feel like, and archiving the complete history of every significant piece of code ever written in a particular programming language forever.
      - kibwen 4y ago