5 ms·

What prevents a given URL from disappearing? Does that just break a particular source version of the Linux kernel? What happens when a given dependency adds ne

What prevents a given URL from disappearing? Does that just break a particular source version of the Linux kernel? What happens when a given dependency adds new kernel-inappropriate features? Are kernel devs going to act like distro maintainers and decide between forking, maintaining patch sets, etc.?

sanderjd 5y ago
To the first question, obviously the sources of dependencies would be brought into the tree. This is easy and there's no reason I'm aware of not to do it for something like the Linux kernel. To the second set of questions, how is this any different than any other dependency the kernel has? If the answer is "the kernel has no dependencies" then yeah, I'm very sympathetic to the argument that bringing in rust libraries is not a good reason to start having dependencies when none previously existed at all, but is that the case?
roca 5y ago
All crate sources are stored in the crates.io package archive, which never deletes packages. A dependency veering off in a direction you don't like is one of the risks of using someone else's code instead of writing it yourself. Cargo makes it easy to use forked dependencies, and forking a dependency is almost always less work than if you'd never used it and written the code yourself from the beginning. (And to be clear this is only a problem for future evolution; a crate author cannot remove or modify an already-published version of their crate.)
- humanrebar 5y ago
  "Never" is a long time, just saying. It'll be impossible to beat the "availability" guarantees of a local mirror (like a thumb drive) of a kernel source tarball. What happens when a crate version has to be removed due to a critical CVE or court order (IP Law violation, perhaps)? There may come a day where crates.io becomes torn between not breaking Linux source and not hosting actively bad source code. Note that some of those concerns do apply to vendoring source as well, but the additional download step also removes options that the kernel maintainers have as long as they ship all the source for the kernel in one tarball. Like more control over the timing of inevitable decisions.
  - 3836293648 5y ago
    Does crates.io actively host any code? I thought it was all just readmes and links to github and docs.rs
    - conradludgate 5y ago
      They do host it. It's registry info is mirrored on github though
  - mcherm 5y ago
    What happens today when a kernel module has to be removed due to a critical CVE or court order? That's not just a rhetorical flourish, I'm actually curious what the answer is. As far as I know, (1) it almost never happens and (2) when it does, the change is made in upstream repos and as a practical matter, everyone downloads those changes and their up-to-date local copies lose that code.
    - humanrebar 5y ago
      Fixing it in the future isn't the point. Breaking previous releases is. The previous tarballs still work and contain the relevant code. Your build wouldn't rely on hosts complying with court orders in countries you might not live in. If the code isn't vendored, just referenced with URLs, the old tarballs stop working.
      - roca 5y ago
        This hypothetical court-order situation is quite far-fetched. If crates.io was ordered to take down some or all versions of a package, an alternative mirror could easily be created elsewhere and you could configure cargo to use it. But I think the kernel would vendor crate dependencies, partly so that people can build without accessing the network, simply because that's policy in many places.
  - notriddle 5y ago
    > What happens when a crate version has to be removed due to a critical CVE or court order (IP Law violation, perhaps)? CVE = The Yank flag. Cargo will refuse to add new yanked packages to a lock file, but if a yanked package is already in the lock file, it will still build. The package is not actually deleted. https://doc.rust-lang.org/cargo/commands/cargo-yank.html https://doc.rust-lang.org/cargo/commands/cargo-yank.html Legal = Hard delete. Nobody will go to jail just to avoid breaking your build. Of course, since crates.io and kernel.org are in the same legal jurisdiction, is there any actual difference here?
- marginalia_nu 5y ago
  This is still fairly short sighted. Websites shut down, large websites with big storage demands are especially vulnerable to attrition. Who wants to pay the mounting bill for keeping decades of revisions of historical rust packages online? I can grab the kernel sources from 1997 and build them today. Will I be able to build rust code from 2022 in 2047, because the 1997 kernel will still build at that date.
  - pdw 5y ago
    "I can grab the kernel sources from 1997 and build them today." Where would you be grabbing it from? ...From a website? "Websites shut down, large websites with big storage demands are especially vulnerable to attrition. Who wants to pay the mounting bill for keeping decades of revisions of historical Linux kernels online?"
    - humanrebar 5y ago
      You make a copy, store it on your medium if choice, and put it in a filing cabinet. I gather that certain organizations use magnetic tape backups for especially important data. For some organizations and individuals, kernel source code could be that important.
      - kibwen 5y ago
        You can easily do this with Rust and Cargo as well.
        marginalia_nu 5y ago
        There is a fairly large difference between archiving your own project's history for as long as you feel like, and archiving the complete history of every significant piece of code ever written in a particular programming language forever.
        kibwen 5y ago

sanderjd 5y ago

To the first question, obviously the sources of dependencies would be brought into the tree. This is easy and there's no reason I'm aware of not to do it for something like the Linux kernel. To the second set of questions, how is this any different than any other dependency the kernel has? If the answer is "the kernel has no dependencies" then yeah, I'm very sympathetic to the argument that bringing in rust libraries is not a good reason to start having dependencies when none previously existed at all, but is that the case?

roca 5y ago

All crate sources are stored in the crates.io package archive, which never deletes packages. A dependency veering off in a direction you don't like is one of the risks of using someone else's code instead of writing it yourself. Cargo makes it easy to use forked dependencies, and forking a dependency is almost always less work than if you'd never used it and written the code yourself from the beginning. (And to be clear this is only a problem for future evolution; a crate author cannot remove or modify an already-published version of their crate.)

humanrebar 5y ago
"Never" is a long time, just saying. It'll be impossible to beat the "availability" guarantees of a local mirror (like a thumb drive) of a kernel source tarball. What happens when a crate version has to be removed due to a critical CVE or court order (IP Law violation, perhaps)? There may come a day where crates.io becomes torn between not breaking Linux source and not hosting actively bad source code. Note that some of those concerns do apply to vendoring source as well, but the additional download step also removes options that the kernel maintainers have as long as they ship all the source for the kernel in one tarball. Like more control over the timing of inevitable decisions.
- 3836293648 5y ago
  Does crates.io actively host any code? I thought it was all just readmes and links to github and docs.rs
  - conradludgate 5y ago
    They do host it. It's registry info is mirrored on github though
- mcherm 5y ago
  What happens today when a kernel module has to be removed due to a critical CVE or court order? That's not just a rhetorical flourish, I'm actually curious what the answer is. As far as I know, (1) it almost never happens and (2) when it does, the change is made in upstream repos and as a practical matter, everyone downloads those changes and their up-to-date local copies lose that code.
  - humanrebar 5y ago
    Fixing it in the future isn't the point. Breaking previous releases is. The previous tarballs still work and contain the relevant code. Your build wouldn't rely on hosts complying with court orders in countries you might not live in. If the code isn't vendored, just referenced with URLs, the old tarballs stop working.
    - roca 5y ago
      This hypothetical court-order situation is quite far-fetched. If crates.io was ordered to take down some or all versions of a package, an alternative mirror could easily be created elsewhere and you could configure cargo to use it. But I think the kernel would vendor crate dependencies, partly so that people can build without accessing the network, simply because that's policy in many places.
- notriddle 5y ago
  > What happens when a crate version has to be removed due to a critical CVE or court order (IP Law violation, perhaps)? CVE = The Yank flag. Cargo will refuse to add new yanked packages to a lock file, but if a yanked package is already in the lock file, it will still build. The package is not actually deleted. https://doc.rust-lang.org/cargo/commands/cargo-yank.html https://doc.rust-lang.org/cargo/commands/cargo-yank.html Legal = Hard delete. Nobody will go to jail just to avoid breaking your build. Of course, since crates.io and kernel.org are in the same legal jurisdiction, is there any actual difference here?
marginalia_nu 5y ago
This is still fairly short sighted. Websites shut down, large websites with big storage demands are especially vulnerable to attrition. Who wants to pay the mounting bill for keeping decades of revisions of historical rust packages online? I can grab the kernel sources from 1997 and build them today. Will I be able to build rust code from 2022 in 2047, because the 1997 kernel will still build at that date.
- pdw 5y ago
  "I can grab the kernel sources from 1997 and build them today." Where would you be grabbing it from? ...From a website? "Websites shut down, large websites with big storage demands are especially vulnerable to attrition. Who wants to pay the mounting bill for keeping decades of revisions of historical Linux kernels online?"
  - humanrebar 5y ago
    You make a copy, store it on your medium if choice, and put it in a filing cabinet. I gather that certain organizations use magnetic tape backups for especially important data. For some organizations and individuals, kernel source code could be that important.
    - kibwen 5y ago
      You can easily do this with Rust and Cargo as well.
      - marginalia_nu 5y ago
        There is a fairly large difference between archiving your own project's history for as long as you feel like, and archiving the complete history of every significant piece of code ever written in a particular programming language forever.
        kibwen 5y ago