7 ms·

Half of the ecosystem depends on the nightly compiler.

by vips7L 4y ago

wongarsu 4y ago
That used to be true a couple years ago, but I don't think I've used a nightly compiler version at any point in the last 12 months. That said, I think for this first phase the kernel is planning on a specific stable version of the compiler with nightly features enabled, because they have a couple specific needs that haven't made it through the stabilization process yet.
smolder 4y ago
I dont't think "half" is accurate, at least not anymore. A lot of things have moved from Nightly to Stable compatibility with stabilization of async, for example. Rocket builds on stable now, while for a long time it didn't. I'd like to see the actual percentage but AFAIK it would take some analysis. Crates.io shows 86417 total crates, searching for "stable" shows 3346 and "nightly" shows 2527... So that's not a good heuristic, probably.
- rdsubhas 4y ago
  Parent is saying the proportion is very high. It's not the exact number that matters. Even if one dependency your project uses needs nightly, then your entire project has to use nightly.
  - nindalf 4y ago
    Less than 20% of devs use nightly. About a third of the devs who use nightly do it because of a dependency. 6-7% of Rust devs is a far cry from the “half” claimed by GP. https://github.com/rust-lang/surveys/raw/main/surveys/2021-annual-survey/2021-annual-survey-summary.pdf https://github.com/rust-lang/surveys/raw/main/surveys/2021-a...
    - arinlen 4y ago
      > Less than 20% of devs use nightly. You tried to make that sound like few Rust developers use nightly, but the same survey you linked to states that, while 7449 said they use the latest stable version for local development, 3333 said they use nightly, with 2853of them using the latest nightly and 481 using a specific version of nightly. All in all, the ratio of Rust developers targeting nightly releases or betas represents something in between a third and a quarter of the whole developer community. That's a lot of people in the Rust community delivering code targeting unstable and unmaintainable releases.
      - nindalf 4y ago
        > unmaintainable releases How did you come to this conclusion? Seems like it must be working for them, because most of them are using nightly out of choice. Plus it’s a service to the community. Their everyday testing helps make the stable releases better. Imagine if no one used nightly - that would be much worse.
  - woodruffw 4y ago
    This is true, but you'd have to show that those nightly dependencies are relatively central to the Rust dependency graph (i.e., that lots of other things depend on them). To the best of my knowledge, that's not really the case (the really central dependencies in the Rust ecosystem are things like serde, libc, anyhow, etc. that all use the stable compiler). In my experience writing Rust professionally (including making heavy use of dependencies), I've only needed the nightly compiler once. And that was for our own code, not an external dependency.
- steveklabnik 4y ago
  It’s impossible to deep link into a PDF, but the raw survey data from last year is out: https://blog.rust-lang.org/inside-rust/2022/06/21/survey-2021-report.html https://blog.rust-lang.org/inside-rust/2022/06/21/survey-202... It shows that the vast majority of people use latest stable. A decent chunk also use nightly, and the reason why “a crate dependency I need requires it” is pretty far down the list of reasons. Oh and a pet theory: the way you feel about this is correlated with the kind of work you do. I’d you’re writing an OS, you probably use nightly still, and so your dependencies probably do too. But if you’re higher up the stack, where unsafe is less common and the appropriate language features are more filled out, you probably never use nightly. This makes talking past people unfortunately easy to do.) (n.b. I’m glad they released the raw data, because otherwise we wouldn’t be able to share this, because it wasn’t included in the blog post this year. I brought this up before publishing but my advice was ignored, oh well!)
  - lifthrasiir 4y ago
    I believe many browser-embedded PDF readers now support `page` fragments [1]: https://raw.githubusercontent.com/rust-lang/surveys/main/surveys/2021-annual-survey/2021-annual-survey-summary.pdf#page=15 https://raw.githubusercontent.com/rust-lang/surveys/main/sur... [1] https://datatracker.ietf.org/doc/html/rfc8118#page-4 https://datatracker.ietf.org/doc/html/rfc8118#page-4
    - steveklabnik 4y ago
      Oh fun! Thanks.
  - simlevesque 4y ago
    Back in the days, before deep links... we'd just say the page number.
TotempaaltJ 4y ago
I've not experienced this. I know it used to be very significant, but on my latest (and biggest) project that I've been working on for ~a year, I haven't run into any nightly-only packages that I couldn't easily do without.
eternauta3k 4y ago
Would the kernel pull in any dependencies, or just reimplement everything like it currently does?
- bonzini 4y ago
  It doesn't use cargo but it does use the core and alloc crates. Some parts such as Mutex are reimplemented for interoperability with C code.
ungamedplayer 4y ago
Almost none of that ecosystem is relevant to kernel.
- arinlen 4y ago
  > Almost none of that ecosystem is relevant to kernel. What exactly leads you to believe that? From the Rust 2021 survey, 2420 said they use nightly to access one or more language features they need, 1028 claimed a Crate dependency requires it, and 876 claimed a tool they use requires it. This, from a sample pool where around 3k users out of around 10k claim they use a nightly version.
  - nindalf 4y ago
    Because the kernel code isn’t pulling dependency code from the internet OR a compiler that changes every day. Kernel code will be checked into the repo, dependencies if any, will be vendored. Nor will much of crates.io be relevant - the kernel will only use no_std with fallible allocations. As for the compiler, it will use a pinned, stable compiler, not nightly. Why are you spreading FUD about nightly + Linux on this thread? At no point has this combination ever been considered.
  - ungamedplayer 4y ago
    Again. Almost none of these can run in kernel space, those that are required will be duplicated in tree. The survey results are not relevant. The rust code must not use stdlib (the name escapes me) which uses an incompatible memory allocation mechanism. The code used to build will likely need to be checked into the kernel git repo and not downloaded live online, I doubt that the mature development process of the kernel wants the problems that other package managers have. I will fight to ensure that this is not the case in the kernels that I work with, because fetching dependencies is for people who can tolerate other people and networks being faulty. I got shit to do.

Half of the ecosystem depends on the nightly compiler.

by vips7L 4y ago

Half of the ecosystem depends on the nightly compiler.

That used to be true a couple years ago, but I don't think I've used a nightly compiler version at any point in the last 12 months. That said, I think for this first phase the kernel is planning on a specific stable version of the compiler with nightly features enabled, because they have a couple specific needs that haven't made it through the stabilization process yet.

smolder 4y ago

I dont't think "half" is accurate, at least not anymore. A lot of things have moved from Nightly to Stable compatibility with stabilization of async, for example. Rocket builds on stable now, while for a long time it didn't. I'd like to see the actual percentage but AFAIK it would take some analysis. Crates.io shows 86417 total crates, searching for "stable" shows 3346 and "nightly" shows 2527... So that's not a good heuristic, probably.

rdsubhas 4y ago
Parent is saying the proportion is very high. It's not the exact number that matters. Even if one dependency your project uses needs nightly, then your entire project has to use nightly.
- nindalf 4y ago
  Less than 20% of devs use nightly. About a third of the devs who use nightly do it because of a dependency. 6-7% of Rust devs is a far cry from the “half” claimed by GP. https://github.com/rust-lang/surveys/raw/main/surveys/2021-annual-survey/2021-annual-survey-summary.pdf https://github.com/rust-lang/surveys/raw/main/surveys/2021-a...
  - arinlen 4y ago
    > Less than 20% of devs use nightly. You tried to make that sound like few Rust developers use nightly, but the same survey you linked to states that, while 7449 said they use the latest stable version for local development, 3333 said they use nightly, with 2853of them using the latest nightly and 481 using a specific version of nightly. All in all, the ratio of Rust developers targeting nightly releases or betas represents something in between a third and a quarter of the whole developer community. That's a lot of people in the Rust community delivering code targeting unstable and unmaintainable releases.
    - nindalf 4y ago
      > unmaintainable releases How did you come to this conclusion? Seems like it must be working for them, because most of them are using nightly out of choice. Plus it’s a service to the community. Their everyday testing helps make the stable releases better. Imagine if no one used nightly - that would be much worse.
- woodruffw 4y ago
  This is true, but you'd have to show that those nightly dependencies are relatively central to the Rust dependency graph (i.e., that lots of other things depend on them). To the best of my knowledge, that's not really the case (the really central dependencies in the Rust ecosystem are things like serde, libc, anyhow, etc. that all use the stable compiler). In my experience writing Rust professionally (including making heavy use of dependencies), I've only needed the nightly compiler once. And that was for our own code, not an external dependency.
steveklabnik 4y ago
It’s impossible to deep link into a PDF, but the raw survey data from last year is out: https://blog.rust-lang.org/inside-rust/2022/06/21/survey-2021-report.html https://blog.rust-lang.org/inside-rust/2022/06/21/survey-202... It shows that the vast majority of people use latest stable. A decent chunk also use nightly, and the reason why “a crate dependency I need requires it” is pretty far down the list of reasons. Oh and a pet theory: the way you feel about this is correlated with the kind of work you do. I’d you’re writing an OS, you probably use nightly still, and so your dependencies probably do too. But if you’re higher up the stack, where unsafe is less common and the appropriate language features are more filled out, you probably never use nightly. This makes talking past people unfortunately easy to do.) (n.b. I’m glad they released the raw data, because otherwise we wouldn’t be able to share this, because it wasn’t included in the blog post this year. I brought this up before publishing but my advice was ignored, oh well!)
- lifthrasiir 4y ago
  I believe many browser-embedded PDF readers now support `page` fragments [1]: https://raw.githubusercontent.com/rust-lang/surveys/main/surveys/2021-annual-survey/2021-annual-survey-summary.pdf#page=15 https://raw.githubusercontent.com/rust-lang/surveys/main/sur... [1] https://datatracker.ietf.org/doc/html/rfc8118#page-4 https://datatracker.ietf.org/doc/html/rfc8118#page-4
  - steveklabnik 4y ago
    Oh fun! Thanks.
- simlevesque 4y ago
  Back in the days, before deep links... we'd just say the page number.

TotempaaltJ 4y ago

I've not experienced this. I know it used to be very significant, but on my latest (and biggest) project that I've been working on for ~a year, I haven't run into any nightly-only packages that I couldn't easily do without.

eternauta3k 4y ago

Would the kernel pull in any dependencies, or just reimplement everything like it currently does?

bonzini 4y ago
It doesn't use cargo but it does use the core and alloc crates. Some parts such as Mutex are reimplemented for interoperability with C code.

ungamedplayer 4y ago

Almost none of that ecosystem is relevant to kernel.

arinlen 4y ago
> Almost none of that ecosystem is relevant to kernel. What exactly leads you to believe that? From the Rust 2021 survey, 2420 said they use nightly to access one or more language features they need, 1028 claimed a Crate dependency requires it, and 876 claimed a tool they use requires it. This, from a sample pool where around 3k users out of around 10k claim they use a nightly version.
- nindalf 4y ago
  Because the kernel code isn’t pulling dependency code from the internet OR a compiler that changes every day. Kernel code will be checked into the repo, dependencies if any, will be vendored. Nor will much of crates.io be relevant - the kernel will only use no_std with fallible allocations. As for the compiler, it will use a pinned, stable compiler, not nightly. Why are you spreading FUD about nightly + Linux on this thread? At no point has this combination ever been considered.
- ungamedplayer 4y ago
  Again. Almost none of these can run in kernel space, those that are required will be duplicated in tree. The survey results are not relevant. The rust code must not use stdlib (the name escapes me) which uses an incompatible memory allocation mechanism. The code used to build will likely need to be checked into the kernel git repo and not downloaded live online, I doubt that the mature development process of the kernel wants the problems that other package managers have. I will fight to ensure that this is not the case in the kernels that I work with, because fetching dependencies is for people who can tolerate other people and networks being faulty. I got shit to do.