11 ms·

I believe Rust will benefit from the reality check that kernel development represents. Kernel development is hard, and bullshit doesn't go very far in that con

I believe Rust will benefit from the reality check that kernel development represents. Kernel development is hard, and bullshit doesn't go very far in that context. Success for Rust in that environment (with some changes along the way) will be a proof of value.

oxff 4y ago
The issue with Kernel development is that it's a dying profession. Nobody is interested in it. That's why there's an attempt with sexy new language, to bring more developers to work on it. Probably less than 30 people in the world understand the thing as a holistic entity. From evolutionary terms, there's a giant risk of losing relevant technical knowledge to keep it up in the future. I do not support adding it to the Kernel, I think we should just throw away the kernel entirely, but I understand why they're looking at Rust.
- zozbot234 4y ago
  AIUI, you can still build a minimal kernel that's easily understood as a whole. And patches to shrink the minimal build even further are highly sought after because they expand the usability of Linux in deeply embedded environments.
- unmole 4y ago
  > That's why there's an attempt with sexy new language, to bring more developers to work on it. At no point did any active kernel developer express any such sentiment. Nobody is worried about not being unable to attract new blood.
  - pohl 4y ago
    It's only spoken of in knowing glances, like Voldemort.
- capableweb 4y ago
  > Probably less than 30 people in the world understand the thing as a holistic entity As we only have two other "big" (popular) kernels to compare to this one, do you think they (Apple and Microsoft) have more people "holistically" understanding the entire kernel or less? Since the nature of those companies are closed-source, I'm fairly certain even less people understand those ones "holistically". > I do not support adding it to the Kernel, I think we should just throw away the kernel entirely, but I understand why they're looking at Rust. How would that work in reality? Re-use the existing tests to build a new kernel from scratch? Sounds like a very far-out idea that wouldn't help with any of the current problems, but I'm happy to entertain the idea and hear your reasoning here.
  - yjftsjthsd-h 4y ago
    > How would that work in reality? Re-use the existing tests to build a new kernel from scratch? Sounds like a very far-out idea that wouldn't help with any of the current problems, but I'm happy to entertain the idea and hear your reasoning here. While I would tend to agree that a full production replacement would be such a massive undertaking as to be impractical, https://github.com/nuta/kerla https://github.com/nuta/kerla does something very like that - Linux userspace ABI on an all-new Rust kernel. (And even at this small scale, I find it mind-blowing that this worked)
  - ImprobableTruth 4y ago
    Since they're complaining about maintainability, I assume they're advocating for microkernels?
- jerf 4y ago
  The kernel isn't dying, it's niche. It always has been and it always will be. Fortunately for the kernel, despite being niche, it has a rock-solid onramp forcing people to get into it. There will always be companies interested in the n'th degree of performance, both generally, and for some specific hardware. Someone has to go do the relevant kernel work for those things. So while you or I may never touch it, it is effectively impossible for a kernel like Linux to just rot away because nobody cares. It would require first a multi-year, if not multi-decade process of fading first.
  - biorach 4y ago
    nice? It's on most of the smartphones on the planet. Plus it is the dominant server OS. Plus in the top 3 in many embedded categories (a highly diverse set of technologies)
    - teddyh 4y ago
      > It's on most of the smartphones on the planet. Sadly enough, probably not for long: https://en.wikipedia.org/wiki/Fuchsia_(operating_system) https://en.wikipedia.org/wiki/Fuchsia_(operating_system)
      - samhw 4y ago
        Haha, I heard another good one about a nun going into a bar..
        teddyh 4y ago
        Even if Fuchsia is crapware, Google has the money to keep polishing that ball of mud, and eventually force it as the new version of Android, quality be damned. The rest of the world will have no recourse but to switch. It will be painful for quite a while, but people will survive. I mean, people survive using Windows, and not enough people switch away from that, either. I’m not saying that Fuchsia is necessarily bad, I’m saying that Google will do anything to get away from GPL code, including, if necessary, forcing Android to Fuchsia. It doesn’t actually matter if Fuchsia is any good.
- biorach 4y ago
  > The issue with Kernel development is that it's a dying profession. Nobody is interested in it. That's a pretty wild assertion and files in the face of the highly active kernel development process. e.g. > Linus has released 5.18-rc1 and closed the merge window for the 5.18 release... 13,207 non-merge changesets were merged during this merge window. https://lwn.net/Articles/890119/ https://lwn.net/Articles/890119/ > I think we should just throw away the kernel entirely That's the dumbest thing I've see on HN in some months. The kernel is deployed in hundreds of millions of devices worldwide and continues to be the dominant OS in many many sectors.
  - oxff 4y ago
    The issue is about the # of new developers joining the development if you look at the Linux kernel development as an organization, not about # of non-merge changesets merged.
  - staunch 4y ago
    > The kernel is deployed in hundreds of millions of devices... There are billions of Android phones alone. Not to mention the huge numbers of servers, IoT devices, embedded computers, and all the SBCs in my closet.
    - samhw 4y ago
      To be charitable, they may have meant something more like 'personal devices which people use directly'. (Though maybe not - it would be odd to exclude servers, which are one of Linux's biggest 'clients'. Perhaps they meant to exclude the more mundane types: routers, lightbulbs, etc.)
    - pjmlp 4y ago
      That Linux kernel is full of Google specifics, uses a microkernel like architecture since Project Treble, can be compiled since years with clang, now uses Rust for Bluetooth drivers,...
      - staunch 4y ago
- sophacles 4y ago
  Oh no. We have some early career devs that put up patches to the kernel recently. They were super excited about getting to do that work, and it was a big day for them - as it should be it's awesome. I guess I need to go let them know they are Nobodies. oxff said so.
- School-Cotton 4y ago
  > I do not support adding it to the Kernel, I think we should just throw away the kernel entirely What would you use instead?
- LeFantome 4y ago
  There may not be many “big and professional” operating system projects but, at the hobby level, it seems that there is a lot of interest in kernel dev actually. HaikuOS, SerenityOS, Redox, and ReactOS are all going strong. The BSDs continue to advance as well. Redox is written in Rust even. I believe Google sees Fuschia as a true Linux competitor. When you say “we should just throw away the kernel entirely”, what are you suggesting?
- matheusmoreira 4y ago
  > The issue with Kernel development is that it's a dying profession. Nobody is interested in it. That's simply false. I'd love to work on the kernel. I have immense respect for the people who work on it. Only reason I haven't tried contributing code is I don't think I'm skilled enough.
- phendrenad2 4y ago
  > 30 people in the world understand the thing as a holistic entity Probably a lot more than 30. There are probably more than 30 PhD students studying kernels at this very minute. However, I think you're right that no one is interested. People greatly underestimate the effect the kernel has on the full OS. They think it's "just" the kernel. This is why so many people keep saying that Windows will get a Linux kernel in Windows 10, no, 11, no, 12, etc. It's just a "kernel", like a grain of corn, something insignificant. So there isn't much interest in working on kernels. It's as unsexy as it gets.
noobermin 4y ago
It's ironic that the framing of some parts in the article is that the kernel mindset is arrogance and self-assuredness and that somehow isn't applied to the rust developers approaching an ecosystem that isn't familiar. It reminds me of tourists who visit another country and try to get locals to do things the way they are familiar with as if something is wrong with the locals and their way of life. I generally agree with gkh's response here. It avoids the arrogance in the other way ("why can't these kids roll their own leftpad") and presents a more valid concern, that of the kernel's actual constraints.
- lmkg 4y ago
  The person who is leading this project and making it happen, Miguel Ojeda, is a long-term kernel developer. Whatever your experience might be of other RIIR evangelism, this project initiative is coming from an insider rather than an outsider.
  - noobermin 4y ago
    I didn't critique the idea of Rust in linux, I am critiquing the attitude of the rust fellow asking for support of importing generic crates into kernel code and thus adding dependencies (that is, the point of the article).
- oscargrouch 4y ago
  Your example is quite funny and also represents how i would approach the matter of becoming a linux kernel developer. Even if C is not my primary language of choice, i would definitely try adapt myself to the ecosystem and not the other way around. You have all the knowledge of other peers, manual, books, all the libraries, the whole ecosystem.. this cant be replaced. Also there's something else about C nowadays, is the lingua-franca, the latim (or english) of programming languages. We use it to expose api's to others in any other language that want to consume it as a library. There's something about culture that people often forget in tech.. it's the real backbone of any project that it's on its own feet.. and when you want to enter in a community you will be better of if you learn and adapt yourself into this community culture instead of creating cultural clashes into the community and try to overtake it (be it hostile or not). People should be aware that this effort will make it possible to create rust-based kernel drivers and that's it. the RIIR folks are delusional and hype fueled and its better if the sane Rust community get away from them or start to get them back into reality as i bet they are not willing to expend 10 or 15 years of their lives rewriting big and complex piece of software for a likely no return as people will tend to keep using the software the have more community and that are stronger. It's a much better approach for Rust or any other programming language to become research darlings and eventually become the primary ecosystem of a research OS that went well and is the thing that will replace Linux. The language alone wont do it, it must be able to be a contender to UNIX and POSIX, and whatever language that is in such a system will probably be the one that will become the dominant one in such a ecosystem. Also another good approach is to virtualize the Linux Api like gvisor does is userspace or the fuchsia OS(and even FreeBSD) does in the kernelspace. So that you can create your OS and kernel in the best way you can looking ahead, and have this Linux compat layer where applications dont even need to be aware they are not actually running in Linux.
hardwaregeek 4y ago
You have a valid point although I wouldn't frame it as adversarial as much as mutually beneficial. There will certainly be some bullshit eliminated from Rust, but I would not be surprised if there is a similar quantity eliminated from the kernel. Even in the most scrutinized C codebase in the world, there are likely memory usage bugs. If Rust can find and eliminate them, while also improving its capabilities, we all benefit.
throwaway82652 4y ago
>Kernel development is hard, and bullshit doesn't go very far in that context. I don't know what it is about Linux that makes people say this. Kernel development has most of the same constraints as any other embedded context, which Rust has plenty of focus on. No it's not as mature as C, but few languages are. Plus if you go looking in the kernel you can still find plenty of bullshit hacky code. It's not special, it's just another random open source software. The quality very much depends on the individual maintainer of that subsystem and how much has been invested in that area.
npigrounet 4y ago
Rust will quickly replace C in the kernel, I have no doubt about it.
- kibwen 4y ago
  Rust is great, but let's not get ahead of ourselves. :P
- rob74 4y ago
  Just as quickly as it replaced C/C++ in Firefox?
  - zozbot234 4y ago
    Firefox is getting new "oxidized" component all the time, Rust is the recommended language for both refactoring and new development. Of course the lowest-hanging fruits are addressed first, but that's normal and advisable.
    - pjmlp 4y ago
      Not after they let go everyone related to Rust, as far as I am aware. Hence the crazy idea of now using WebAssembly in Firefox modules instead.
      - steveklabnik 4y ago
        That's not correct. Firefox continues to gain new Rust code. Compare https://web.archive.org/web/20201109025230/https://4e6.github.io/firefox-lang-stats/ https://web.archive.org/web/20201109025230/https://4e6.githu..., the first version of this page archive.org saved after the layoffs. 9.5% Rust, 2.9 million LOC. Today https://4e6.github.io/firefox-lang-stats/ https://4e6.github.io/firefox-lang-stats/ 10.1% Rust, 3.4 million LOC.
        pjmlp 4y ago
        10% in a browser currently having 3% market share with a decreasing tendency towards 0%. Most of that code is surely related to what was replaced initially instead of new subsystems being ported.
GolDDranks 4y ago
I think Rust is a good position in the sense that the language and community has a track record and culture of going and solving problems instead of sitting on them forever. I agree that many challenges of kernel development are going to end up strengthening and evolving Rust as a language.
- pjmlp 4y ago
  Solving async traits, having a proper concurrency runtime story, reducing the reliance on third party crates to ease error handling does seem to take out forever.
  - FridgeSeal 4y ago
    > Solving async traits Pretty sure async traits are coming soon (next couple of versions?) which is pretty speedy considering what I understand to be a semi-thorny problem. > having a proper concurrency runtime story Do you mean the language/stdlib shipping an async runtime? > reducing the reliance on third party crates to ease error handling I for one don't rely on 3rd party crates for error handling often? Anyhow is the most common one I use, but mostly out of habit and laziness, not actual hard requirement....
    - mcronce 4y ago
      I usually use `thiserror` for errors, but all that is is a shortcut macro for `impl Error` and `impl Display`
    - jupp0r 4y ago
      Async error handling has been a nightmare last time I tried a few months ago. Has that improved lately?
      - lijogdfljk 4y ago
        Pardon my ignorance, but what do you mean? Handling errors with the async runtime..? If you just mean handling errors from funcs .. it's no different than non-async, no?

oxff 4y ago

The issue with Kernel development is that it's a dying profession. Nobody is interested in it. That's why there's an attempt with sexy new language, to bring more developers to work on it. Probably less than 30 people in the world understand the thing as a holistic entity. From evolutionary terms, there's a giant risk of losing relevant technical knowledge to keep it up in the future. I do not support adding it to the Kernel, I think we should just throw away the kernel entirely, but I understand why they're looking at Rust.

zozbot234 4y ago
AIUI, you can still build a minimal kernel that's easily understood as a whole. And patches to shrink the minimal build even further are highly sought after because they expand the usability of Linux in deeply embedded environments.
unmole 4y ago
> That's why there's an attempt with sexy new language, to bring more developers to work on it. At no point did any active kernel developer express any such sentiment. Nobody is worried about not being unable to attract new blood.
- pohl 4y ago
  It's only spoken of in knowing glances, like Voldemort.
capableweb 4y ago
> Probably less than 30 people in the world understand the thing as a holistic entity As we only have two other "big" (popular) kernels to compare to this one, do you think they (Apple and Microsoft) have more people "holistically" understanding the entire kernel or less? Since the nature of those companies are closed-source, I'm fairly certain even less people understand those ones "holistically". > I do not support adding it to the Kernel, I think we should just throw away the kernel entirely, but I understand why they're looking at Rust. How would that work in reality? Re-use the existing tests to build a new kernel from scratch? Sounds like a very far-out idea that wouldn't help with any of the current problems, but I'm happy to entertain the idea and hear your reasoning here.
- yjftsjthsd-h 4y ago
  > How would that work in reality? Re-use the existing tests to build a new kernel from scratch? Sounds like a very far-out idea that wouldn't help with any of the current problems, but I'm happy to entertain the idea and hear your reasoning here. While I would tend to agree that a full production replacement would be such a massive undertaking as to be impractical, https://github.com/nuta/kerla https://github.com/nuta/kerla does something very like that - Linux userspace ABI on an all-new Rust kernel. (And even at this small scale, I find it mind-blowing that this worked)
- ImprobableTruth 4y ago
  Since they're complaining about maintainability, I assume they're advocating for microkernels?
jerf 4y ago
The kernel isn't dying, it's niche. It always has been and it always will be. Fortunately for the kernel, despite being niche, it has a rock-solid onramp forcing people to get into it. There will always be companies interested in the n'th degree of performance, both generally, and for some specific hardware. Someone has to go do the relevant kernel work for those things. So while you or I may never touch it, it is effectively impossible for a kernel like Linux to just rot away because nobody cares. It would require first a multi-year, if not multi-decade process of fading first.
- biorach 4y ago
  nice? It's on most of the smartphones on the planet. Plus it is the dominant server OS. Plus in the top 3 in many embedded categories (a highly diverse set of technologies)
  - teddyh 4y ago
    > It's on most of the smartphones on the planet. Sadly enough, probably not for long: https://en.wikipedia.org/wiki/Fuchsia_(operating_system) https://en.wikipedia.org/wiki/Fuchsia_(operating_system)
    - samhw 4y ago
      Haha, I heard another good one about a nun going into a bar..
      - teddyh 4y ago
        Even if Fuchsia is crapware, Google has the money to keep polishing that ball of mud, and eventually force it as the new version of Android, quality be damned. The rest of the world will have no recourse but to switch. It will be painful for quite a while, but people will survive. I mean, people survive using Windows, and not enough people switch away from that, either. I’m not saying that Fuchsia is necessarily bad, I’m saying that Google will do anything to get away from GPL code, including, if necessary, forcing Android to Fuchsia. It doesn’t actually matter if Fuchsia is any good.
biorach 4y ago
> The issue with Kernel development is that it's a dying profession. Nobody is interested in it. That's a pretty wild assertion and files in the face of the highly active kernel development process. e.g. > Linus has released 5.18-rc1 and closed the merge window for the 5.18 release... 13,207 non-merge changesets were merged during this merge window. https://lwn.net/Articles/890119/ https://lwn.net/Articles/890119/ > I think we should just throw away the kernel entirely That's the dumbest thing I've see on HN in some months. The kernel is deployed in hundreds of millions of devices worldwide and continues to be the dominant OS in many many sectors.
- oxff 4y ago
  The issue is about the # of new developers joining the development if you look at the Linux kernel development as an organization, not about # of non-merge changesets merged.
- staunch 4y ago
  > The kernel is deployed in hundreds of millions of devices... There are billions of Android phones alone. Not to mention the huge numbers of servers, IoT devices, embedded computers, and all the SBCs in my closet.
  - samhw 4y ago
    To be charitable, they may have meant something more like 'personal devices which people use directly'. (Though maybe not - it would be odd to exclude servers, which are one of Linux's biggest 'clients'. Perhaps they meant to exclude the more mundane types: routers, lightbulbs, etc.)
  - pjmlp 4y ago
    That Linux kernel is full of Google specifics, uses a microkernel like architecture since Project Treble, can be compiled since years with clang, now uses Rust for Bluetooth drivers,...
    - staunch 4y ago
sophacles 4y ago
Oh no. We have some early career devs that put up patches to the kernel recently. They were super excited about getting to do that work, and it was a big day for them - as it should be it's awesome. I guess I need to go let them know they are Nobodies. oxff said so.
School-Cotton 4y ago
> I do not support adding it to the Kernel, I think we should just throw away the kernel entirely What would you use instead?
LeFantome 4y ago
There may not be many “big and professional” operating system projects but, at the hobby level, it seems that there is a lot of interest in kernel dev actually. HaikuOS, SerenityOS, Redox, and ReactOS are all going strong. The BSDs continue to advance as well. Redox is written in Rust even. I believe Google sees Fuschia as a true Linux competitor. When you say “we should just throw away the kernel entirely”, what are you suggesting?
matheusmoreira 4y ago
> The issue with Kernel development is that it's a dying profession. Nobody is interested in it. That's simply false. I'd love to work on the kernel. I have immense respect for the people who work on it. Only reason I haven't tried contributing code is I don't think I'm skilled enough.
phendrenad2 4y ago
> 30 people in the world understand the thing as a holistic entity Probably a lot more than 30. There are probably more than 30 PhD students studying kernels at this very minute. However, I think you're right that no one is interested. People greatly underestimate the effect the kernel has on the full OS. They think it's "just" the kernel. This is why so many people keep saying that Windows will get a Linux kernel in Windows 10, no, 11, no, 12, etc. It's just a "kernel", like a grain of corn, something insignificant. So there isn't much interest in working on kernels. It's as unsexy as it gets.

noobermin 4y ago

It's ironic that the framing of some parts in the article is that the kernel mindset is arrogance and self-assuredness and that somehow isn't applied to the rust developers approaching an ecosystem that isn't familiar. It reminds me of tourists who visit another country and try to get locals to do things the way they are familiar with as if something is wrong with the locals and their way of life. I generally agree with gkh's response here. It avoids the arrogance in the other way ("why can't these kids roll their own leftpad") and presents a more valid concern, that of the kernel's actual constraints.

lmkg 4y ago
The person who is leading this project and making it happen, Miguel Ojeda, is a long-term kernel developer. Whatever your experience might be of other RIIR evangelism, this project initiative is coming from an insider rather than an outsider.
- noobermin 4y ago
  I didn't critique the idea of Rust in linux, I am critiquing the attitude of the rust fellow asking for support of importing generic crates into kernel code and thus adding dependencies (that is, the point of the article).
oscargrouch 4y ago
Your example is quite funny and also represents how i would approach the matter of becoming a linux kernel developer. Even if C is not my primary language of choice, i would definitely try adapt myself to the ecosystem and not the other way around. You have all the knowledge of other peers, manual, books, all the libraries, the whole ecosystem.. this cant be replaced. Also there's something else about C nowadays, is the lingua-franca, the latim (or english) of programming languages. We use it to expose api's to others in any other language that want to consume it as a library. There's something about culture that people often forget in tech.. it's the real backbone of any project that it's on its own feet.. and when you want to enter in a community you will be better of if you learn and adapt yourself into this community culture instead of creating cultural clashes into the community and try to overtake it (be it hostile or not). People should be aware that this effort will make it possible to create rust-based kernel drivers and that's it. the RIIR folks are delusional and hype fueled and its better if the sane Rust community get away from them or start to get them back into reality as i bet they are not willing to expend 10 or 15 years of their lives rewriting big and complex piece of software for a likely no return as people will tend to keep using the software the have more community and that are stronger. It's a much better approach for Rust or any other programming language to become research darlings and eventually become the primary ecosystem of a research OS that went well and is the thing that will replace Linux. The language alone wont do it, it must be able to be a contender to UNIX and POSIX, and whatever language that is in such a system will probably be the one that will become the dominant one in such a ecosystem. Also another good approach is to virtualize the Linux Api like gvisor does is userspace or the fuchsia OS(and even FreeBSD) does in the kernelspace. So that you can create your OS and kernel in the best way you can looking ahead, and have this Linux compat layer where applications dont even need to be aware they are not actually running in Linux.

hardwaregeek 4y ago

You have a valid point although I wouldn't frame it as adversarial as much as mutually beneficial. There will certainly be some bullshit eliminated from Rust, but I would not be surprised if there is a similar quantity eliminated from the kernel. Even in the most scrutinized C codebase in the world, there are likely memory usage bugs. If Rust can find and eliminate them, while also improving its capabilities, we all benefit.

throwaway82652 4y ago

>Kernel development is hard, and bullshit doesn't go very far in that context. I don't know what it is about Linux that makes people say this. Kernel development has most of the same constraints as any other embedded context, which Rust has plenty of focus on. No it's not as mature as C, but few languages are. Plus if you go looking in the kernel you can still find plenty of bullshit hacky code. It's not special, it's just another random open source software. The quality very much depends on the individual maintainer of that subsystem and how much has been invested in that area.

npigrounet 4y ago

Rust will quickly replace C in the kernel, I have no doubt about it.

kibwen 4y ago
Rust is great, but let's not get ahead of ourselves. :P
rob74 4y ago
Just as quickly as it replaced C/C++ in Firefox?
- zozbot234 4y ago
  Firefox is getting new "oxidized" component all the time, Rust is the recommended language for both refactoring and new development. Of course the lowest-hanging fruits are addressed first, but that's normal and advisable.
  - pjmlp 4y ago
    Not after they let go everyone related to Rust, as far as I am aware. Hence the crazy idea of now using WebAssembly in Firefox modules instead.
    - steveklabnik 4y ago
      That's not correct. Firefox continues to gain new Rust code. Compare https://web.archive.org/web/20201109025230/https://4e6.github.io/firefox-lang-stats/ https://web.archive.org/web/20201109025230/https://4e6.githu..., the first version of this page archive.org saved after the layoffs. 9.5% Rust, 2.9 million LOC. Today https://4e6.github.io/firefox-lang-stats/ https://4e6.github.io/firefox-lang-stats/ 10.1% Rust, 3.4 million LOC.
      - pjmlp 4y ago
        10% in a browser currently having 3% market share with a decreasing tendency towards 0%. Most of that code is surely related to what was replaced initially instead of new subsystems being ported.

GolDDranks 4y ago

I think Rust is a good position in the sense that the language and community has a track record and culture of going and solving problems instead of sitting on them forever. I agree that many challenges of kernel development are going to end up strengthening and evolving Rust as a language.

pjmlp 4y ago
Solving async traits, having a proper concurrency runtime story, reducing the reliance on third party crates to ease error handling does seem to take out forever.
- FridgeSeal 4y ago
  > Solving async traits Pretty sure async traits are coming soon (next couple of versions?) which is pretty speedy considering what I understand to be a semi-thorny problem. > having a proper concurrency runtime story Do you mean the language/stdlib shipping an async runtime? > reducing the reliance on third party crates to ease error handling I for one don't rely on 3rd party crates for error handling often? Anyhow is the most common one I use, but mostly out of habit and laziness, not actual hard requirement....
  - mcronce 4y ago
    I usually use `thiserror` for errors, but all that is is a shortcut macro for `impl Error` and `impl Display`
  - jupp0r 4y ago
    Async error handling has been a nightmare last time I tried a few months ago. Has that improved lately?
    - lijogdfljk 4y ago
      Pardon my ignorance, but what do you mean? Handling errors with the async runtime..? If you just mean handling errors from funcs .. it's no different than non-async, no?