23 ms·

Rust for Linux redux

161 points by akvadrako 5y ago

encryptluks2 5y ago
I'd honestly just prefer Rust to have a separate patched kernel and not to have it mainlined.
- mattashii 5y ago
  Could you clarify? It seems to me that a build option is preferable to needing to maintain/checkout a seperate branch altogether
  - encryptluks2 5y ago
    Many distros ship hardened and patched kernels. As long as the distro provides packages it wouldn't really be any extra work.
    - remexre 5y ago
      It being a build option doesn't preclude distros shipping multiple packages, some with it enabled and some not?
- pjmlp 5y ago
  Here you go, https://source.android.com/ https://source.android.com/
staticassertion 5y ago
> Since most of our CVE type problems are usually programming mistakes nowadays, the lack of review could contribute to an increase in programming fault type bugs which aren't forbidden by the safer memory model. That's an odd statement. What CVEs aren't due to programming mistakes? I'm not sure if the majority of CVEs for the Linux kernel come down to memory safety, though I would not be surprised, but certainly a huge number are. > without a clear and obvious benefit beyond promises that can only truly be fully fulfilled with a whole kernel written in Rust. That's not true, really. You don't need a completely safe kernel to have an improvement to safety. If every device driver were memory safe tomorrow we'd be better off. That said, I think this will be an interesting, possibly losing, battle. The Linux Kernel is extremely monolithic, it has a lack of testing and code review, decades of dug-in investments, and a strong history of not prioritizing security or even considering it to be a legitimate goal. Fixing that seems like it will itself take decades, whereas the current approach really feels like it's trying to get it done ASAP. If they can do it, cool. As a Linux user I'll possibly benefit. I'm curious to see how it plays out.
- CraigJPerry 5y ago
  >> it has a lack of testing and code review Is that really true? I mean my first reaction is that the linked article / thread is a direct counterpoint. All the review and testing the rust facilities are going through. Are these changes not going through substantively the same process as other proposed changes to the kernel?
  - medstrom 5y ago
    I don't think it lacks code review, it's famously hard to get half-baked stuff through Torvalds.
    - staticassertion 5y ago
      That isn't really true.
    - wott 5y ago
      The ton of driver code that piles up in the Linux kernel every minute doesn't go through Torvalds. It is delegated. And my reviewing of a few drivers source short commits is enough to tell me that those delegates do not perform a satisfyingly thorough review by any mean. Heck, I saw patches of just a couple dozen lines which exhibited bad copy-pasting errors anyone without prior knowledge could have spotted. You don't need to know what the code does to spot some, you don't either need to know what the driven device does to spot some: purely formal errors with bad macros definitions for example. This kind of stuff wouldn't even pass the first internal review where I worked, which just looked at formal appearance (then there were more in depth reviews, and then there was an external review, but we'd make as sure as possible that our code would be clean before going out). So first you have people (employees of company A) which sends code to a public, external project without having done a proper internal review. Then you have someone else (employee from company B) who claims to have reviewed the commit but hasn't done it properly or at all. And then possibly a third someone who validates this, but doesn't actually check either. It has become a job, a task like another, with the same people who do the same sloppy job as quickly as possible to get rid of it and go home earlier or slack, in the same proportion that you can find in any other position in the world.
ayane_m 5y ago
The FUD around Rusting the kernel reminds me of similar sentiments surrounding autonomous vehicles. Better != perfect; it's an evolutionary step. Even if there are unintended negative consequences of Rust code, if they are less frequent than the rate we deal with bugs today, then it's worth using.
- simion314 5y ago
  We need to keep in mind that are 2 types of autonomus vehicles discussion. We have a company that uses a ton of hardware, radar, lidar and many cameras and then we have the other ones that want to move fast and break things using only cameras, a GPU and many beta testers. It is normal that the aproach of brute forcing it with ton of data gets a ton of criticism. My other criticism is the bad statistics used. It is like I create the "robot athlete" and I compare it stats with the average of all athletes including the young children and the people with some physical problem. You should compare self driving with cars with exact same safety features and save driver demographics. Bonus if you calculate all deaths caused by illegal driving and then ask the giants WTF not put the money into first solve the speeding and drunk / tired driving , I bet ANN work better on this problem. Rust in Linux seems to me a waste. IMO the Unix philosophy is great but it needs a better implementation , one that is based on the present day hardware and expectations.
  - sebzim4500 5y ago
    > Bonus if you calculate all deaths caused by illegal driving and then ask the giants WTF not put the money into first solve the speeding and drunk / tired driving , I bet ANN work better on this problem. Why would they want to do that though? No one would buy a car with those features. I guess you could get a few people to buy one if the insurance was way lower, but you certainly wouldn't get decent market penetration.
    - simion314 5y ago
      So say someone builds a system that you install in cars that monitors how you drive, it can detect if you respect the speeding laws, if you drive normal or you have risky behavior. Few will want this in their car but they will want it on the others car so I can see this possibilities: - a law that requires it in all cars - a law that requires it in new cars - a law that requires it only for new drivers (less then 2 year) and for people caught speeding/drunk. - a law that will make certain roads only available for self driving cars or for people with this safety system. It would be a compromise between AI drivers camp and people that still want to drive themselves. Maybe you will say something about privacy, this system can be implemented with no connectivity to spy on you. It could be read only when you want to pay your insurance or in case of an accident. But also AI can be implemented in a different way, like you could have some quality cameras recording traffic and have the AI detect who is using the phone, who is not paying attention , who is moving in a erratic pattern , so it would be an advanced spending camera. The argument is that the AI driver camp will demand humans to be removed from the road, because their AI driver is better then the average. To prevent you losing the right to drive then this average needs to be improved and this can be done by removing the bad drivers and a good place to start is the ones that do not respect the rules. Otherwise you might object to install a safety system in your car but then the big companies will lobby hard and you will have to use the Tesla/Google or Apple AI powered cars, then not only that he government will know all about your movement but the ad companies will know it too.
robertwt7 5y ago
It is good that rust is slowly being adopted everywhere. Gives me more reason to learn it..
- wiz21c 5y ago
  Rust helps a lot with memory or concurrency issues. But not the rest. And in my experience, although the concurrency/memory issue are surely the most time consuming bugs, the "rest" is what takes 90% of the time...
  - kortex 5y ago
    To each their own. I find Rust a huge step up from C and even C++...especially C++. I've been writing Rust for months and C++ for years. I still feel more comfortable and confident my Rust code has fewer bugs and does what I intend vs C++.
  - viraptor 5y ago
    There's more that it helps with. For one example Rust's Enums and richer types allow fewer mistakes -vs- confusion where pointers and integers degrade to a common type. A random line: https://github.com/torvalds/linux/blob/122fa8c588316aacafe7e5a393bb3e875eaf5b25/include/crypto/aead.h#L283 https://github.com/torvalds/linux/blob/122fa8c588316aacafe7e... > void crypto_aead_clear_flags(struct crypto_aead *tfm, u32 flags) These are valid due to poor type usage, but make no sense: crypto_aead_clear_flags(..., PAGE_SIZE) crypto_aead_clear_flags(..., EBUSY) crypto_aead_clear_flags(..., SOME_RANDOM_THING) We can hope they would be caught in review if the code happens to do the right thing by accident, or use static analysis and hope it triggers in that case, or use a language with clearer types where it's much easier to use types preventing this issue.
    - pjmlp 5y ago
      You could the same with C++ or Ada, but naturally it will never be a thing in Linux.
      - 5y ago
greatgib 5y ago
For me, the issues with Rust are: - It is not fun but frustrating to work in Rust, and contrary to C, you are limited by the language/compiler on what you can do. - building/compiling the kernel is not trivial, and you will add a new huge dependency that you have to deal with to build the kernel for whatever target. Let's suppose you want to build for MIPS, then you need to have Rust supporting MIPS. As an exemple, there is a common package in python that decided to start having their module in Rust instead of C. Now a lot of users of the module are pissed off, with good reason, because you can't build/install the module anymore in a little bit older or non conventional distributions.
- wiz21c 5y ago
  You are not limited. You are strongly driven. It's different. Painfully different, I agree. But as time goes by, it becomes painful.
- choeger 5y ago
  The latter point is an important one. Rust as a language for libraries cannot work the same way as rust as a language for applications. For the latter it is OK to depend on the cargo toolchain and be opinionated when it comes to things like dynamic linking. For the former you ideally want support in any common compiler (clang, GCC) and as little dependencies and constraints as possible.
  - bsdubernerd 5y ago
    It's also a problem for application themselves IMO. Cargo combined with dependency pinning brings most of the disadvantages of similar environments with centralized package handling: the ease of adding package dependencies increases the number of dependencies themselves very rapidly. Overly-narrow version pinning forces per-package lockstep updates of the various dependencies, which in turn means multiple versions of the same package will get pulled two-or-three levels deep. This ensures each single rust package or update you build is almost a guaranteed rebuild-the-world even with a shared cache dir. And we're not touching the problem of building projects where rust is only part of it: annoying if you want to link other stuff into rust, even more so if you want to do the opposite. I'm following a couple of projects that transitioned to rust, and my experience as a contributor is not stellar. A minimal rust project can take hundreds of mb of disk space just in dependencies, and double that for build space. The solution for some has been providing build bots, but again doesn't help me as a contributor, where I need to be able to rebuild from source. This has on me almost the same effect produced by large projects: I only contribute if I have a large vested interest in the package, otherwise I just avoid because it's time consuming.
kortex 5y ago
> In general, I would avoid exposing Rust subsystems to C. It is possible, of course, and it gives you the advantages of Rust in the implementation of the subsystem. However, by having to expose a C API, you would be losing most of the advantages of the richer type system, plus the guarantees that Rust bring as a language for the consumers of the subsystem. This feels like putting the cart before the horse. "we shouldn't integrate Rust into the kernel in a modular sort of way cause it's not as optimal as more thorough type-aware integration." Like, cross that bridge when you come to it, eh? The kernel is currently comprised of hundreds if not thousands of components, talking to each other via C API/ABIs. And talking to hardware is almost entirely ABI. That is how the kernel do. Would we benefit from more type knowledge across boundaries? Absolutely, but this is hard (especially given how different the c and rust approaches to allocation are), and shouldn't stand in the way of progress.
- zozbot234 5y ago
  Exposing a C API/ABI is also the only supported way of making separate Rust crates that can be truly interoperable beyond the constraints of a single rustc build. It's not a limit to high-level interfaces because one can always build those as higher-level layers within a single crate, and there are tools (bindgen and cbindgen) to make this workflow a bit easier.
- brundolf 5y ago
  I agree. I've migrated multiple large codebases from a weakly-typed language to a strongly-typed language, and in my experience this is the only way to do it. And it's also not as bad as the OP makes it sound: you get your stronger guarantees gradually, and that's fine. One gotcha, though: a stronger type system encourages the removal of runtime checks. When doing a partial conversion, I've found it's best to leave any existing runtime checks in place, even if the type system makes them "redundant", because it doesn't really until the calling code has also been converted. Once it has, you can go back and remove them.
  - brundolf 5y ago
    (Clarification that these codebases were obviously not as large as Linux, and I don't mean to imply that the job was anywhere near the same scale as this one would be, I just think some of the same lessons are broadly applicable)
dash2 5y ago
I think this is gonna be a hard sell. The standard economic analysis of cooperatives is that, being run by their employees, they find it hard to embrace disruptive change (e.g. change which involves firing people). Hence, you probably buy your food from a shareholder-owned business, not an employee-owned cooperative. Linux is a bit like a cooperative in that decisions are made by the software engineers, not by shareholders or managers. In particular, most Linux contributors are probably heavily invested in C. If Rust gets the big boost of being allowed into the kernel, that could make C - and their own skills - be perceived as less valuable, maybe even obsolete eventually. I'll do some economic imperialism here, and claim that Rust's technical merits or otherwise are a second-order issue. Linux developers have an interest in keeping C dominant, so that will probably happen.
- jleahy 5y ago
  The average Linux kernel developer is probably confident in 5-10 languages, and I can’t imagine they would really have trouble picking up Rust. That kind of argument might hold for the average junior javascript dev, but kernel developers are normally an experienced bunch and C is rarely learnt today as a sole language.
  - paulddraper 5y ago
    Be that as it may, thanks to selection bias, the vast majority of kernel devs still probably prefer working in C.
  - dash2 5y ago
    Yes, I'm sure they're extremely smart, but equally, they are deeply expert in C, and it is not that easy to become deeply expert in Rust (or any other language). Even smart people can't just pick up new tools in a day.
    - Mic92 5y ago
      They way rust is used in the kernel is a lot more constrained in scope and APIs compared to normal userland rust and hence easier to learn. A lot of added rust API are also just high-level abstraction of already existing concepts/code used in the kernel.
InTheArena 5y ago
Every now and then you see companies make insanely large big bets that radically change a fundamental technology because of a in addressable need. I'm increasingly convinced that a good chunk of MacOS/iOS security problems are due to the simple fact that it is very difficult to write correct system level code w/ C. If (and it is a big if) the performance of rust can be in the same neighborhood as C - and these kinks can be worked out, then as a architect, I think it might be time for a ideological come to Jesus moment/protest reformation of the kernel.
Santosh83 5y ago
If there is industry backing, it will happen. The coming generation of programmers (and hence the future of the kernel) are nowhere as near enthused to program in C as they're Rust, so this transition is inevitable given time. Especially if the kernel should not slowly die out over the decades, stuck permanently to a fossilised language.
phendrenad2 5y ago
I understand what the kernel devs are afraid of. Rust needs to demonstrate beyond a shadow of a doubt that it's a good replacement for C in the kernel, or they risk alienating people who don't want to learn another language, and losing those developers to FreeBSD or something.
frazzyfraz 5y ago
This is just a non-clever attempt by double-vaxxed NPCs to SJW Linux over.
ThinkBeat 5y ago
The kernel has accumulated a lot of technical debt. Linus speaks openly about it. Mixing in the new flavor into a codebase that is 95% (?) using (and abusing) C with warts and all does not strike me as a good idea. Interaction between assembly, C and Rust is not nearly well, enough understood to start. Now if it does go ahead, it will be a great platform to figure out the edge cases, bugs and all those things nobody thought about. I am in favor of Linux.NEXT Write a new kernel in Rust (if it's the best choice) Take all the learnings from Linux and all the research in operating systems that have occurred since and make Linux.NEXT. Steal from Linux, WindowsNT, Quebes, OS/400, Solaris, Minix, z/OS Plan 9, Fuchsia, BeOS/Haiku, ESXi, QNX, and of course TempleOS Creating an operating system suitable for 2020 with the way technology, infrastructure, connectiveness, pervasiveness, privacy conscious, built for the hostile world of the Internet.
- pjmlp 5y ago
  > Steal from Linux, WindowsNT, Quebes, OS/400, Solaris, Minix, z/OS Plan 9, Fuchsia, BeOS/Haiku, ESXi, QNX, and of course TempleOS Some of those in the list are using C++, and now adopting Rust as well, actually.
  - ThinkBeat 5y ago
    Yes, most are using a lot of C. Some C++, some other things entirely. I do not advocate "stealing code" but stealing ideas and inspiration :) The perfect OS doesnt not exist today and it never will. But we can do better than we are today.
    - pjmlp 5y ago
      Well ideas there are plenty to steal from, that is true. But maybe we could start by actually catching up with having full stack OSes written in safe systems programming languages. https://youtu.be/z_dt7NG38V4 https://youtu.be/z_dt7NG38V4 https://youtu.be/NqKyHEJe9_w https://youtu.be/NqKyHEJe9_w

161 points by akvadrako 5y ago

encryptluks2 5y ago

I'd honestly just prefer Rust to have a separate patched kernel and not to have it mainlined.

mattashii 5y ago
Could you clarify? It seems to me that a build option is preferable to needing to maintain/checkout a seperate branch altogether
- encryptluks2 5y ago
  Many distros ship hardened and patched kernels. As long as the distro provides packages it wouldn't really be any extra work.
  - remexre 5y ago
    It being a build option doesn't preclude distros shipping multiple packages, some with it enabled and some not?
pjmlp 5y ago
Here you go, https://source.android.com/ https://source.android.com/

staticassertion 5y ago

> Since most of our CVE type problems are usually programming mistakes nowadays, the lack of review could contribute to an increase in programming fault type bugs which aren't forbidden by the safer memory model. That's an odd statement. What CVEs aren't due to programming mistakes? I'm not sure if the majority of CVEs for the Linux kernel come down to memory safety, though I would not be surprised, but certainly a huge number are. > without a clear and obvious benefit beyond promises that can only truly be fully fulfilled with a whole kernel written in Rust. That's not true, really. You don't need a completely safe kernel to have an improvement to safety. If every device driver were memory safe tomorrow we'd be better off. That said, I think this will be an interesting, possibly losing, battle. The Linux Kernel is extremely monolithic, it has a lack of testing and code review, decades of dug-in investments, and a strong history of not prioritizing security or even considering it to be a legitimate goal. Fixing that seems like it will itself take decades, whereas the current approach really feels like it's trying to get it done ASAP. If they can do it, cool. As a Linux user I'll possibly benefit. I'm curious to see how it plays out.

CraigJPerry 5y ago
>> it has a lack of testing and code review Is that really true? I mean my first reaction is that the linked article / thread is a direct counterpoint. All the review and testing the rust facilities are going through. Are these changes not going through substantively the same process as other proposed changes to the kernel?
- medstrom 5y ago
  I don't think it lacks code review, it's famously hard to get half-baked stuff through Torvalds.
  - staticassertion 5y ago
    That isn't really true.
  - wott 5y ago
    The ton of driver code that piles up in the Linux kernel every minute doesn't go through Torvalds. It is delegated. And my reviewing of a few drivers source short commits is enough to tell me that those delegates do not perform a satisfyingly thorough review by any mean. Heck, I saw patches of just a couple dozen lines which exhibited bad copy-pasting errors anyone without prior knowledge could have spotted. You don't need to know what the code does to spot some, you don't either need to know what the driven device does to spot some: purely formal errors with bad macros definitions for example. This kind of stuff wouldn't even pass the first internal review where I worked, which just looked at formal appearance (then there were more in depth reviews, and then there was an external review, but we'd make as sure as possible that our code would be clean before going out). So first you have people (employees of company A) which sends code to a public, external project without having done a proper internal review. Then you have someone else (employee from company B) who claims to have reviewed the commit but hasn't done it properly or at all. And then possibly a third someone who validates this, but doesn't actually check either. It has become a job, a task like another, with the same people who do the same sloppy job as quickly as possible to get rid of it and go home earlier or slack, in the same proportion that you can find in any other position in the world.

ayane_m 5y ago

The FUD around Rusting the kernel reminds me of similar sentiments surrounding autonomous vehicles. Better != perfect; it's an evolutionary step. Even if there are unintended negative consequences of Rust code, if they are less frequent than the rate we deal with bugs today, then it's worth using.

simion314 5y ago
We need to keep in mind that are 2 types of autonomus vehicles discussion. We have a company that uses a ton of hardware, radar, lidar and many cameras and then we have the other ones that want to move fast and break things using only cameras, a GPU and many beta testers. It is normal that the aproach of brute forcing it with ton of data gets a ton of criticism. My other criticism is the bad statistics used. It is like I create the "robot athlete" and I compare it stats with the average of all athletes including the young children and the people with some physical problem. You should compare self driving with cars with exact same safety features and save driver demographics. Bonus if you calculate all deaths caused by illegal driving and then ask the giants WTF not put the money into first solve the speeding and drunk / tired driving , I bet ANN work better on this problem. Rust in Linux seems to me a waste. IMO the Unix philosophy is great but it needs a better implementation , one that is based on the present day hardware and expectations.
- sebzim4500 5y ago
  > Bonus if you calculate all deaths caused by illegal driving and then ask the giants WTF not put the money into first solve the speeding and drunk / tired driving , I bet ANN work better on this problem. Why would they want to do that though? No one would buy a car with those features. I guess you could get a few people to buy one if the insurance was way lower, but you certainly wouldn't get decent market penetration.
  - simion314 5y ago
    So say someone builds a system that you install in cars that monitors how you drive, it can detect if you respect the speeding laws, if you drive normal or you have risky behavior. Few will want this in their car but they will want it on the others car so I can see this possibilities: - a law that requires it in all cars - a law that requires it in new cars - a law that requires it only for new drivers (less then 2 year) and for people caught speeding/drunk. - a law that will make certain roads only available for self driving cars or for people with this safety system. It would be a compromise between AI drivers camp and people that still want to drive themselves. Maybe you will say something about privacy, this system can be implemented with no connectivity to spy on you. It could be read only when you want to pay your insurance or in case of an accident. But also AI can be implemented in a different way, like you could have some quality cameras recording traffic and have the AI detect who is using the phone, who is not paying attention , who is moving in a erratic pattern , so it would be an advanced spending camera. The argument is that the AI driver camp will demand humans to be removed from the road, because their AI driver is better then the average. To prevent you losing the right to drive then this average needs to be improved and this can be done by removing the bad drivers and a good place to start is the ones that do not respect the rules. Otherwise you might object to install a safety system in your car but then the big companies will lobby hard and you will have to use the Tesla/Google or Apple AI powered cars, then not only that he government will know all about your movement but the ad companies will know it too.

robertwt7 5y ago

It is good that rust is slowly being adopted everywhere. Gives me more reason to learn it..

wiz21c 5y ago
Rust helps a lot with memory or concurrency issues. But not the rest. And in my experience, although the concurrency/memory issue are surely the most time consuming bugs, the "rest" is what takes 90% of the time...
- kortex 5y ago
  To each their own. I find Rust a huge step up from C and even C++...especially C++. I've been writing Rust for months and C++ for years. I still feel more comfortable and confident my Rust code has fewer bugs and does what I intend vs C++.
- viraptor 5y ago
  There's more that it helps with. For one example Rust's Enums and richer types allow fewer mistakes -vs- confusion where pointers and integers degrade to a common type. A random line: https://github.com/torvalds/linux/blob/122fa8c588316aacafe7e5a393bb3e875eaf5b25/include/crypto/aead.h#L283 https://github.com/torvalds/linux/blob/122fa8c588316aacafe7e... > void crypto_aead_clear_flags(struct crypto_aead *tfm, u32 flags) These are valid due to poor type usage, but make no sense: crypto_aead_clear_flags(..., PAGE_SIZE) crypto_aead_clear_flags(..., EBUSY) crypto_aead_clear_flags(..., SOME_RANDOM_THING) We can hope they would be caught in review if the code happens to do the right thing by accident, or use static analysis and hope it triggers in that case, or use a language with clearer types where it's much easier to use types preventing this issue.
  - pjmlp 5y ago
    You could the same with C++ or Ada, but naturally it will never be a thing in Linux.
    - 5y ago

greatgib 5y ago

For me, the issues with Rust are: - It is not fun but frustrating to work in Rust, and contrary to C, you are limited by the language/compiler on what you can do. - building/compiling the kernel is not trivial, and you will add a new huge dependency that you have to deal with to build the kernel for whatever target. Let's suppose you want to build for MIPS, then you need to have Rust supporting MIPS. As an exemple, there is a common package in python that decided to start having their module in Rust instead of C. Now a lot of users of the module are pissed off, with good reason, because you can't build/install the module anymore in a little bit older or non conventional distributions.

wiz21c 5y ago
You are not limited. You are strongly driven. It's different. Painfully different, I agree. But as time goes by, it becomes painful.
choeger 5y ago
The latter point is an important one. Rust as a language for libraries cannot work the same way as rust as a language for applications. For the latter it is OK to depend on the cargo toolchain and be opinionated when it comes to things like dynamic linking. For the former you ideally want support in any common compiler (clang, GCC) and as little dependencies and constraints as possible.
- bsdubernerd 5y ago
  It's also a problem for application themselves IMO. Cargo combined with dependency pinning brings most of the disadvantages of similar environments with centralized package handling: the ease of adding package dependencies increases the number of dependencies themselves very rapidly. Overly-narrow version pinning forces per-package lockstep updates of the various dependencies, which in turn means multiple versions of the same package will get pulled two-or-three levels deep. This ensures each single rust package or update you build is almost a guaranteed rebuild-the-world even with a shared cache dir. And we're not touching the problem of building projects where rust is only part of it: annoying if you want to link other stuff into rust, even more so if you want to do the opposite. I'm following a couple of projects that transitioned to rust, and my experience as a contributor is not stellar. A minimal rust project can take hundreds of mb of disk space just in dependencies, and double that for build space. The solution for some has been providing build bots, but again doesn't help me as a contributor, where I need to be able to rebuild from source. This has on me almost the same effect produced by large projects: I only contribute if I have a large vested interest in the package, otherwise I just avoid because it's time consuming.

kortex 5y ago

> In general, I would avoid exposing Rust subsystems to C. It is possible, of course, and it gives you the advantages of Rust in the implementation of the subsystem. However, by having to expose a C API, you would be losing most of the advantages of the richer type system, plus the guarantees that Rust bring as a language for the consumers of the subsystem. This feels like putting the cart before the horse. "we shouldn't integrate Rust into the kernel in a modular sort of way cause it's not as optimal as more thorough type-aware integration." Like, cross that bridge when you come to it, eh? The kernel is currently comprised of hundreds if not thousands of components, talking to each other via C API/ABIs. And talking to hardware is almost entirely ABI. That is how the kernel do. Would we benefit from more type knowledge across boundaries? Absolutely, but this is hard (especially given how different the c and rust approaches to allocation are), and shouldn't stand in the way of progress.

zozbot234 5y ago
Exposing a C API/ABI is also the only supported way of making separate Rust crates that can be truly interoperable beyond the constraints of a single rustc build. It's not a limit to high-level interfaces because one can always build those as higher-level layers within a single crate, and there are tools (bindgen and cbindgen) to make this workflow a bit easier.
brundolf 5y ago
I agree. I've migrated multiple large codebases from a weakly-typed language to a strongly-typed language, and in my experience this is the only way to do it. And it's also not as bad as the OP makes it sound: you get your stronger guarantees gradually, and that's fine. One gotcha, though: a stronger type system encourages the removal of runtime checks. When doing a partial conversion, I've found it's best to leave any existing runtime checks in place, even if the type system makes them "redundant", because it doesn't really until the calling code has also been converted. Once it has, you can go back and remove them.
- brundolf 5y ago
  (Clarification that these codebases were obviously not as large as Linux, and I don't mean to imply that the job was anywhere near the same scale as this one would be, I just think some of the same lessons are broadly applicable)

dash2 5y ago

I think this is gonna be a hard sell. The standard economic analysis of cooperatives is that, being run by their employees, they find it hard to embrace disruptive change (e.g. change which involves firing people). Hence, you probably buy your food from a shareholder-owned business, not an employee-owned cooperative. Linux is a bit like a cooperative in that decisions are made by the software engineers, not by shareholders or managers. In particular, most Linux contributors are probably heavily invested in C. If Rust gets the big boost of being allowed into the kernel, that could make C - and their own skills - be perceived as less valuable, maybe even obsolete eventually. I'll do some economic imperialism here, and claim that Rust's technical merits or otherwise are a second-order issue. Linux developers have an interest in keeping C dominant, so that will probably happen.

jleahy 5y ago
The average Linux kernel developer is probably confident in 5-10 languages, and I can’t imagine they would really have trouble picking up Rust. That kind of argument might hold for the average junior javascript dev, but kernel developers are normally an experienced bunch and C is rarely learnt today as a sole language.
- paulddraper 5y ago
  Be that as it may, thanks to selection bias, the vast majority of kernel devs still probably prefer working in C.
- dash2 5y ago
  Yes, I'm sure they're extremely smart, but equally, they are deeply expert in C, and it is not that easy to become deeply expert in Rust (or any other language). Even smart people can't just pick up new tools in a day.
  - Mic92 5y ago
    They way rust is used in the kernel is a lot more constrained in scope and APIs compared to normal userland rust and hence easier to learn. A lot of added rust API are also just high-level abstraction of already existing concepts/code used in the kernel.

InTheArena 5y ago

Every now and then you see companies make insanely large big bets that radically change a fundamental technology because of a in addressable need. I'm increasingly convinced that a good chunk of MacOS/iOS security problems are due to the simple fact that it is very difficult to write correct system level code w/ C. If (and it is a big if) the performance of rust can be in the same neighborhood as C - and these kinks can be worked out, then as a architect, I think it might be time for a ideological come to Jesus moment/protest reformation of the kernel.

Santosh83 5y ago

If there is industry backing, it will happen. The coming generation of programmers (and hence the future of the kernel) are nowhere as near enthused to program in C as they're Rust, so this transition is inevitable given time. Especially if the kernel should not slowly die out over the decades, stuck permanently to a fossilised language.

phendrenad2 5y ago

I understand what the kernel devs are afraid of. Rust needs to demonstrate beyond a shadow of a doubt that it's a good replacement for C in the kernel, or they risk alienating people who don't want to learn another language, and losing those developers to FreeBSD or something.

frazzyfraz 5y ago

This is just a non-clever attempt by double-vaxxed NPCs to SJW Linux over.

ThinkBeat 5y ago

The kernel has accumulated a lot of technical debt. Linus speaks openly about it. Mixing in the new flavor into a codebase that is 95% (?) using (and abusing) C with warts and all does not strike me as a good idea. Interaction between assembly, C and Rust is not nearly well, enough understood to start. Now if it does go ahead, it will be a great platform to figure out the edge cases, bugs and all those things nobody thought about. I am in favor of Linux.NEXT Write a new kernel in Rust (if it's the best choice) Take all the learnings from Linux and all the research in operating systems that have occurred since and make Linux.NEXT. Steal from Linux, WindowsNT, Quebes, OS/400, Solaris, Minix, z/OS Plan 9, Fuchsia, BeOS/Haiku, ESXi, QNX, and of course TempleOS Creating an operating system suitable for 2020 with the way technology, infrastructure, connectiveness, pervasiveness, privacy conscious, built for the hostile world of the Internet.

pjmlp 5y ago
> Steal from Linux, WindowsNT, Quebes, OS/400, Solaris, Minix, z/OS Plan 9, Fuchsia, BeOS/Haiku, ESXi, QNX, and of course TempleOS Some of those in the list are using C++, and now adopting Rust as well, actually.
- ThinkBeat 5y ago
  Yes, most are using a lot of C. Some C++, some other things entirely. I do not advocate "stealing code" but stealing ideas and inspiration :) The perfect OS doesnt not exist today and it never will. But we can do better than we are today.
  - pjmlp 5y ago
    Well ideas there are plenty to steal from, that is true. But maybe we could start by actually catching up with having full stack OSes written in safe systems programming languages. https://youtu.be/z_dt7NG38V4 https://youtu.be/z_dt7NG38V4 https://youtu.be/NqKyHEJe9_w https://youtu.be/NqKyHEJe9_w