7 ms·

Linus on Rust and the Kernel DMA Layer

91 points by tasn 2y ago

piercebot 2y ago
I actually think Linus has had some of his rough edges smoothed out over the years. This was not nearly as combative as it could have been or may have been in the past!
Pet_Ant 2y ago
Good to see Linus step up and publicly support Rust again. I was worried the drama might make he retreat and abandon the experiment.
julienfr112 2y ago
I'm the average rust fanboy, but DMA is something I would rather do in C, as there would probably be more line of unsafe rust code than safe...
- dijit 2y ago
  I think the "drama" is that the DMA guys want to write C, only. However they have opinions about how people are interfacing with their code for DMA from Rust, and the rust developers are maybe bent out of shape about it. Linus is saying: If you're not doing it, then you don't get to have opinions.
  - an_guy 2y ago
    > Linus is saying: If you're not doing it, then you don't get to have opinions. But eventually he would have to, if those rust interfaces becomes part of kernel. The entire drama is about DMA guys not wanting to deal with this in future.
- tuetuopay 2y ago
  Even unsafe rust is safer than C. Error management, type safety, modern language, etc. There are many reasons to use Rust in place of C, even if your whole code is one giant unsafe blob. I run eBPF code written in Rust in production, and oh boy how more readable it is compared to C. Even simple stuff like `if let Some`, actual sum types, generics, are enough to warrant using Rust over C. And its endtrypoint is basically unsafe { ... }.
  - arminiusreturns 2y ago
    eBPF is such a supertool that is slept on. I have been working on turning my nftables into it, (on low-latency/high-throughput targets) may I ask how you are using it with Rust?
    - tuetuopay 2y ago
      Sure! I use the aya framework (https://aya-rs.dev https://aya-rs.dev) that provides the kernel-side bindings to write the probes in Rust, and the userspace tooling to load it in the kernel, interacts with maps, etc. Quite a joy to work with, and has all the niceties you'd expect from using Rust. We write XDP apps for custom dataplanes where traditionally DPDK would be used (routers and such). Our upcoming network acls are written this way, so close to your netfilter usage.
      - arminiusreturns 2y ago
        Very cool and fun. I do worry about running so much stuff in kernel space though. Imagine a widespread 0day that hits the kernel, or kernel panics causing kernel crashes that require reboots, the user space priv escalation, etc... Are you doing fail-open or fail-closed? I've been on the fence on that.
      - 112233 2y ago
        What is the current legal status of eBPF? It used to be that loading any non-trivial eBPF into kernel involved GPL-only stuff, and both the eBPF program and the program that used it had to be GPL.
        tuetuopay 2y ago
        Well, yes, but this only matters if you distribute the compiled probes: you'll have to distribute the source for the probes. And, thankfully, this is not AGPL where network interaction counts, so: - a cloud provider writing a dataplane using ebpf does not have to provide the source - a company selling on-prem software (e.g. esxi) that ships with ebpf probes would need to distribute their source (note: IANAL)
- db48x 2y ago
  I disagree. The Rust code has an abstraction for DMA’d objects that gives them a safe interface.
- wtallis 2y ago
  They're trying to do DMA in C, by calling the existing DMA code that's written in C from new drivers written in Rust. Their attempt to make an interface from Rust to the existing DMA code is what's being rejected by the guy maintaining the existing DMA code.
skerit 2y ago
So if I understand correctly, Linus & co say no maintainer is forced to write or accept Rust code in their little piece of the kernel. But maintainers claim that other people simply interfacing with their code through Rust bindings would require the maintainers to change that Rust code anyway if any changes they make would break those bindings? Am I getting that right?
- braiamp 2y ago
  No, Linus is saying that to have a say, you have to stake something. You can't veto something and at the same time not engaging with it in any other constructive way. You either ignore it totally, or you don't and become part of the process. In this specific case, you do not get to decide who the client of your code/subsystem is, and the only way you have a say about it is by offering actual feedback about it.
  - IshKebab 2y ago
    I don't think that's true though, given the Linux development model where all internal APIs are private and unstable and can be changed at any time. It means there is a two-way dependency between API providers and users - because it isn't a public API you are allowed to change it, but you have to fix all the places it is used too. If those places now include Rust then it would require you to learn Rust (or depend on other people). Really they just need to make a decision at this point. Either Rust is allowed and maintainers are going to have to learn it, or there's no point and the Rust people are going to have to fork Linux. I guess there is a third option: a stable driver ABI. But they've avoided that at all costs for years so I don't see it happening. And besides, Rust is going to be used for more than just drivers eventually.
    - db48x 2y ago
      > Really they just need to make a decision at this point. They already _have_ made this decision. Why do people keep misunderstanding this. While ordinarily if you change the API you are required to change all of the callers of that API to match, in this one special case the maintainer of a module can declare that they don’t want to touch the Rust code and that if they change the API then they are not going to change the Rust code that calls it. In that case the Rust for Linux team will step in and make whatever changes to the Rust code are needed.
softwaredoug 2y ago
It’s amazing to me how much reality show level entertainment we get out of Linux kernel discussions. Most of us don’t have a dog in the fight, yet it’s still engaging content to watch someone else’s drama and “workplace politics” play out .
- sheepscreek 2y ago
  This is how shit gets done in the real world (pretty much anywhere) when many people are involved. So yeah, it is reality in that regard.
- knowitnone 2y ago
  a good number of us are users of the Linux kernel so we should have a dog in this fight. So this DMA guy gets mad and start introducing junk into the code and nobody catches it, who get hurt? I hope he just steps away; it'll be better for his sanity.
  - Vilian 2y ago
    Linus said that he like to work with guy and respect him technically, the DMA is stable because of him, just because one day the guy step over his fence he should be removed?, how many maintainers did something similar in 20 years?, honestly this drama is overblown by people who never saw drama in the kernel and think that rust is the first and only big change that got maintainers reluctant to merge, RT took 20 years, Rust has a lot more devs, companies and investment behind
    - dcrazy 2y ago
      The drama came from Hector getting fed up with being stymied by Christoph overstepping his authority, trying to go around the system by starting a social media flame war, and eventually quitting.
kvemkon 2y ago
Discussion started on neighbor thread: https://news.ycombinator.com/item?id=43123391 https://news.ycombinator.com/item?id=43123391
ChrisArchitect 2y ago
[dupe] https://news.ycombinator.com/item?id=43123104 https://news.ycombinator.com/item?id=43123104
sc68cal 2y ago
It's a shame that it took Hector Martin completely burning out and publicly melting down for Linus to step up and finally Do Something about this situation. This should have been resolved much earlier by Linus, so that less R4L contributors didn't have to publicly quit before we got to this point.
- Vilian 2y ago
  From his blog seems to me that the entitled users where one of the biggest problems, if wasn't them complaining and always demanding more he wouldn't feel pressured to upstream everything immediately
dcrazy 2y ago
This is quite a reasonable policy and I wish it had been clearer to everyone earlier.
hyperman1 2y ago
I wonder why the delay in Linus' reaction. Could it be that he hoped a hands off approuch would let the people involved find a compromise themselves? Or is it just normal that he is unavailable for a few days.
- vetrom 2y ago
  In any position of real leadership, unless you have exigent circumstances that dictate otherwise, you want to lead, not dictate. In that sense, there's real value in letting a situation develop its own consensus as the actors involved communicate and complete discovery of the factors and motivators involved in any situation, especially if it's clear that the actors have yet to explore that space.
- ammar2 2y ago
  He does open with: > I was hopeful, and I've tried to just see if this long thread results in anything constructive, but this seems to be going backwards (or at least not forwards). So probably just hoping some people on either side of the issue would see the light. But he also probably shouldn't have chimed in earlier then.

91 points by tasn 2y ago

I actually think Linus has had some of his rough edges smoothed out over the years. This was not nearly as combative as it could have been or may have been in the past!

Pet_Ant 2y ago

Good to see Linus step up and publicly support Rust again. I was worried the drama might make he retreat and abandon the experiment.

julienfr112 2y ago

I'm the average rust fanboy, but DMA is something I would rather do in C, as there would probably be more line of unsafe rust code than safe...

dijit 2y ago
I think the "drama" is that the DMA guys want to write C, only. However they have opinions about how people are interfacing with their code for DMA from Rust, and the rust developers are maybe bent out of shape about it. Linus is saying: If you're not doing it, then you don't get to have opinions.
- an_guy 2y ago
  > Linus is saying: If you're not doing it, then you don't get to have opinions. But eventually he would have to, if those rust interfaces becomes part of kernel. The entire drama is about DMA guys not wanting to deal with this in future.
tuetuopay 2y ago
Even unsafe rust is safer than C. Error management, type safety, modern language, etc. There are many reasons to use Rust in place of C, even if your whole code is one giant unsafe blob. I run eBPF code written in Rust in production, and oh boy how more readable it is compared to C. Even simple stuff like `if let Some`, actual sum types, generics, are enough to warrant using Rust over C. And its endtrypoint is basically unsafe { ... }.
- arminiusreturns 2y ago
  eBPF is such a supertool that is slept on. I have been working on turning my nftables into it, (on low-latency/high-throughput targets) may I ask how you are using it with Rust?
  - tuetuopay 2y ago
    Sure! I use the aya framework (https://aya-rs.dev https://aya-rs.dev) that provides the kernel-side bindings to write the probes in Rust, and the userspace tooling to load it in the kernel, interacts with maps, etc. Quite a joy to work with, and has all the niceties you'd expect from using Rust. We write XDP apps for custom dataplanes where traditionally DPDK would be used (routers and such). Our upcoming network acls are written this way, so close to your netfilter usage.
    - arminiusreturns 2y ago
      Very cool and fun. I do worry about running so much stuff in kernel space though. Imagine a widespread 0day that hits the kernel, or kernel panics causing kernel crashes that require reboots, the user space priv escalation, etc... Are you doing fail-open or fail-closed? I've been on the fence on that.
    - 112233 2y ago
      What is the current legal status of eBPF? It used to be that loading any non-trivial eBPF into kernel involved GPL-only stuff, and both the eBPF program and the program that used it had to be GPL.
      - tuetuopay 2y ago
        Well, yes, but this only matters if you distribute the compiled probes: you'll have to distribute the source for the probes. And, thankfully, this is not AGPL where network interaction counts, so: - a cloud provider writing a dataplane using ebpf does not have to provide the source - a company selling on-prem software (e.g. esxi) that ships with ebpf probes would need to distribute their source (note: IANAL)
db48x 2y ago
I disagree. The Rust code has an abstraction for DMA’d objects that gives them a safe interface.
wtallis 2y ago
They're trying to do DMA in C, by calling the existing DMA code that's written in C from new drivers written in Rust. Their attempt to make an interface from Rust to the existing DMA code is what's being rejected by the guy maintaining the existing DMA code.

skerit 2y ago

So if I understand correctly, Linus & co say no maintainer is forced to write or accept Rust code in their little piece of the kernel. But maintainers claim that other people simply interfacing with their code through Rust bindings would require the maintainers to change that Rust code anyway if any changes they make would break those bindings? Am I getting that right?

braiamp 2y ago
No, Linus is saying that to have a say, you have to stake something. You can't veto something and at the same time not engaging with it in any other constructive way. You either ignore it totally, or you don't and become part of the process. In this specific case, you do not get to decide who the client of your code/subsystem is, and the only way you have a say about it is by offering actual feedback about it.
- IshKebab 2y ago
  I don't think that's true though, given the Linux development model where all internal APIs are private and unstable and can be changed at any time. It means there is a two-way dependency between API providers and users - because it isn't a public API you are allowed to change it, but you have to fix all the places it is used too. If those places now include Rust then it would require you to learn Rust (or depend on other people). Really they just need to make a decision at this point. Either Rust is allowed and maintainers are going to have to learn it, or there's no point and the Rust people are going to have to fork Linux. I guess there is a third option: a stable driver ABI. But they've avoided that at all costs for years so I don't see it happening. And besides, Rust is going to be used for more than just drivers eventually.
  - db48x 2y ago
    > Really they just need to make a decision at this point. They already _have_ made this decision. Why do people keep misunderstanding this. While ordinarily if you change the API you are required to change all of the callers of that API to match, in this one special case the maintainer of a module can declare that they don’t want to touch the Rust code and that if they change the API then they are not going to change the Rust code that calls it. In that case the Rust for Linux team will step in and make whatever changes to the Rust code are needed.

softwaredoug 2y ago

It’s amazing to me how much reality show level entertainment we get out of Linux kernel discussions. Most of us don’t have a dog in the fight, yet it’s still engaging content to watch someone else’s drama and “workplace politics” play out .

sheepscreek 2y ago
This is how shit gets done in the real world (pretty much anywhere) when many people are involved. So yeah, it is reality in that regard.
knowitnone 2y ago
a good number of us are users of the Linux kernel so we should have a dog in this fight. So this DMA guy gets mad and start introducing junk into the code and nobody catches it, who get hurt? I hope he just steps away; it'll be better for his sanity.
- Vilian 2y ago
  Linus said that he like to work with guy and respect him technically, the DMA is stable because of him, just because one day the guy step over his fence he should be removed?, how many maintainers did something similar in 20 years?, honestly this drama is overblown by people who never saw drama in the kernel and think that rust is the first and only big change that got maintainers reluctant to merge, RT took 20 years, Rust has a lot more devs, companies and investment behind
  - dcrazy 2y ago
    The drama came from Hector getting fed up with being stymied by Christoph overstepping his authority, trying to go around the system by starting a social media flame war, and eventually quitting.

kvemkon 2y ago

Discussion started on neighbor thread: https://news.ycombinator.com/item?id=43123391 https://news.ycombinator.com/item?id=43123391

ChrisArchitect 2y ago

[dupe] https://news.ycombinator.com/item?id=43123104 https://news.ycombinator.com/item?id=43123104

sc68cal 2y ago

It's a shame that it took Hector Martin completely burning out and publicly melting down for Linus to step up and finally Do Something about this situation. This should have been resolved much earlier by Linus, so that less R4L contributors didn't have to publicly quit before we got to this point.

Vilian 2y ago
From his blog seems to me that the entitled users where one of the biggest problems, if wasn't them complaining and always demanding more he wouldn't feel pressured to upstream everything immediately

dcrazy 2y ago

This is quite a reasonable policy and I wish it had been clearer to everyone earlier.

hyperman1 2y ago

I wonder why the delay in Linus' reaction. Could it be that he hoped a hands off approuch would let the people involved find a compromise themselves? Or is it just normal that he is unavailable for a few days.

vetrom 2y ago
In any position of real leadership, unless you have exigent circumstances that dictate otherwise, you want to lead, not dictate. In that sense, there's real value in letting a situation develop its own consensus as the actors involved communicate and complete discovery of the factors and motivators involved in any situation, especially if it's clear that the actors have yet to explore that space.
ammar2 2y ago
He does open with: > I was hopeful, and I've tried to just see if this long thread results in anything constructive, but this seems to be going backwards (or at least not forwards). So probably just hoping some people on either side of the issue would see the light. But he also probably shouldn't have chimed in earlier then.