3 ms·

One thing I keep hoping to see in all of these kernels in Rust is an async first kernel. Is there something that makes this particularly difficult or do folks n

One thing I keep hoping to see in all of these kernels in Rust is an async first kernel. Is there something that makes this particularly difficult or do folks not see the value in it? I know from following along with Phil Oppermann’s OS in Rust series that is definitely possible, but these last few OS’ in Rust seem to not be attempting this, https://os.phil-opp.com/async-await/ https://os.phil-opp.com/async-await/

spullara 3y ago
There are so many posts about how async Rust isn't the right abstraction it would surprise me if someone doubled-down on it right now.
pjmlp 3y ago
It would help if async Rust was actually a fully done feature, instead of half way there. As Niko Matsakis puts it, async/await is Rust in hard mode, you don't need that when having to also worry about everything writing an OS from scratch entails.
- bluejekyll 3y ago
  Yes, there might be some significant gaps that make kernel development much more difficult. Off the top of my head, the current state of async traits is still being fleshed out, but it’s coming. The async_trait macros rely on boxing, so I could see that as a big downside in the kernel space. And I know that we don’t yet have an async streaming interface stabilized in the std lib. I’m not sure if these are blocking issues or not, thus my question.
  - surajrmal 3y ago
    async trait macros are no longer needed for the majority of use cases as of 1.75. additionally, I doubt a hobby OS will hit bottlenecks where boxing like this is a problem. Async streaming not being standardized as an interface would only matter if the kernel wanted to use libraries, and I would assume most kernels will be light on external dependencies so it shouldn't matter. If you are targeting a posix syscall interface, a lot of that doesn't really require async to pull off. You need to switch threads whenever an operation is blocked rather than run unrelated work on that thread. I suppose you can still model that with async, but it certainly doesn't feel as helpful.
Yoric 3y ago
Out of curiosity, what would be the benefit?
- bluejekyll 3y ago
  In theory, it would simplify concurrency primitives in the kernel space, especially on the main thread, which is something I’ve notice is not an area that a lot of kernels explore.
ahepp 3y ago
Maybe I'm misunderstanding your proposal, but it seems to undermine process isolation. If you trust processes to yield under a co-operative multitasking system, why not trust them to leave each other's memory alone?
- bluejekyll 3y ago
  I was thinking of the internal Kernel tasks that might block, where it might be possible to have things happen in parallel, like bringing various hardware components online. See the sibling discussion on 200ms startup times.
lasiotus 3y ago
I tried with async in the kernel first, but the cruft that was needed two years ago was not worth it in the kernel itself. The net I/O is actually async-first, see here: https://github.com/moturus/moto-runtime/blob/main/src/net.rs https://github.com/moturus/moto-runtime/blob/main/src/net.rs File I/O will move to this model later (the current file I/O code is quite old and mostly a placeholder).
- bluejekyll 3y ago
  I think I missed this on my quick pass through the code, I see a lot async usage here as well, https://github.com/moturus/moto-runtime/blob/main/src/net_async.rs https://github.com/moturus/moto-runtime/blob/main/src/net_as... Very cool. Thanks for pointing this out.

spullara 3y ago

There are so many posts about how async Rust isn't the right abstraction it would surprise me if someone doubled-down on it right now.

pjmlp 3y ago

It would help if async Rust was actually a fully done feature, instead of half way there. As Niko Matsakis puts it, async/await is Rust in hard mode, you don't need that when having to also worry about everything writing an OS from scratch entails.

bluejekyll 3y ago
Yes, there might be some significant gaps that make kernel development much more difficult. Off the top of my head, the current state of async traits is still being fleshed out, but it’s coming. The async_trait macros rely on boxing, so I could see that as a big downside in the kernel space. And I know that we don’t yet have an async streaming interface stabilized in the std lib. I’m not sure if these are blocking issues or not, thus my question.
- surajrmal 3y ago
  async trait macros are no longer needed for the majority of use cases as of 1.75. additionally, I doubt a hobby OS will hit bottlenecks where boxing like this is a problem. Async streaming not being standardized as an interface would only matter if the kernel wanted to use libraries, and I would assume most kernels will be light on external dependencies so it shouldn't matter. If you are targeting a posix syscall interface, a lot of that doesn't really require async to pull off. You need to switch threads whenever an operation is blocked rather than run unrelated work on that thread. I suppose you can still model that with async, but it certainly doesn't feel as helpful.

Yoric 3y ago

Out of curiosity, what would be the benefit?

bluejekyll 3y ago
In theory, it would simplify concurrency primitives in the kernel space, especially on the main thread, which is something I’ve notice is not an area that a lot of kernels explore.

ahepp 3y ago

Maybe I'm misunderstanding your proposal, but it seems to undermine process isolation. If you trust processes to yield under a co-operative multitasking system, why not trust them to leave each other's memory alone?

bluejekyll 3y ago
I was thinking of the internal Kernel tasks that might block, where it might be possible to have things happen in parallel, like bringing various hardware components online. See the sibling discussion on 200ms startup times.

lasiotus 3y ago

I tried with async in the kernel first, but the cruft that was needed two years ago was not worth it in the kernel itself. The net I/O is actually async-first, see here: https://github.com/moturus/moto-runtime/blob/main/src/net.rs https://github.com/moturus/moto-runtime/blob/main/src/net.rs File I/O will move to this model later (the current file I/O code is quite old and mostly a placeholder).

bluejekyll 3y ago
I think I missed this on my quick pass through the code, I see a lot async usage here as well, https://github.com/moturus/moto-runtime/blob/main/src/net_async.rs https://github.com/moturus/moto-runtime/blob/main/src/net_as... Very cool. Thanks for pointing this out.