4 ms·

Make sense might be an overstatement but ok. Then why do functions with sync syscalls (ie file, timers or mutex ops) not expose the same contractual differences

by klabb3 3y ago

Make sense might be an overstatement but ok. Then why do functions with sync syscalls (ie file, timers or mutex ops) not expose the same contractual differences? They’re just regular functions in most languages including Rust. Perhaps anything involving syscalls should be exposed and contractual. I doubt it, but maybe it’s important for some obscure ownership-of-resources reason. But then why the inconsistency between traditional and pooled syscalls? The only difference is whether the runtime sits in the kernel or in user space. The only one who should care is the runtime folks. My take has been for years that this is throwing complexity over the fence and shaming users for not getting it. And even when they do get it, they Arc<Mutex> everything anyways in which case you are throwing the baby out with the bathwater (RAII, single ownership, static borrowing).

naasking 3y ago
> Then why do functions with sync syscalls (ie file, timers or mutex ops) not expose the same contractual differences? They’re just regular functions in most languages including Rust. Because the kernel doesn't expose that contract, so they don't have that behaviour. > The only difference is whether the runtime sits in the kernel or in user space. In other words, what contracts you have control over and are allowed to provide. > My take has been for years that this is throwing complexity over the fence and shaming users for not getting it. I'm sure how we got here would seem baffling if you're going to just ignore the history of the C10K problem that led us to this point. You can of course paper over any platform-specific quirks and provide a uniform interface if you like, at the cost of some runtime overhead, but eliminating as much of this kind of implicit runtime overhead as possible seems like one of Rust's goals. Other languages, like Go, have a different set of goals and so can provide that uniform interface. It's probably also possible to have some of that uniform interface via a crate, if some were so inclined, but that doesn't mean it should be in the core which has a broader goal.
- klabb3 3y ago
  > I'm sure how we got here would seem baffling if you're going to just ignore the history of the C10K problem that led us to this point. I am not unaware of pooled syscalls. I worked on the internals of an async Rust runtime, although that should not matter for critiquing language features. The archeological dig into why things are the way they can come up with a perfectly reasonable story, yet at the same time lead to a suboptimal state for a given goal - which is where the opinion space lies - the space where I’m expressing my own. > but eliminating as much of this kind of implicit runtime overhead as possible seems like one of Rust's goals Yes, certainly. And this is where the perplexity manifests from my pov. Async is a higher level feature, with important contractual ecosystem-wide implications. My thesis is that async in rust is not a good solution to the higher level problems, because it interacts poorly with other core features of Rust, and because it modularizes poorly. Once you take the event loop(s) and lift it up into a runtime, the entire point (afaik - I don’t see any other?) to abstract away tedious lower level event and buffer maintenance. If you just want performance and total control, it’s already right there with the much simpler event loop primitives. In short, I fail to see how arguments for async can stand on performance merits alone. Some people disagree about the ergonomics issues, which I am always happy to argue in good faith.
- gpderetta 3y ago
  >>[mutexes] are just regular functions in most languages including Rust. >Because the kernel doesn't expose that contract, so they don't have that behaviour Which OS are we talking about? Linux doesn't really have mutices as primitives. You can build async mutexes on top of eventfd and soon even on top of futexes with io_uring.
  - steveklabnik 3y ago
    I’m on mobile so it’s tough to look up the details, but IIRC at least Windows? Implementations of the standard library mutexes are here https://github.com/rust-lang/rust/tree/master/library/std/src/sys/locks/mutex https://github.com/rust-lang/rust/tree/master/library/std/sr... And of course it’s pthreads on Linux.
    - gpderetta 3y ago
      Windows has a lot of mutex and signaling primitives. Some of them can be asynchronously waited on. It also has an IoRing and I wouldn't be surprised if keyed events (the local futex equivalent) will be supported in the future. As an aside, it is interesting that rust uses pthread_mutex for its standard library mutex. GCC/libstdc++ regrets that decision as its std::mutex is now way larger than it needs to be but it is now permanently baked in the ABI. I guess rust still doesn't guarantee ABI stability so the decision could be reversed in the future.

by klabb3 3y ago

naasking 3y ago

> Then why do functions with sync syscalls (ie file, timers or mutex ops) not expose the same contractual differences? They’re just regular functions in most languages including Rust. Because the kernel doesn't expose that contract, so they don't have that behaviour. > The only difference is whether the runtime sits in the kernel or in user space. In other words, what contracts you have control over and are allowed to provide. > My take has been for years that this is throwing complexity over the fence and shaming users for not getting it. I'm sure how we got here would seem baffling if you're going to just ignore the history of the C10K problem that led us to this point. You can of course paper over any platform-specific quirks and provide a uniform interface if you like, at the cost of some runtime overhead, but eliminating as much of this kind of implicit runtime overhead as possible seems like one of Rust's goals. Other languages, like Go, have a different set of goals and so can provide that uniform interface. It's probably also possible to have some of that uniform interface via a crate, if some were so inclined, but that doesn't mean it should be in the core which has a broader goal.

klabb3 3y ago
> I'm sure how we got here would seem baffling if you're going to just ignore the history of the C10K problem that led us to this point. I am not unaware of pooled syscalls. I worked on the internals of an async Rust runtime, although that should not matter for critiquing language features. The archeological dig into why things are the way they can come up with a perfectly reasonable story, yet at the same time lead to a suboptimal state for a given goal - which is where the opinion space lies - the space where I’m expressing my own. > but eliminating as much of this kind of implicit runtime overhead as possible seems like one of Rust's goals Yes, certainly. And this is where the perplexity manifests from my pov. Async is a higher level feature, with important contractual ecosystem-wide implications. My thesis is that async in rust is not a good solution to the higher level problems, because it interacts poorly with other core features of Rust, and because it modularizes poorly. Once you take the event loop(s) and lift it up into a runtime, the entire point (afaik - I don’t see any other?) to abstract away tedious lower level event and buffer maintenance. If you just want performance and total control, it’s already right there with the much simpler event loop primitives. In short, I fail to see how arguments for async can stand on performance merits alone. Some people disagree about the ergonomics issues, which I am always happy to argue in good faith.
gpderetta 3y ago
>>[mutexes] are just regular functions in most languages including Rust. >Because the kernel doesn't expose that contract, so they don't have that behaviour Which OS are we talking about? Linux doesn't really have mutices as primitives. You can build async mutexes on top of eventfd and soon even on top of futexes with io_uring.
- steveklabnik 3y ago
  I’m on mobile so it’s tough to look up the details, but IIRC at least Windows? Implementations of the standard library mutexes are here https://github.com/rust-lang/rust/tree/master/library/std/src/sys/locks/mutex https://github.com/rust-lang/rust/tree/master/library/std/sr... And of course it’s pthreads on Linux.
  - gpderetta 3y ago
    Windows has a lot of mutex and signaling primitives. Some of them can be asynchronously waited on. It also has an IoRing and I wouldn't be surprised if keyed events (the local futex equivalent) will be supported in the future. As an aside, it is interesting that rust uses pthread_mutex for its standard library mutex. GCC/libstdc++ regrets that decision as its std::mutex is now way larger than it needs to be but it is now permanently baked in the ABI. I guess rust still doesn't guarantee ABI stability so the decision could be reversed in the future.