10 ms·
Async Rust is not safe with io_uring
- n_plus_1_acc 2y agoIt's not about memory safety, as you might assume from the title. There's no soundness bug involved.
- ethegwo 2y agoRust not only provides memory safety, it also promises at least I/O safety in the future: https://rust-lang.github.io/rfcs/3128-io-safety.html https://rust-lang.github.io/rfcs/3128-io-safety.html
- the8472 2y agoLeaking file descriptors is safe, at least as far as the IO-safety RFC defines it. It's just a regular bug.
- fallingsquirrel 2y agoResource leaks have nothing to do with safety. That's true both for memory safety and i/o safety. See for yourself with `mem::forget(File::open(...)?)`
- ekidd 2y agoYeah, "safe Rust" is officially allowed to leak memory and other resources. - The easiest way to do this is mem::forget, a safe function which exists to leak memory deliberately. - The most common real way to leak memory is to create a loop using the Rc<T> or Arc<T> reference count types. I've never seen this in any of my company's Rust code, but we don't write our own cyclic data structures, either. This is either a big deal to you, or a complete non-issue, depending on your program architecture. Basically, "safe Rust" aims to protect you from memory corruption, undefined behavior, and data races between threads. It does not protect you from leaks, other kinds of race conditions, or (obviously) logic bugs.
- chubot 2y agoYeah the article is giving what looks like a bad definition: According to: https://rust-lang.github.io/rfcs/3128-io-safety.html https://rust-lang.github.io/rfcs/3128-io-safety.html Rust’s standard library almost provides I/O safety, a guarantee that if one part of a program holds a raw handle privately, other parts cannot access it. According to the article: I/O Safety: Ensuring that accepted TCP streams are properly closed without leaking connections. These are not the same definition. As I've mentioned several times [1], in Rust, the word "safety" is being abused to the point of causing confusion, and it looks like this is another instance of that. [1] https://news.ycombinator.com/item?id=31726723 https://news.ycombinator.com/item?id=31726723
- ethegwo 2y agoThank you! I will update the post and fix it.
- morning-coffee 2y agoWhere do you conclude "Rust promises I/O safety in the future"? An RFC is not a promise to do anything... it may represent a desire... a possibility, but you taking that leap and implying a promise is a flagrant misrepresentation. Now let's take "the future" part... you seem to be impugning Async Rust for something it's not even purported to do in the present. What's the point of this? You found a bug in monoio it seems... I don't see the argument you've presented as supporting the premise that "Async Rust is not safe".
- remram 2y agoThat doesn't mean the Rust project promise no one will write a library that is not safe... The title of this submission is aimed specifically at "Async Rust", ie the language. The reality is that one third-party library with 4k stars on GitHub and 0.2.4 version number has a non-memory-unsafe leak. I'd say the title is a full-on lie.
- not_a_dane 2y ago[flagged]
- eqvinox 2y agoNo it won't.
- simion314 2y agoIf Rust would be as good as advertised we would have at least 5 Rust kernels 10x faster then Linux or BSD and 100% safe and all servers in the world would run those kernels. Is there at least one kernel with a big tech behind it or with a big army or Rust evangelists giving 5$ a month to finance it?
- deleted 2y ago[deleted]
- Ygg2 2y agoThat'd be a Sisyphean job. Writing kernel in Rust from scratch, I understand, but this? Please, let this be a joke.
- lifthrasiir 2y agoThis aspect of io_uring does affect a lot of surface APIs, as I have experienced at work. At least for me I didn't have to worry much about borrowing though.
- eqvinox 2y agoHmm. Does it? Python's futures have an explicit .cancel() operation. And the C io_uring usage I'm looking at knows to cancel events too… It's really that Rust might've made a poor choice here, as the article points out: > Async Rust makes a few core assumptions about futures: > 1. The state of futures only change when they are polled. > 2. Futures are implicitly cancellable by simply never polling them again. But at the same time, maybe this is just that Rust's Futures need to be used different here, in conjunction with a separate mechanism to manage the I/O operation that knows things need to be cancelled?
- lifthrasiir 2y agoThat part of the article is kinda irrelevant in my opinion. Futures do require polling to move forward, but polling can be forced by an external signal (otherwise the whole future model wouldn't work!). Therefore io_uring can be safely implemented by having central worker threads which then signal outstanding futures; that was how I ended up doing at work as well. So the article actually seems to ask whether such out-of-band mechanism can be entirely prevented or not.
- eqvinox 2y agoThe sibling comment to yours points out cancelling Futures is dropping Futures, what's your experience / do you think that would work to prevent needing the out-of-band mechanism?
- lifthrasiir 2y agoThat's backwards... Rust's way to do cancellation is simply to drop the future (i.e. let it deallocated). There is one big caveat here though, namely the lack of async drop as others pointed out. In the current Rust io_uring-like stuffs can be safely implemented with an additional layer of abstraction. Some io_uring operations can be ongoing when it looks fine to borrow the buffer, sure. Your API just has to ensure that it is not possible to borrow until all operations are finished then! Maybe it can error, or you can require something like `buf.borrow().await`. This explicit borrowing is not an alien concept in Rust (cf. RefCell etc.) and probably the best design at the moment, but it does need dynamic bookkeeping which some may want to eliminate.
- api 2y agoThere are async libraries like glommio, which I’m using for a new project, that avoid this I think, but they require you to factor things a little differently from tokio. Maybe cancellation itself is problematic. There’s a reason it was dropped from threading APIs and AFAIK there is no way to externally cancel a goroutine. Goroutines are like async tasks with all the details hidden from you as it’s a higher level language.
- eqvinox 2y agoI don't think it's possible to get away with fundamentally no cancellation support, there are enough edge cases that need it even if most applications don't have such edge cases. FWIW, this was also painful to do across threads in our C event loop, but there was no way around the fact that we needed it (cf. https://github.com/FRRouting/frr/blob/56d994aecab08b9462f2c80b64b9d1c3bbf097e8/lib/event.c#L1531 https://github.com/FRRouting/frr/blob/56d994aecab08b9462f2c8... )
- badmintonbaseba 2y agoI don't think that cancellation is inherently problematic, but it needs to be cooperative. One-sided cancellation of threads (and probably goroutines) can never work. Cooperative cancellation can be implemented in languages that mark their suspension points explicitly in their coroutines, like Rust, Python and C++. I think Python's asyncio models cancellation fairly well with asyncio.CancelledError being raised from the suspension points, although you need to have some discipline to use async context managers or try/finally, and to wait for cancelled tasks at appropriate places. But you can write your coroutines with the expectation that they eventually make forward progress or exit normally (via return or exception). It looks like Rust's cancellation model is far more blunt, if you are just allowed to drop the coroutine.
- ekidd 2y ago> It looks like Rust's cancellation model is far more blunt, if you are just allowed to drop the coroutine. You can only drop it if you own it (and nobody has borrowed it), which means you can only drop it at an `await` point. This effectively means you need to use RAII guard objects like in C++ in async code if you want to guarantee cleanup of external resources. But it's otherwise completely well behaved with epoll-based systems. I find that a bigger issue in my async Rust code is using Tokio-style async "streams", where a cancelled sender looks exactly like a clean "end of stream". In this case, I use something like: enum StreamValue<T> { Value(T), End, } If I don't see StreamValue::End before the stream closes, then I assume the sender failed somehow and treat it as a broken stream (sort of like a Unix EPIPE error). This can obviously be wrapped. But any wrapper still requires the sender to explictly close the stream when done, and not via an implicit Drop.
- deleted 2y ago[deleted]
- smatija 2y agoThis reference at the bottom of article was very interesting to me: https://without.boats/blog/io-uring/ https://without.boats/blog/io-uring/ "So I think this is the solution we should all adopt and move forward with: io-uring controls the buffers, the fastest interfaces on io-uring are the buffered interfaces, the unbuffered interfaces make an extra copy. We can stop being mired in trying to force the language to do something impossible. But there are still many many interesting questions ahead."
- xyst 2y agoNotably, io_uring syscall has been a significant source of vulnerabilities. Last year, Google security team decided to disable it in their products (ChromeOS, Android, GKE) and production servers [1]. Containerd maintainers soon followed Google recommendations and updated seccomp profile to disallow io_uring calls [2]. io_uring was called out specifically for exposing increased attack surface by kernel security team as well long before G report was released [3]. Seems like less of a rust issue and more of a bug(s) in io_uring? I suppose user space apps can provide bandaid fix but ultimately needs to be handled at kernel. [1] https://security.googleblog.com/2023/06/learnings-from-kctf-vrps-42-linux.html?m=1 https://security.googleblog.com/2023/06/learnings-from-kctf-... [2] https://github.com/containerd/containerd/pull/9320 https://github.com/containerd/containerd/pull/9320 [3] https://lwn.net/Articles/902466/ https://lwn.net/Articles/902466/
- the8472 2y agoThose are separate issues. The blog post is about using it properly in userspace, google's concerns are about kernel security bugs.
- pferde 2y agoThat is all well and true, and the vulnerabilities are getting fixed, but that is off-topic to the posted article. The article is more about the Rust io_uring async implementation breaking assumption that Rust's async makes, in that a Future can only get modified when it's poll()-ed. I'm guessing that assumption came from an expectation that all async runtimes live in userland, and this newfangled kernel-backed runtime does things on its own inside the kernel, thus breaking the original assumption.
- pornel 2y agoThe problem has been known from the beginning, because async I/O on Windows has the same issues as io_uring. Rust went with poll-based API and synchronous cancellation design anyway, because that fits ownership and borrowing. Making async cancellation work safely even in presence of memory leaks (destructors don't always run) and panics remains an unsolved design problem.
- MuffinFlavored 2y agoHow common of a pattern is it to accept in a loop but also on a timeout so that you can pre-empt and go do some other work?
- majewsky 2y agoThe timeout is only a stand-in for the generic need to be able to cancel an acceptance loop. You could just as well want to cancel accept() when SIGINT/SIGTERM/etc is received, or when recreating the server socket, e.g. in response to a configuration change. Most server processes have a need like this.
- sylware 2y agodude, machine code generated with gcc/clang is not safe in the first place. This is only the tip of the iceberg.
- whytevuhuni 2y agoI don't get it. What's the ideal scenario here? That going to the sleep branch of the select should cancel the accept? Will cancelling the accept terminate any already-accepted connections? Shouldn't it be delayed instead? Shouldn't newly accepted connections be dropped only if the listener is dropped, rather than when the listener.accept() future is dropped? If listener.accept() is dropped, the queue should be with the listener object, and thus the event should still be available in that queue on the next listener.accept(). This seems more like a bug with the runtime than anything.
- ethegwo 2y agothe ideal scenario is like cancellable io provide by monoio, I write an example of this in the blog: https://github.com/ethe/io-uring-is-not-cancellation-safe/blob/cancelable-io/src/main.rs#L47 https://github.com/ethe/io-uring-is-not-cancellation-safe/bl... . However, it has lots of limitation, and do not have a perfect way to do this at the moment.
- whytevuhuni 2y agoThat's an implementation detail. What's the user-facing behavior? What should happen with mistakenly-accepted connections? Even the blog admits that cancellation of any kind, is racing with the kernel which might complete the accept request anyway. Even if you call `.cancel()`, the queue might have an accepted connection FD in it. Even if it doesn't, it might do by the time the kernel acknowledges the cancellation. So you now have a mistakenly-accepted connection. What do you do with it? Drop it? That seems like the wrong answer, whoever writes a loop like the one in the blog will definitely not expect some of the connections mysteriously being dropped.
- whytevuhuni 2y agoOkay, looks like withoutboats gave the answer to this in another thread [1], and that seems like the right answer. The accept() future being dropped must not result in any cancellation of any kind, unless the listener itself is also dropped. This is an implementation issue with monoio that just needs more polishing. And given how hard io_uring is to get right, monoio should be given that time before being chosen to be used in production for anything. [1] https://news.ycombinator.com/item?id=41994308 https://news.ycombinator.com/item?id=41994308
- withoutboats3 2y agoThis is nothing to do with async Rust; monoio (and possibly other io-uring libraries) are just exposing a flawed API. My ringbahn library written in 2019 correctly handled this case by having a dropped accept future register a cancellation callback to be executed when the accept completes. https://github.com/ringbahn/ringbahn https://github.com/ringbahn/ringbahn
- ethegwo 2y agoThe rest of this blog discusses how to continue processing operations after cancellation fails, which is blocked by the Rust abstraction. Yes, not everyone (probably very few) defines this as a safety issue, I wrote about this at the end of the blog.
- withoutboats3 2y agoI don't consider Yosh Wuyts's concept of "halt safety" coherent, meaningful or worth engaging with. It's true that linear types would enable the encoding of additional liveness guarantees that Rust's type system as it exists cannot encode, but this doesn't have anything to do with broken io-uring libraries leaking resources.
- ethegwo 2y agoContinuing process after cancellation failure is a challenge I face in my actual work, and I agree that "halt-safety" lacks definition and context. I have also learned and been inspired a lot from your blogs, I appreciate it.
- lowbloodsugar 2y agoAgree. When I hear “I wish Rust was Haskell” I assume the speaker is engaged in fantasy, not in engineering. The kernel is written in C and seems to be able to manage just fine. Problem is not Rust. Problem is wishing Rust was Haskell.
- pjdesno 2y agoDoesn't this close the incoming connection, rather than allowing another pending accept to receive it?
- newpavlov 2y agoYet another example of async Rust being a source of unexpected ways to shoot yourself in the foot... Async advocates can argue as long as they want about "you're holding it wrong", but to me it sounds like people arguing that you can safely use C/C++ just by being "careful".
- speed_spread 2y agoAsync has its uses, but there should also be a way to ensure that a Rust stack does not use async at all, like there is for unsafe. Most codebases could do without the added complexity. There will be better ways to do concurrency in the future (hehe)
- dboreham 2y agoAgree. If people want to delude themselves that async is useful, that's fine. But don't inflict it on the rest of us by viral propagation throughout the dependency ecosystem.
- vacuity 2y agoI claim that async/await is far more basic (/fundamental/simple, not necessarily easy) than most acknowledge, but it should indeed be more composable with sync code. It is a means of interacting with asynchronous phenomena, which underlie the OS-hardware connection. The composability is necessary because no one is going to write a massive state machine for their entire program.
- int_19h 2y agoPart the problem is ABI. Fibers are a more sensible approach to async, but as soon as you introduce them, FFI becomes a massive pain (see e.g. Go) in any environment which doesn't have fibers as a first-class primitive that everybody supports. Promise-based async has the advantage that you can desugar it down to C-compatible ABI.
- 2y ago
- Sytten 2y agoWe so need a way to express cancellation safety other than documentation. This is not just an io_grind problem, you have a lot of futures in tokio that are not cancel safe. Are there some RFC of the subject?
- pjdesno 2y agoSince io_uring has similar semantics to just about every hardware device ever (e.g. NVMe submission and completion queues), are there any implications of this for Rust in the kernel? Or in SPDK and other user-level I/O frameworks? Note that I don't know a lot about Rust, and I'm not familiar with the rules for Rust in the kernel, so it's possible that it's either not a problem or the problematic usages violate the kernel coding rules. (although in the latter case it doesn't help with non-kernel frameworks like SPDK)
- gspr 2y agoI think async Rust is far from entering the kernel. Edit: I realize my comment might come off as a bit snarky or uninformative to someone who isn't familiar with Rust. That was not the intention. "Async Rust" is particular framework for abstracting over various non-blocking IO operations (and more). It allows terse code to be written using a few convenient keywords, that causes a certain state machine (consisting of ordinary Rust code adhering to certain rules) to be generated, which in turn can be coupled with an "async runtime" (of which there are many) to perform the IO actions described by the code. The rules that govern the code generated by these convenient keywords, i.e. the code that the async runtimes execute, are apparently not a great fit for io_uring and the like. However, I don't think anyone is proposing writing such code inside the kernel, nor that any of the async runtimes actually make sense in a kernel setting. The issues in this article don't exist when there is no async Rust code. Asynchronous operations can, of course, still be performed, but one has to manage that without the convenient scaffolding afforded by async Rust and the runtimes.
- amoss 2y agoWho is Barbara?
- jerf 2y agoThere are certain counterintuitive things that you have to learn if you want to be a "systems engineer", in a general sense, and this whole async thing has been one of the clearest lessons to me over the years of how seemingly identical things sometimes can not be abstracted over. Here by "async" I don't so much mean async/await versus threads, but these kernel-level event interfaces regardless of which abstraction a programming language lays on top of them. At the 30,000 foot view, all the async abstractions are basically the same, right? You just tell the kernel "I want to know about these things, wake me up when they happen." Surely the exact way in which they happen is not something so fundamental that you couldn't wrap an abstraction around all of them, right? And to some extent you can, but the result is generally so lowest-common-denominator as to appeal to nobody. Instead, every major change in how we handle async has essentially obsoleted the entire programming stack based on the previous ones. Changing from select to epoll was not just a matter of switching out the fundamental primitive, it tended to cascade up almost the entire stack. Huge swathes of code had to be rewritten to accommodate it, not just the core where you could do a bit of work and "just" swap out epoll for select. Now we're doing it again with io_uring. You can't "just" swap out your epoll for io_uring and go zoomier. It cascades quite a ways up the stack. It turns out the guarantees that these async handlers provide are very different and very difficult to abstract. I've seen people discuss how to bring io_uring to Go and the answer seems to basically be "it breaks so much that it is questionable if it is practically possible". An ongoing discussion on an Erlang forum seems to imply it's not easy there (https://erlangforums.com/t/erlang-io-uring-support/765); https://erlangforums.com/t/erlang-io-uring-support/765); I'd bet it reaches up "less far" into the stack but it's still a huge change to BEAM, not "just" swapping out the way async events come in. I'm sure many other similar discussions are happening everywhere with regards to how to bring io_uring into existing code, both runtimes and user-level code. This does not mean the problem is unsolvable by any means. This is not a complaint, or a pronunciation of doom, or an exhortation to panic, or anything like that. We did indeed collectively switch from select to epoll. We will collectively switch to io_uring eventually. Rust will certainly be made to work with it. I am less certain about the ability of shared libraries to be efficiently and easily written that work in both environments, though; if you lowest-common-denominator enough to work in both you're probably taking on the very disadvantages of epoll in the first place. But programmers are clever and have a lot of motivation here. I'm sure interesting solutions will emerge. I'm just highlighting that as you grow in your programming skill and your software architecture abilities and general system engineering, this provides a very interesting window into how abstractions can not just leak a little, but leak a lot, a long ways up the stack, much farther than your intuition may suggest. Even as I am typing this, my own intuition is still telling me "Oh, how hard can this really be?" And the answer my eyes and my experience give my intuition is, "Very! Even if I can't tell you every last reason why in exhaustive detail, the evidence is clear!" If it were "just" a matter of switching, as easy as it feels like it ought to be, we'd all already be switched. But we're not, because it isn't.
- ordu 2y ago> The title of this blog might sound a bit dramatic, but everyone has different definitions and understandings of "safety." Still in Rust community "safety" is used in a very specific understanding, so I don't think it is correct to use any definition you like while speaking about Rust. Or at least, the article should start with your specific definition of safety/unsafety. I don't want to reject the premise of the article, that this kind of safety is very important, but for Rust unsafety without using "unsafe" is much more important that an OS dying from leaked connections. I have read through the article looking for rust's kind of unsafety and I was found that I was tricked. It is very frustrating, it looks to me as a lie with some lame excuses afterwards.
- airstrike 2y agoIt's definitely clickbaity. The author knew exactly what they were doing. Time to flag, hide and move on.
- ethegwo 2y agoI apologize for not using a good title, but I think the issues I listed (especially my argument that we should break issues down to bugs in runtime and the limitations of Rust abstractions) are worth discussing, even if there are many people who argue otherwise
- zbentley 2y agoAgreed, though a better title would probably not use the term "safe" unqualified (e.g. "Async rust with io_uring leaks resources (and risks corrupting program state").
- hitekker 2y agoDon't be hard on yourself. I recall times when Rustaceans eagerly marketed "memory safety" to mean "safety" in general. Sometimes even extending "safety" to mean "bug free". I chalk up the confusion to true believers playing the motte-and-bailey game. When the marketing meaning feels good, they use it all over the place. Then, when the substance doesn't live up to the hype, the fans retreat to the technical definition. This is not a new phenomena: https://news.ycombinator.com/item?id=41994254 https://news.ycombinator.com/item?id=41994254
- duped 2y agoWhile it's true that the "state" of a future is only mutated in the poll() implementation, it's up to the author of the future implementation to clone/send/call the Waker provided in the context argument to signal to the executor that poll() should be called again by the executor, which I believe is how one should handle this case.
- lsofzz 2y ago<3
- wg0 2y agoI have tried to learn rust and borrow checker is no problem but I can't get lifetimes and then Rc, Box, Arc Pinning along with async Rust are a whole another story. Having programmed in raw C, I know Rust is more like Typescript if you once try it after writing Javascript, you can't go back for anything serious in plain Javascript. You would want to have some guard rails better than having no guard rails.
- riskable 2y agoTry embedded rust: Get an RP2040 board and fool around with that. It'll make a lot more sense to you if the parts you don't understand are encapsulation types like RC, Box, and Arc because those aren't really used in embedded rust! Lifetimes are still used but not nearly as much.
- wg0 2y agoI guess that would be the book: https://docs.rust-embedded.org/book/ https://docs.rust-embedded.org/book/
- NooneAtAll3 2y ago> // we loose the chance to handle the previous one. lose?
- ethegwo 2y agoThanks, I will fix it.
- airstrike 2y ago[flagged]
- withoutboats3 2y ago[flagged]
- airstrike 2y ago[flagged]
- withoutboats3 2y agoMy sincere apologies if I read this uncharitably, I interpreted your comment about sarcasm to itself be sarcastic. I will not be publishing any such post. To clarify what you seem not to understand: the code in the blog post depends on a library called monoio, and based on the post's description of the behavior of that code (I haven't actually read the source for monoio), monoio contains a bug. I don't dispute that this code behaves incorrectly, just that it has anything to do with Rust's facility for async IO rather than with the specific IO library employed in the post.
- throw-away_42 2y ago[flagged]
- dang 2y agoI know the GP comment was provocative, but please don't respond to a bad comment by breaking the site guidelines yourself. That only makes things worse. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html Edit: this has been a problem in other places too: https://news.ycombinator.com/item?id=41657430 https://news.ycombinator.com/item?id=41657430 https://news.ycombinator.com/item?id=41034778 https://news.ycombinator.com/item?id=41034778 Could you please not post like this to Hacker News? It's not what this site is for, and destroys what it is for. (https://news.ycombinator.com/item?id=41996506 https://news.ycombinator.com/item?id=41996506 was much better, of course.) If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
- zbentley 2y agoMy hot take is that the root of this issue is that the destructor side of RAII in general is a bad idea. That is, registering custom code in destructors and running them invisibly, implicitly, maybe sometimes but only if you're polite, is not and never was a good pattern. This pattern causes issues all over the place: in C++ with headaches around destruction failure and exceptions; in C++ with confusing semantics re: destruction of incompletely-initialized things; in Rust with "async drop"; in Rust (and all equivalent APIs) in situations like the one in this article, wherein failure to remember to clean up resources on IO multiplexer cancellation causes trouble; in Java and other GC-ful languages where custom destructors create confusion and bugs around when (if ever) and in the presence of what future program state destruction code actually runs. Ironically, two of my least favorite programming languages are examples of ways to mitigate this issue: Golang and JavaScript runtimes: Golang provides "defer", which, when promoted widely enough as an idiom, makes destructor semantics explicit and provides simple and consistent error semantics. "defer" doesn't actually solve the problem of leaks/partial state being left around, but gives people an obvious option to solve it themselves by hand. JavaScript runtimes go to a similar extreme: no custom destructors, and a stdlib/runtime so restrictive and thick (vis-a-vis IO primitives like sockets and weird in-memory states) that it's hard for users to even get into sticky situations related to auto-destruction. Zig also does a decent job here, but only with memory allocations/allocators (which are ironically one of the few resource types that can be handled automatically in most cases). I feel like Rust could have been the definitive solution to RAII-destruction-related issues, but chose instead to double down on the C++ approach to everyone's detriment. Specifically, because Rust has so much compile-time metadata attached to values in the program (mutability-or-not, unsafety-or-not, movability/copyabiliy/etc.), I often imagine a path-not-taken in which automatic destruction (and custom automatic destructor code) was only allowed for types and destructors that provably interacted only with in-user-memory state. Things referencing other state could be detected at compile time and required to deal with that state in explicit, non-automatic destructor code (think Python context-managers or drop handles requiring an explicit ".execute()" call). I don't think that world would honestly be too different from the one we live in. The rust runtime wouldn't have to get much thicker--we'd have to tag data returned from syscalls that don't imply the existence of cleanup-required state (e.g. select(2), and allocator calls--since we could still automatically run destructors that only interact with cleanup-safe user-memory-only values), and untagged data (whether from e.g. fopen(2) or an unsafe/opaque FFI call or asm! block) would require explicit manual destruction. This wouldn't solve all problems. Memory leaks would still be possible. Automatic memory-only destructors would still risk lockups due to e.g. pagefaults/CoW dirtying or infinite loops, and could still crash. But it would "head off at the pass" tons of issues--not just the one in the article: Side-effectful functions would become much more explicit (and not as easily concealable with if-error-panic-internally); library authors would be encouraged to separate out external-state-containing structs from user-memory-state-containing ones; destructor errors would become synonymous with specific programmer errors related to in-memory twiddling (e.g. out of bounds accesses) rather than failures to account for every possible state of an external resource, and as a result automatic destructor errors unconditionally aborting the program would become less contentious; the surface area for challenges like "async drop" would be massively reduced or sidestepped entirely by removing the need for asynchronous destructors; destructor-related crash information would be easier to obtain even in non-unwinding environments... Maybe I'm wrong and this would require way too much manual work on the part of users coding to APIs requiring explicit destructor calls. But heck, I can dream, can't I?
- whalesalad 2y agoI am really confused that rust was not designed to do async out of the box? Am I wrong that third party libraries are required (tokio) to do this?
- LudwigNagasena 2y agoRust provides async syntax out of the box, but you have to Bring Your Own Runtime.
- ciconia 2y agoDisclaimer: I'm not a Rust programmer. Having written a few libs for working with io_uring (in Ruby), cancellation is indeed tricky, with regards to keeping track of buffers. This is an area where working with fibers (i.e. stackful coroutines) is beneficial. If you keep metadata (and even buffers) for the ongoing I/O op on the stack, there's much less book-keeping involved. Managing the I/O op lifetime, especially cleaning up, becomes much simpler, as long as you make sure to not return before receiving a CQE, even after having issued a cancellation SQE.