15 ms·

Monoio – A thread-per-core Rust async runtime with io_uring

158 points by losfair 5y ago

rektide 5y ago
Have any other Rust async runtimes use io_uring/gotten at all good yet? Best of the best modern systems programmers gotta get good sometime. Not sure if it's happening yet. Ok here's one point of call: https://github.com/tokio-rs/tokio-uring https://github.com/tokio-rs/tokio-uring
- otabdeveloper4 5y ago
  "Async runtimes" is a thing that only makes sense for interpreted languages and their limitations. This thing is pure cargo cult and shows that Rust mostly appeals to newbies who don't know what they are doing.
  - jeltz 5y ago
    Care to explain why they only make sense for interpreted languages and what limitations you are talking about? Async in Rust is mostly just a convenient syntax around ideas which has been around in the C and C++ worlds for decades with libraries like libevent.
  - jandrewrogers 5y ago
    Async models are idiomatic for high-performance server code regardless of the programming language, particularly for anything I/O or concurrency intensive. The reason people use thread-per-core software architectures, which naturally require an async model, is because they have much higher throughput than the alternatives. If software performance, efficiency, and scalability are primary objectives, you are going to be writing a lot of async code in a systems language like Rust or C++. People that “know what they are doing” understand this and why. Hence the interest in async libraries for Rust.
    - MuffinFlavored 5y ago
      can you take code that is blocking in nature and make it async (like an FFI dlsym call), or at a low level does that just boil down to lots of polling/timers?
      - vlovich123 5y ago
        Not sure what code you’re looking at but in reality all code is asynchronous by nature (ie you ask the HW to do something and it tells you it’s done some time later). Then we layer blocking syscalls on top and then you layer async underneath. Io_uring is an attempt at getting everything asynchronous top to bottom. Some times we use polling if there’s sufficient traffic and interrupts are inefficient but that’s still asynchronous, just batched.
      - weq 5y ago
        http://think-async.com/Asio/asio-1.5.3/doc/asio/overview/core/async.html http://think-async.com/Asio/asio-1.5.3/doc/asio/overview/cor...
    - errantmind 5y ago
      The GP mentioned "async runtimes". There are other approaches to async that don't involve using an async runtime, like epoll / kqueue. I personally prefer writing synchronous code, running in multiple threads pulling from a shared work queue. It isn't a one-size-fits-all solution but it is widely applicable, and you get to avoid the complexity of writing 'async code'
      - alexchamberlain 5y ago
        Async runtimes are just wrappers around the underlying async technology, like epoll, kqueue and io_uring. They give structure in a similar way if/for/while put structure around the underlying PC jumps.
        errantmind 5y ago
        Yes, but they also introduce a lot of complexity to the developer experience. It is turtles all the way down with async code, as they say
  - staticassertion 5y ago
    Yes, io_uring, a linux subsystem, is clearly just a cargo cult feature for people who don't know what they're doing.
    - errantmind 5y ago
      Last I saw there was no substantial evidence of io_uring delivering a significant improvement over epoll, but I haven't looked in the last year. I wouldn't assume it is good just because it is in Linux (this is coming from someone who uses Linux exclusively)
      - staticassertion 5y ago
        I don't assume it's good because of Linux, I have a generally negative view of Linux. It's just stupid to attribute io_uring to rust or call it cargo culting for Rust to use io_uring. The two projects are unrelated. As for performance, I wouldn't judge based on a year ago, obviously a lot has changed since then - you can find numbers if you'd like, I saw the maintainer posting benchmarks only a few weeks ago.
        errantmind 5y ago
        Yea, I'll have to check it out again. I see very mixed results, it is likely io_uring may not fully replace epoll for all situations as some people thought it would. That's fine though https://github.com/axboe/liburing/issues/189 https://github.com/axboe/liburing/issues/189
      - threeseed 5y ago
        Up to 2.44x faster for network zerocopy send: https://lore.kernel.org/io-uring/cover.1638282789.git.asml.silence@gmail.com/ https://lore.kernel.org/io-uring/cover.1638282789.git.asml.s... Up to 1.6x faster than epoll: https://twitter.com/axboe/status/1362271500489793539 https://twitter.com/axboe/status/1362271500489793539 io_uring 11% faster for non-polled IO: https://twitter.com/axboe/status/1465358880502861829/photo/1 https://twitter.com/axboe/status/1465358880502861829/photo/1 And it's not like io_uring is complete. With every kernel release the gap over epoll gets bigger.
  - CyberRabbi 5y ago
    “Async” / event loop systems will usually be more efficient because 1) they don’t have to store unnecessary processor state to memory between handling events and 2) they are associated with cooperative event handlers and the assumption of a cooperative system provides more opportunities for optimization.
    - jlokier 5y ago
      Ironically, it is not true that "async" (stackless) event loop systems don't store unnecessary processor state to memory. Those systems store the entire processing state in future object memory. It is the same state that threaded systems store. Stackful context switching systems (which includes some kinds of efficient threads - I would count Linux kernel internal threads among these) store that same state in the stack and context objects when context switching, so in principle store about the same amount of state. But some async, stackless systems do a bunch of extra work unwinding and restoring entire call stacks whenever an async task pauses in await, and therefore take more CPU time to context switch than stackful systems do.
      - CyberRabbi 5y ago
        > It is the same state that threaded systems store. Nope. Synchronous task switching systems additionally have to store CPU register state. In the cooperative case you need to store the caller-saved registers, in the pre-emptive case you need to store all the registers. Async systems simply don’t have to do that extra work. > But some async, stackless systems do a bunch of extra work unwinding and restoring entire call stacks whenever an async task pauses in await Just the really bad ones. In any competent system a callback is stored ready to handle the event. No “unwinding” necessary.
        gpderetta 5y ago
        A cooperative synchronous task switching (i.e. fiber based) need only save the exact same information as an async based one (i.e. stackless coroutines): at a minimum a context pointer and an instruction pointer. Plus any live registers (which might be none). You only need to save caller saved registers fs your context switching routine uses the conventional ABI, but that's not a requirement.
  - threeseed 5y ago
    As I posted above I integrated Hyper (web server) with Glommio (io_uring based async runtime). Based on my limited benchmarks against Vertx/Scala and Nginx it was significantly faster, had zero failed transactions and used a fraction of the memory/CPU. That means I need less servers and have a better end user experience.
- threeseed 5y ago
  Glommio uses io_uring: https://github.com/DataDog/glommio https://github.com/DataDog/glommio And I integrated Hyper as an example: https://github.com/DataDog/glommio/blob/master/examples/hyper.rs https://github.com/DataDog/glommio/blob/master/examples/hype... And the performance was blisteringly quick (6x better latency streaming from a file compared to Nginx).
- mitchtbaum 5y ago
  > Have any other Rust async runtimes use io_uring/gotten at all good yet? yes, check out `actix-rt` https://github.com/actix/actix-net https://github.com/actix/actix-net
  - ibraheemdev 5y ago
    actix-rt is a wrapper around tokio's single threaded runtime, and (optionally) tokio-uring.
ezekiel68 5y ago
I truly appreciate that this team uses nightly rust and only runs only Linux currently (due to relying on io_uring). Truly in the spirit of systems programming - focusing on a single, tight, efficient implementation first and leaving other considerations such as cross-platform compatibility for later. Lets those of us who love to live on the bleeding edge have our nice things too! :)
- MuffinFlavored 5y ago
  > for later or for never :P why not prove like... "this is the best. all other platforms/circumstances/situations are subpar" if the performance differences are enough, i can picture people making excuses to avoid all of those other platforms (or just never using this... more likely)
  - staticassertion 5y ago
    Agreed. I wouldn't care if Rust dropped Windows/ OSX support entirely. Totally niche platforms.
    - aneutron 5y ago
      I think you're being sarcastic, but I'm not totally sure.
      - staticassertion 5y ago
        I'm honestly not. Desktop platforms like Windows are incredibly niche. How much rust software actually runs on those? Some, of course. But probably an incredible minority. Supporting them is smart because people expect it, and not supporting it would be bad optics, but if a language legitimately only targeted Linux, and was better for it, I'd be fine with that - they target the most popular OS by far.
        jsty 5y ago
        > How much rust software actually runs on those? Some of rust's most well-known and widely distributed success stories for a start - Firefox & Dropbox
jgilias 5y ago
I wonder how much and for what Rust is being used at Bytedance, given that this seems to be under their GitHub org. It's pretty interesting that they are apparently using it. Edit: For those wondering who Bytedance are, I can save you a Google search. These are the people making TikTok.
- rfoo 5y ago
  Bytedance uses Rust extensively in Feishu/Lark (which competes with Microsoft Teams). It is even listed at https://www.rust-lang.org/production/users https://www.rust-lang.org/production/users From what I read (in Chinese), this "Monoio" is meant to be used for the proxy/whatever part of their next-generation service mesh. So maybe a corp-wide thing.
- mleonhard 5y ago
  It looks like they shared a lot of internal projects: https://github.com/orgs/bytedance/repositories https://github.com/orgs/bytedance/repositories I'm pleasantly surprised. Have many other Chinese tech companies embraced FOSS?
  - justincormack 5y ago
    Yes its an increasing trend. Much of it is way less well known outside China. But lots of code from Alibaba, Tencent and smaller companies etc. many new CNCF projects are from China.
Zababa 5y ago
There are four articles in Chinese about the design and implementation on one of the author's blog. Here's a link to the first one: https://www.ihcblog.com/rust-runtime-design-1/ https://www.ihcblog.com/rust-runtime-design-1/. I don't know the subject and Chinese enough to know how much is lost by automatic translation (Google in my case), but it looks relatively good. As an aside, Google translation integrated in Chrome breaks the formatting of the code blocks, which is surprising since they're in a <pre> block.
moggers 5y ago
I've found in my own use of rust I want async/nonblocking for two things. 1. Be able to timeout a read of a socket 2. Be able to select over multiple sockets, and read whichever is ready first Usually a combination of both. epoll/io_uring(I guess? I only ever did research on epoll) seem like the solution being handed to me on a silver platter, however my understanding is if you want to use either of those you're meant to use async in rust, and that while there are some libraries which provide interfaces for this kind of behavior outside of async, they're usually very ad-hoc and that the community is just very laser focused on async as a language construct. What I don't understand is why does Rust consider it necessary to introduce async, futures, runtimes, an executor, async versions of TcpListeners, Files, etc for this? Why can't I just have a function in the standard library that takes a slice of std::net::TcpListeners, a timeout, blocks, and then gives me whichever is ready to read, when its ready to read, or nothing if the timeout is reached? Its not like I was going to do anything else on that thread while I wait for a packet to be received, it can happily be parked. Instead I have to select a runtime, and libraries compatible with the runtime, replace all the TcpListeners from the standard library I'm using with tokio TcpListeners or whatever, deal with API change, and now I have to deal with the "turtles all the way down" problem of async as well. That's not even to get into the whole nightmare that a lot of really sick libraries which I want to use in a blocking nature, are now only providing async APIs, which means its "async or the highway". I am very much not happy about this situation and I don't know what I can do about it. It seems the only response I ever get is "just use it, its easy" but that is not at all convincing me. I don't want to use it!
- mleonhard 5y ago
  If you need to handle <100,000 sockets, you will probably be fine with a thread per socket. Call set_read_timeout to implement the deadline. Run load tests and adjust the socket limit. Async lets one process handle millions of sockets. If you want to handle millions of sockets with threads, you can use `mio` [0]. Mio's API has footguns that can cause UB. If your wire protocol is complicated, you may find yourself implementing something like async, but worse. [0] https://crates.io/crates/mio https://crates.io/crates/mio
- GolDDranks 5y ago
zinclozenge 5y ago
There's a ton of bleeding edge and novel work being done by Chinese companies and communities due to their insane scale requirements. I've been trying to learn the language to gain more insight and stay current with what they're doing.
jkelleyrtp 5y ago
It's exciting to see another thread-per-core async runtime for Rust. It's truly understated how difficult the Send + Sync requirements in Tokio are for writing regular code. It's typically rare for async tasks in Tokio to be used across two threads simultaneously, but now all of your data must be Send+Sync. Plus, Tokio is unique in that it's one of the very few runtimes in existence that's work-stealing (meaning the task can move off its original thread). Most other runtimes in other languages do not have that requirement, meaning you can use traditional Cell/RefCell/Rc instead of their slower, atomic variants. Right now, the best you can do is write a thread pool that spawns a Tokio LocalSet to run !Send futures. In fact, this is what Actin-web does to achieve its crazy performance. Web requests finish so quickly you rarely need work-stealing to achieve good performance, and often the cost of atomics/stealing is greater than the performance gain.
- verdagon 5y ago
  I've been thinking about how one could get around the Send + Sync requirements, it's a fascinating conundrum, from a language design standpoint. If we didn't have the Send + Sync requirements, and multiple async functions are running concurrently on the same thread, and multiple of them locked the same RefCell, might that cause a panic?
  - masklinn 5y ago
    > If we didn't have the Send + Sync requirements, and multiple async functions are running concurrently on the same thread, and multiple of them locked the same RefCell, might that cause a panic? Hopefully.
- asymptotic 5y ago
  > the best you can do is write a thread pool that spawns a Tokio LocalSet to run !Send futures Can you go into this a bit more or provide a code reference? I'm new to writing async Rust code and am interested in this idea.
  - jkelleyrtp 5y ago
_vvhw 5y ago
With context switches becoming more and more expensive relative to faster and faster I/O devices, almost the same order of magnitude, I believe that thread-per-core is where things are heading, because the alternative of not doing thread-per-core might literally be halving your throughput. That's also the most exciting thing about io_uring for me: how it enables a simple, single-threaded and yet highly performant thread-per-core control plane, outsourcing to the kernel thread pool for the async I/O data plane, instead of outsourcing to a user space thread pool as in the past. It's much more efficient and at the same time, much easier to reason about. There's no longer the need for multithreading to leak into the control plane. My experience with io_uring has been mostly working on TigerBeetleDB [1], a new distributed database that can process a million financial transactions a second, and I find it's a whole new way of thinking... that you can now just submit I/O directly from the control plane without blocking and without the cost of a context switch. It really changes the kinds of designs you can achieve, especially in the storage space (e.g. things like LSM-tree compactions can become much more parallel and incremental, while also becoming much simpler, i.e. no longer any need to think of memory barriers). Fantastic also to now have a unified API for networking/storage. So much good stuff in io_uring. Exciting times. [1] https://www.tigerbeetle.com https://www.tigerbeetle.com
- sitkack 5y ago
  Totally. And these thread per core apps can talk directly to virtio or NVMe passed into the kvm guest, you can get the best of having a unix host but have applications that run directly under KVM w/o sacrificing a rich control plane. And control plane reliability doesn't impact the data plane. Wonderful times!
billconan 5y ago
I have a stupid question, why isn't an async runtime a language feature of rust? we don't seem to see so many async runtimes in other languages? They seem to have a default way to run async tasks?
- FerdSlav 5y ago
  an async runtime inherently requires knowledge of the underlying system which Rust remains agnostic to. Rust can be written to target bare-metal (where no OS exists) to WASM and everything in between. If a runtime was added to the language itself you would inherently limit the flexibility of the platforms by which it could target.
- pixel_fcker 5y ago
  It sort of goes against Rust’s philosophy to bake anything like that into the language. Where they screwed up was not providing the machinery to make libraries agnostic of the runtime an end user wants to use in their program, so libraries either depend on a specific runtime explicitly or use features to allow users to switch runtimes at compile time. This causes a lot of headaches for library maintainers and end users both. There’s a lot of interest in adding said machinery (through collections of traits in std) to enable libraries to be generic over different runtimes, but a solution is still some ways off.
- masklinn 5y ago
  > I have a stupid question, why isn't an async runtime a language feature of rust? Because it was considered inimical to the core values and purpose of the language, which is to be a systems language. An async runtime being a language feature means a runtime is a language feature, and that is very much undesirable (in fact it used to be part of the language and was removed as it "settled" into its niche from its original design, which was much higher level and more applicative).
renox 5y ago
Does someone knows the pro/cons of io_uring vs DPDK in these these one thread per core setup?

158 points by losfair 5y ago

Have any other Rust async runtimes use io_uring/gotten at all good yet? Best of the best modern systems programmers gotta get good sometime. Not sure if it's happening yet. Ok here's one point of call: https://github.com/tokio-rs/tokio-uring https://github.com/tokio-rs/tokio-uring

otabdeveloper4 5y ago
"Async runtimes" is a thing that only makes sense for interpreted languages and their limitations. This thing is pure cargo cult and shows that Rust mostly appeals to newbies who don't know what they are doing.
- jeltz 5y ago
  Care to explain why they only make sense for interpreted languages and what limitations you are talking about? Async in Rust is mostly just a convenient syntax around ideas which has been around in the C and C++ worlds for decades with libraries like libevent.
- jandrewrogers 5y ago
  Async models are idiomatic for high-performance server code regardless of the programming language, particularly for anything I/O or concurrency intensive. The reason people use thread-per-core software architectures, which naturally require an async model, is because they have much higher throughput than the alternatives. If software performance, efficiency, and scalability are primary objectives, you are going to be writing a lot of async code in a systems language like Rust or C++. People that “know what they are doing” understand this and why. Hence the interest in async libraries for Rust.
  - MuffinFlavored 5y ago
    can you take code that is blocking in nature and make it async (like an FFI dlsym call), or at a low level does that just boil down to lots of polling/timers?
    - vlovich123 5y ago
      Not sure what code you’re looking at but in reality all code is asynchronous by nature (ie you ask the HW to do something and it tells you it’s done some time later). Then we layer blocking syscalls on top and then you layer async underneath. Io_uring is an attempt at getting everything asynchronous top to bottom. Some times we use polling if there’s sufficient traffic and interrupts are inefficient but that’s still asynchronous, just batched.
    - weq 5y ago
      http://think-async.com/Asio/asio-1.5.3/doc/asio/overview/core/async.html http://think-async.com/Asio/asio-1.5.3/doc/asio/overview/cor...
  - errantmind 5y ago
    The GP mentioned "async runtimes". There are other approaches to async that don't involve using an async runtime, like epoll / kqueue. I personally prefer writing synchronous code, running in multiple threads pulling from a shared work queue. It isn't a one-size-fits-all solution but it is widely applicable, and you get to avoid the complexity of writing 'async code'
    - alexchamberlain 5y ago
      Async runtimes are just wrappers around the underlying async technology, like epoll, kqueue and io_uring. They give structure in a similar way if/for/while put structure around the underlying PC jumps.
      - errantmind 5y ago
        Yes, but they also introduce a lot of complexity to the developer experience. It is turtles all the way down with async code, as they say
- staticassertion 5y ago
  Yes, io_uring, a linux subsystem, is clearly just a cargo cult feature for people who don't know what they're doing.
  - errantmind 5y ago
    Last I saw there was no substantial evidence of io_uring delivering a significant improvement over epoll, but I haven't looked in the last year. I wouldn't assume it is good just because it is in Linux (this is coming from someone who uses Linux exclusively)
    - staticassertion 5y ago
      I don't assume it's good because of Linux, I have a generally negative view of Linux. It's just stupid to attribute io_uring to rust or call it cargo culting for Rust to use io_uring. The two projects are unrelated. As for performance, I wouldn't judge based on a year ago, obviously a lot has changed since then - you can find numbers if you'd like, I saw the maintainer posting benchmarks only a few weeks ago.
      - errantmind 5y ago
        Yea, I'll have to check it out again. I see very mixed results, it is likely io_uring may not fully replace epoll for all situations as some people thought it would. That's fine though https://github.com/axboe/liburing/issues/189 https://github.com/axboe/liburing/issues/189
    - threeseed 5y ago
      Up to 2.44x faster for network zerocopy send: https://lore.kernel.org/io-uring/cover.1638282789.git.asml.silence@gmail.com/ https://lore.kernel.org/io-uring/cover.1638282789.git.asml.s... Up to 1.6x faster than epoll: https://twitter.com/axboe/status/1362271500489793539 https://twitter.com/axboe/status/1362271500489793539 io_uring 11% faster for non-polled IO: https://twitter.com/axboe/status/1465358880502861829/photo/1 https://twitter.com/axboe/status/1465358880502861829/photo/1 And it's not like io_uring is complete. With every kernel release the gap over epoll gets bigger.
- CyberRabbi 5y ago
  “Async” / event loop systems will usually be more efficient because 1) they don’t have to store unnecessary processor state to memory between handling events and 2) they are associated with cooperative event handlers and the assumption of a cooperative system provides more opportunities for optimization.
  - jlokier 5y ago
    Ironically, it is not true that "async" (stackless) event loop systems don't store unnecessary processor state to memory. Those systems store the entire processing state in future object memory. It is the same state that threaded systems store. Stackful context switching systems (which includes some kinds of efficient threads - I would count Linux kernel internal threads among these) store that same state in the stack and context objects when context switching, so in principle store about the same amount of state. But some async, stackless systems do a bunch of extra work unwinding and restoring entire call stacks whenever an async task pauses in await, and therefore take more CPU time to context switch than stackful systems do.
    - CyberRabbi 5y ago
      > It is the same state that threaded systems store. Nope. Synchronous task switching systems additionally have to store CPU register state. In the cooperative case you need to store the caller-saved registers, in the pre-emptive case you need to store all the registers. Async systems simply don’t have to do that extra work. > But some async, stackless systems do a bunch of extra work unwinding and restoring entire call stacks whenever an async task pauses in await Just the really bad ones. In any competent system a callback is stored ready to handle the event. No “unwinding” necessary.
      - gpderetta 5y ago
        A cooperative synchronous task switching (i.e. fiber based) need only save the exact same information as an async based one (i.e. stackless coroutines): at a minimum a context pointer and an instruction pointer. Plus any live registers (which might be none). You only need to save caller saved registers fs your context switching routine uses the conventional ABI, but that's not a requirement.
- threeseed 5y ago
  As I posted above I integrated Hyper (web server) with Glommio (io_uring based async runtime). Based on my limited benchmarks against Vertx/Scala and Nginx it was significantly faster, had zero failed transactions and used a fraction of the memory/CPU. That means I need less servers and have a better end user experience.
threeseed 5y ago
Glommio uses io_uring: https://github.com/DataDog/glommio https://github.com/DataDog/glommio And I integrated Hyper as an example: https://github.com/DataDog/glommio/blob/master/examples/hyper.rs https://github.com/DataDog/glommio/blob/master/examples/hype... And the performance was blisteringly quick (6x better latency streaming from a file compared to Nginx).
mitchtbaum 5y ago
> Have any other Rust async runtimes use io_uring/gotten at all good yet? yes, check out `actix-rt` https://github.com/actix/actix-net https://github.com/actix/actix-net
- ibraheemdev 5y ago
  actix-rt is a wrapper around tokio's single threaded runtime, and (optionally) tokio-uring.

ezekiel68 5y ago

I truly appreciate that this team uses nightly rust and only runs only Linux currently (due to relying on io_uring). Truly in the spirit of systems programming - focusing on a single, tight, efficient implementation first and leaving other considerations such as cross-platform compatibility for later. Lets those of us who love to live on the bleeding edge have our nice things too! :)

MuffinFlavored 5y ago
> for later or for never :P why not prove like... "this is the best. all other platforms/circumstances/situations are subpar" if the performance differences are enough, i can picture people making excuses to avoid all of those other platforms (or just never using this... more likely)
- staticassertion 5y ago
  Agreed. I wouldn't care if Rust dropped Windows/ OSX support entirely. Totally niche platforms.
  - aneutron 5y ago
    I think you're being sarcastic, but I'm not totally sure.
    - staticassertion 5y ago
      I'm honestly not. Desktop platforms like Windows are incredibly niche. How much rust software actually runs on those? Some, of course. But probably an incredible minority. Supporting them is smart because people expect it, and not supporting it would be bad optics, but if a language legitimately only targeted Linux, and was better for it, I'd be fine with that - they target the most popular OS by far.
      - jsty 5y ago
        > How much rust software actually runs on those? Some of rust's most well-known and widely distributed success stories for a start - Firefox & Dropbox

jgilias 5y ago

I wonder how much and for what Rust is being used at Bytedance, given that this seems to be under their GitHub org. It's pretty interesting that they are apparently using it. Edit: For those wondering who Bytedance are, I can save you a Google search. These are the people making TikTok.

rfoo 5y ago
Bytedance uses Rust extensively in Feishu/Lark (which competes with Microsoft Teams). It is even listed at https://www.rust-lang.org/production/users https://www.rust-lang.org/production/users From what I read (in Chinese), this "Monoio" is meant to be used for the proxy/whatever part of their next-generation service mesh. So maybe a corp-wide thing.
mleonhard 5y ago
It looks like they shared a lot of internal projects: https://github.com/orgs/bytedance/repositories https://github.com/orgs/bytedance/repositories I'm pleasantly surprised. Have many other Chinese tech companies embraced FOSS?
- justincormack 5y ago
  Yes its an increasing trend. Much of it is way less well known outside China. But lots of code from Alibaba, Tencent and smaller companies etc. many new CNCF projects are from China.

Zababa 5y ago

There are four articles in Chinese about the design and implementation on one of the author's blog. Here's a link to the first one: https://www.ihcblog.com/rust-runtime-design-1/ https://www.ihcblog.com/rust-runtime-design-1/. I don't know the subject and Chinese enough to know how much is lost by automatic translation (Google in my case), but it looks relatively good. As an aside, Google translation integrated in Chrome breaks the formatting of the code blocks, which is surprising since they're in a <pre> block.

moggers 5y ago

I've found in my own use of rust I want async/nonblocking for two things. 1. Be able to timeout a read of a socket 2. Be able to select over multiple sockets, and read whichever is ready first Usually a combination of both. epoll/io_uring(I guess? I only ever did research on epoll) seem like the solution being handed to me on a silver platter, however my understanding is if you want to use either of those you're meant to use async in rust, and that while there are some libraries which provide interfaces for this kind of behavior outside of async, they're usually very ad-hoc and that the community is just very laser focused on async as a language construct. What I don't understand is why does Rust consider it necessary to introduce async, futures, runtimes, an executor, async versions of TcpListeners, Files, etc for this? Why can't I just have a function in the standard library that takes a slice of std::net::TcpListeners, a timeout, blocks, and then gives me whichever is ready to read, when its ready to read, or nothing if the timeout is reached? Its not like I was going to do anything else on that thread while I wait for a packet to be received, it can happily be parked. Instead I have to select a runtime, and libraries compatible with the runtime, replace all the TcpListeners from the standard library I'm using with tokio TcpListeners or whatever, deal with API change, and now I have to deal with the "turtles all the way down" problem of async as well. That's not even to get into the whole nightmare that a lot of really sick libraries which I want to use in a blocking nature, are now only providing async APIs, which means its "async or the highway". I am very much not happy about this situation and I don't know what I can do about it. It seems the only response I ever get is "just use it, its easy" but that is not at all convincing me. I don't want to use it!

mleonhard 5y ago
If you need to handle <100,000 sockets, you will probably be fine with a thread per socket. Call set_read_timeout to implement the deadline. Run load tests and adjust the socket limit. Async lets one process handle millions of sockets. If you want to handle millions of sockets with threads, you can use `mio` [0]. Mio's API has footguns that can cause UB. If your wire protocol is complicated, you may find yourself implementing something like async, but worse. [0] https://crates.io/crates/mio https://crates.io/crates/mio
GolDDranks 5y ago

zinclozenge 5y ago

There's a ton of bleeding edge and novel work being done by Chinese companies and communities due to their insane scale requirements. I've been trying to learn the language to gain more insight and stay current with what they're doing.

jkelleyrtp 5y ago

It's exciting to see another thread-per-core async runtime for Rust. It's truly understated how difficult the Send + Sync requirements in Tokio are for writing regular code. It's typically rare for async tasks in Tokio to be used across two threads simultaneously, but now all of your data must be Send+Sync. Plus, Tokio is unique in that it's one of the very few runtimes in existence that's work-stealing (meaning the task can move off its original thread). Most other runtimes in other languages do not have that requirement, meaning you can use traditional Cell/RefCell/Rc instead of their slower, atomic variants. Right now, the best you can do is write a thread pool that spawns a Tokio LocalSet to run !Send futures. In fact, this is what Actin-web does to achieve its crazy performance. Web requests finish so quickly you rarely need work-stealing to achieve good performance, and often the cost of atomics/stealing is greater than the performance gain.

verdagon 5y ago
I've been thinking about how one could get around the Send + Sync requirements, it's a fascinating conundrum, from a language design standpoint. If we didn't have the Send + Sync requirements, and multiple async functions are running concurrently on the same thread, and multiple of them locked the same RefCell, might that cause a panic?
- masklinn 5y ago
  > If we didn't have the Send + Sync requirements, and multiple async functions are running concurrently on the same thread, and multiple of them locked the same RefCell, might that cause a panic? Hopefully.
asymptotic 5y ago
> the best you can do is write a thread pool that spawns a Tokio LocalSet to run !Send futures Can you go into this a bit more or provide a code reference? I'm new to writing async Rust code and am interested in this idea.
- jkelleyrtp 5y ago

_vvhw 5y ago

With context switches becoming more and more expensive relative to faster and faster I/O devices, almost the same order of magnitude, I believe that thread-per-core is where things are heading, because the alternative of not doing thread-per-core might literally be halving your throughput. That's also the most exciting thing about io_uring for me: how it enables a simple, single-threaded and yet highly performant thread-per-core control plane, outsourcing to the kernel thread pool for the async I/O data plane, instead of outsourcing to a user space thread pool as in the past. It's much more efficient and at the same time, much easier to reason about. There's no longer the need for multithreading to leak into the control plane. My experience with io_uring has been mostly working on TigerBeetleDB [1], a new distributed database that can process a million financial transactions a second, and I find it's a whole new way of thinking... that you can now just submit I/O directly from the control plane without blocking and without the cost of a context switch. It really changes the kinds of designs you can achieve, especially in the storage space (e.g. things like LSM-tree compactions can become much more parallel and incremental, while also becoming much simpler, i.e. no longer any need to think of memory barriers). Fantastic also to now have a unified API for networking/storage. So much good stuff in io_uring. Exciting times. [1] https://www.tigerbeetle.com https://www.tigerbeetle.com

sitkack 5y ago
Totally. And these thread per core apps can talk directly to virtio or NVMe passed into the kvm guest, you can get the best of having a unix host but have applications that run directly under KVM w/o sacrificing a rich control plane. And control plane reliability doesn't impact the data plane. Wonderful times!

billconan 5y ago

I have a stupid question, why isn't an async runtime a language feature of rust? we don't seem to see so many async runtimes in other languages? They seem to have a default way to run async tasks?

FerdSlav 5y ago
an async runtime inherently requires knowledge of the underlying system which Rust remains agnostic to. Rust can be written to target bare-metal (where no OS exists) to WASM and everything in between. If a runtime was added to the language itself you would inherently limit the flexibility of the platforms by which it could target.
pixel_fcker 5y ago
It sort of goes against Rust’s philosophy to bake anything like that into the language. Where they screwed up was not providing the machinery to make libraries agnostic of the runtime an end user wants to use in their program, so libraries either depend on a specific runtime explicitly or use features to allow users to switch runtimes at compile time. This causes a lot of headaches for library maintainers and end users both. There’s a lot of interest in adding said machinery (through collections of traits in std) to enable libraries to be generic over different runtimes, but a solution is still some ways off.
masklinn 5y ago
> I have a stupid question, why isn't an async runtime a language feature of rust? Because it was considered inimical to the core values and purpose of the language, which is to be a systems language. An async runtime being a language feature means a runtime is a language feature, and that is very much undesirable (in fact it used to be part of the language and was removed as it "settled" into its niche from its original design, which was much higher level and more applicative).

renox 5y ago

Does someone knows the pro/cons of io_uring vs DPDK in these these one thread per core setup?