4 ms·

Based solely on the one example function on the homepage, this seems to be pre-empting asynchronous functions, presumably a fiber yielding and resuming internal

Based solely on the one example function on the homepage, this seems to be pre-empting asynchronous functions, presumably a fiber yielding and resuming internally, so that the code looks synchronous. libuv's API is based on callbacks. So both the implementation and the usage are very different.

throwaway81523 4y ago
This uses coroutines so there is no preemption. It is much higher level than libuv. I would say it is higher level than Seastar, which uses callbacks. After about 10 minutes of skimming the docs, I don't see a mention of what the actual coroutine mechanism is, and I don't see any mention of Boost or C++20 coroutines. There are a bunch of IO primitives that operate with their coroutine system but I didn't check to see how they work. I'd like to see versions of all the usual Linux system calls exposed through io_uring instead of having to use different async options or threadpools for everything. I don't know how close we are to that now. I don't know whether userver uses io_uring at all, but it would be nice if it does. There is a comparison chart with several other such frameworks in various languages, Seastar is not in the chart, but it would be nice if it was. DPDK is not mentioned unless I missed it (which is possible). I don't know if it is still supposed to be faster than Linux kernel networking. Seastar has good support for it. The coroutines are stackful and it looks like you have to allocate space for them on instantiation. I don't know if there is any attempt to catch stack overflows. The obvious approaches using mmap might help though they aren't really sound. There is a hardware extension (CHERI) for pointer ownership on ARM and RISC-V that might be cool for this, so I hope it gets traction. I've never gotten into the deep weeds of cooperative multitasking, but people who have say it gets ugly as the program gets more complicated, and that it tends to give way to OS's with preemption and processes for that reason. C++ itself is also famously fraught with hazards. Overall userver looks useful and worth looking into so I'm glad to hear about this release.
- Arnavion 4y ago
  >This uses coroutines so there is no preemption. Right, I meant cooperative yielding (which fibers are), but accidentally called it the exact opposite. Whoops. >I don't see a mention of what the actual coroutine mechanism is, and I don't see any mention of Boost or C++20 coroutines It's boost coroutines. The /uboost_coro directory is a vendored copy. It's used in /core/src/engine/coro/pool.hpp
  - throwaway81523 4y ago
    Thanks. I have been wondering for a while if Seastar might someday switch to this approach. I also wonder how the overall performance compares with current versions of Go and Goroutines. I don't like Go much from a pure language perspective, but I do like having a garbage collected runtime. I last heard you pay around 2x overhead for it, which is not that bad for most purposes.
- jnordwick 4y ago
  > through io_uring instead of having to use different async options or threadpools for everything io_uring is a threadpool, and to get performance above epoll, it probably needs to expose the pools and the scheduling choices. eBPF for sceduling? All the simplicity of io_uring basically goes out the window wit that. I added ZEROCOPY to an io_uring instance and it wasn't a fun experience. It is a very strange interface for networking code. Sending is already mostly async for TCP, and for UDP almost useless (esp without send/recv mmsg calls - you have to make all the packet ordering explicit when create the requests - complete pain in the ass. On the receiving end you have to have an outstanding read request even though you have no idea how much you are going to read (reminds me of the HTTP async hack). The Mmap Packet interface on that is better, just dropping packets off in your address space - for the most part read is a useless call for high perforamnce stuff - you're always reading and buffering. Reading UDP sucks also since no mmsg call and you feel like you're pickign up thousands of pennies off the ground because the asshole wouldn't give you a $10 bill. It doesn't compose well, hard to debug, etc. There are other interfaces (besides the old, decrepit 200+ verb RDMA/RoCe/iWarp crapfest coming out of the latest standards running jokes). Do something besides BSD sockets on the high end. Anybody rememebr SysV IPC - that might be intereting to try again. BSD netowrking is not the end all of networking APIs.
  - ncmncm 4y ago
    If you are serious about network throughput, you are doing kernel bypass. Packets show up in your process memory in a ring buffer via DMA, asynchronously, with your code just polling for updates to a head index also updated via DMA from the network hardware. Packets are sent by appending them to another ring buffer and then writing to a device register also mapped.
    - mgaunard 4y ago
      There isn't a significant difference between this and io_uring, both in terms of performance and API. With io_uring the kernel is doing that for you (you have control of the affinity and how aggressively the kernel thread polls), and copying it to another ring buffer, which is then picked up by userland. You still get sub-microsecond latency even with this two-level approach, and the advantage is that it works on every NIC that linux has a driver for.

throwaway81523 4y ago

This uses coroutines so there is no preemption. It is much higher level than libuv. I would say it is higher level than Seastar, which uses callbacks. After about 10 minutes of skimming the docs, I don't see a mention of what the actual coroutine mechanism is, and I don't see any mention of Boost or C++20 coroutines. There are a bunch of IO primitives that operate with their coroutine system but I didn't check to see how they work. I'd like to see versions of all the usual Linux system calls exposed through io_uring instead of having to use different async options or threadpools for everything. I don't know how close we are to that now. I don't know whether userver uses io_uring at all, but it would be nice if it does. There is a comparison chart with several other such frameworks in various languages, Seastar is not in the chart, but it would be nice if it was. DPDK is not mentioned unless I missed it (which is possible). I don't know if it is still supposed to be faster than Linux kernel networking. Seastar has good support for it. The coroutines are stackful and it looks like you have to allocate space for them on instantiation. I don't know if there is any attempt to catch stack overflows. The obvious approaches using mmap might help though they aren't really sound. There is a hardware extension (CHERI) for pointer ownership on ARM and RISC-V that might be cool for this, so I hope it gets traction. I've never gotten into the deep weeds of cooperative multitasking, but people who have say it gets ugly as the program gets more complicated, and that it tends to give way to OS's with preemption and processes for that reason. C++ itself is also famously fraught with hazards. Overall userver looks useful and worth looking into so I'm glad to hear about this release.

Arnavion 4y ago
>This uses coroutines so there is no preemption. Right, I meant cooperative yielding (which fibers are), but accidentally called it the exact opposite. Whoops. >I don't see a mention of what the actual coroutine mechanism is, and I don't see any mention of Boost or C++20 coroutines It's boost coroutines. The /uboost_coro directory is a vendored copy. It's used in /core/src/engine/coro/pool.hpp
- throwaway81523 4y ago
  Thanks. I have been wondering for a while if Seastar might someday switch to this approach. I also wonder how the overall performance compares with current versions of Go and Goroutines. I don't like Go much from a pure language perspective, but I do like having a garbage collected runtime. I last heard you pay around 2x overhead for it, which is not that bad for most purposes.
jnordwick 4y ago
> through io_uring instead of having to use different async options or threadpools for everything io_uring is a threadpool, and to get performance above epoll, it probably needs to expose the pools and the scheduling choices. eBPF for sceduling? All the simplicity of io_uring basically goes out the window wit that. I added ZEROCOPY to an io_uring instance and it wasn't a fun experience. It is a very strange interface for networking code. Sending is already mostly async for TCP, and for UDP almost useless (esp without send/recv mmsg calls - you have to make all the packet ordering explicit when create the requests - complete pain in the ass. On the receiving end you have to have an outstanding read request even though you have no idea how much you are going to read (reminds me of the HTTP async hack). The Mmap Packet interface on that is better, just dropping packets off in your address space - for the most part read is a useless call for high perforamnce stuff - you're always reading and buffering. Reading UDP sucks also since no mmsg call and you feel like you're pickign up thousands of pennies off the ground because the asshole wouldn't give you a $10 bill. It doesn't compose well, hard to debug, etc. There are other interfaces (besides the old, decrepit 200+ verb RDMA/RoCe/iWarp crapfest coming out of the latest standards running jokes). Do something besides BSD sockets on the high end. Anybody rememebr SysV IPC - that might be intereting to try again. BSD netowrking is not the end all of networking APIs.
- ncmncm 4y ago
  If you are serious about network throughput, you are doing kernel bypass. Packets show up in your process memory in a ring buffer via DMA, asynchronously, with your code just polling for updates to a head index also updated via DMA from the network hardware. Packets are sent by appending them to another ring buffer and then writing to a device register also mapped.
  - mgaunard 4y ago
    There isn't a significant difference between this and io_uring, both in terms of performance and API. With io_uring the kernel is doing that for you (you have control of the affinity and how aggressively the kernel thread polls), and copying it to another ring buffer, which is then picked up by userland. You still get sub-microsecond latency even with this two-level approach, and the advantage is that it works on every NIC that linux has a driver for.