4 ms·

My side project has been to rewrite https://github.com/erpc-io/eRPC https://github.com/erpc-io/eRPC which does RPCs over UDP with some congestion control suppos

by scrubs 5y ago

My side project has been to rewrite https://github.com/erpc-io/eRPC https://github.com/erpc-io/eRPC which does RPCs over UDP with some congestion control supposedly quite fast. Paper: https://www.usenix.org/system/files/nsdi19-kalia.pdf https://www.usenix.org/system/files/nsdi19-kalia.pdf. I never got the code to work in AWS; I believe the author focused on Mellanox NICS but that's not really commodity H/W, which is where my interests lay. So I dug into it .. and well I'll have my own library soon. I should be able to send UDP w/o congestion control sometime this week. eRPC uses DPDK (100% user space NIC TX/RX control) plus the author's own other ideas to get performance. Since I'm getting into DPDK way, way late in the game, I hope and believe DPDK is and will be the better way to go than to turn to kernel fix-ups. Getting the kernel out of the way with pinned threads seems -- or more exotic with special H/W if preferred --- is cleaner if one can develop from scratch. DPDK is all day long zero copy, and always has been. This library will be a part of something bigger. For many frameworks, a key architecture point is: I got a RPC/packet/message. Ok, now what? * process it in-place e.g. on the thread that was doing RX? * if I don't delegate am I making back-pressure? * delegate it to another core? How to efficiently do that? Hint: https://blog.acolyer.org/2017/12/04/ffwd-delegation-is-much-faster-than-you-think/ https://blog.acolyer.org/2017/12/04/ffwd-delegation-is-much-... * if I delegate ... to whom? Maybe I'm partitioning? * If I delegate how do I get a response back? In DPDK I believe these are easier to decide and well-manage in code.

goombacloud 5y ago
Depending on the requirements of the RPC you could implement the logic in eBPF code over the XDP framework which can directly execute the code inside the NIC driver and modify the packet buffer to be a response packet that gets send out immediately without ever going through the rest of the kernel nor userspace.
- scrubs 5y ago
  That option was not on my radar. >execute the code inside the NIC driver this is ... a NIC+FPGA which can execute code?
  - spijdar 5y ago
    No, it's offloading the code onto the processor on the NIC (which usually have several general purpose CPU cores). According to Wikipedia [0] it's not widely supported in hardware, but a few companies like Intel are "working on it". I guess the upshot is it gracefully "degrades" to running on the host CPU if on an unsupported NIC. [0] https://en.wikipedia.org/wiki/Express_Data_Path https://en.wikipedia.org/wiki/Express_Data_Path
    - goombacloud 5y ago
      Offloading to the NIC is of course the fastest way but even doing it in the NIC driver (check if the NIC driver supports it or you have to use the slower generic fallback) you already have advantages similar to doing it with DPDK.
anujkaliaitd 5y ago
(Author of eRPC.) I'm glad you've found the project useful, and my apologies for the code and build quality, and NIC compatibility issues that you faced. I'm very interested in learning from your experience (perhaps even as a GitHub issue on our repo), especially w.r.t. how the code organization should be improved, although I understand the list might be too long :). I know of some issues myself that I plan to fix, e.g., using macros to decide which Transport class to use -- this should just use virtual methods; messy inter-mixing of congestion control and packet I/O; some "performance optimizations" like preallocated buffers that are just not worth it.
- scrubs 5y ago
  I'll be in touch you can be sure. Code is in GIT but I will not make it public until I have UDP working --- hopefully +/- 3 days. I do and will continue mention your paper (which I also found on HN) and your repo because it has a lot of really good ideas ... ideas which I'm forced to engage to mastery. I haven't yet had time to focus on how you deal with packet loss via and with congestion management. Anything you can do --- I'll have to reread your paper (well written BTW) to pull my own weight here --- is a real benefit to the community.
Matthias247 5y ago
If you want to get packets fast into userspace you can now apparently also AF_XDP on Linux which is also effectively a kernel bypass. The driver will put packets into a ringbuffer before they hit the remaining kernel, and userspace can fetch them effectively there. I haven’t benchmarked for the same application, but it promises to give similar levels of performance. I think the other questions are more ones of library design and less about the packet API one uses. On top of all kinds of APIs one could build singlethreaded state machines which deque packets and process them. Or other architectures if required.
- Agingcoder 5y ago
  I think the paper describing xdp mentioned a 30% performance drop compared to a full kernel bypass - it's quite good.

by scrubs 5y ago

goombacloud 5y ago

Depending on the requirements of the RPC you could implement the logic in eBPF code over the XDP framework which can directly execute the code inside the NIC driver and modify the packet buffer to be a response packet that gets send out immediately without ever going through the rest of the kernel nor userspace.

scrubs 5y ago
That option was not on my radar. >execute the code inside the NIC driver this is ... a NIC+FPGA which can execute code?
- spijdar 5y ago
  No, it's offloading the code onto the processor on the NIC (which usually have several general purpose CPU cores). According to Wikipedia [0] it's not widely supported in hardware, but a few companies like Intel are "working on it". I guess the upshot is it gracefully "degrades" to running on the host CPU if on an unsupported NIC. [0] https://en.wikipedia.org/wiki/Express_Data_Path https://en.wikipedia.org/wiki/Express_Data_Path
  - goombacloud 5y ago
    Offloading to the NIC is of course the fastest way but even doing it in the NIC driver (check if the NIC driver supports it or you have to use the slower generic fallback) you already have advantages similar to doing it with DPDK.

anujkaliaitd 5y ago

(Author of eRPC.) I'm glad you've found the project useful, and my apologies for the code and build quality, and NIC compatibility issues that you faced. I'm very interested in learning from your experience (perhaps even as a GitHub issue on our repo), especially w.r.t. how the code organization should be improved, although I understand the list might be too long :). I know of some issues myself that I plan to fix, e.g., using macros to decide which Transport class to use -- this should just use virtual methods; messy inter-mixing of congestion control and packet I/O; some "performance optimizations" like preallocated buffers that are just not worth it.

scrubs 5y ago
I'll be in touch you can be sure. Code is in GIT but I will not make it public until I have UDP working --- hopefully +/- 3 days. I do and will continue mention your paper (which I also found on HN) and your repo because it has a lot of really good ideas ... ideas which I'm forced to engage to mastery. I haven't yet had time to focus on how you deal with packet loss via and with congestion management. Anything you can do --- I'll have to reread your paper (well written BTW) to pull my own weight here --- is a real benefit to the community.

Matthias247 5y ago

If you want to get packets fast into userspace you can now apparently also AF_XDP on Linux which is also effectively a kernel bypass. The driver will put packets into a ringbuffer before they hit the remaining kernel, and userspace can fetch them effectively there. I haven’t benchmarked for the same application, but it promises to give similar levels of performance. I think the other questions are more ones of library design and less about the packet API one uses. On top of all kinds of APIs one could build singlethreaded state machines which deque packets and process them. Or other architectures if required.

Agingcoder 5y ago
I think the paper describing xdp mentioned a 30% performance drop compared to a full kernel bypass - it's quite good.