2 ms·

There are some things that current RPC frameworks simply cannot touch. I've been up to my elbows in TCP lately. Working on a low-latency, select-based socket s

There are some things that current RPC frameworks simply cannot touch. I've been up to my elbows in TCP lately. Working on a low-latency, select-based socket server for streaming gaming applications. Each instance runs on 1 thread so there is absolutely no context switching delay when servicing player requests. The TCP streams are responsible for moving player inputs and game state as quickly as possible between system elements. There's also an externally facing websocket being serviced inside of these threads. The whole networking chunk of the loop usually completes within few tens of uS. Socket selects hang out for up to 10uS. I considered something like AspNetCore/kestrel, but that's a lot of machinery, GC liability, thread contention, et.al. gRPC also crossed my mind but I found some headache with the tooling.

Karrot_Kream 4y ago
Do you mean the `select` call? libuv and friends can offer higher throughput than `select`.
- User23 4y ago
  It’s been many many years since I’ve done low level multiplexing but I think epoll() is the no longer quite so new hotness? Ah I see libuv is a cross platform lib. I’d probably use that for production code, but it’s still really valuable to work with the low level API enough to understand it.
  - Karrot_Kream 4y ago
    Yeah epoll is what libuv calls on Linux. And agreed I think it's very valuable to try and use epoll or equivalent yourself at least once to understand what's happening.
- bob1029 4y ago
  Whatever OS primitive this API effectively calls is what I use: https://learn.microsoft.com/en-us/dotnet/api/system.net.sockets.socket.select?view=net-7.0#system-net-sockets-socket-select(system-collections-ilist-system-collections-ilist-system-collections-ilist-system-int32) https://learn.microsoft.com/en-us/dotnet/api/system.net.sock... Edit: This is the source: https://source.dot.net/#System.Net.Sockets/System/Net/Sockets/SocketPal.Unix.cs,a83ac8f2e1e985a6 https://source.dot.net/#System.Net.Sockets/System/Net/Socket...
  - Karrot_Kream 4y ago
    I didn't dive past the links you dropped, but reading the doc comment makes it seem like the call is using `poll` (as an alternative to `select` for FD size reasons.) It's well recognized that the polling model of `select` and `poll` are slow and it was specifically this problem that led to the framing of the C10K problem. The solution is to use an async style of programming which is what libuv and other wrappers offer (piggybacking off `epoll` on Linux.)
    - dist1ll 4y ago
      Interestingly enough, polling is still preferred in some cases of low-latency networking, because the overhead of NIC interrupts can become significant under high utilization. Of course in those cases you're running on bare-metal with kernel bypass networking, so there's no user<->kernel switch like in poll
    - bob1029 4y ago
      Async style brings downsides with regard to real time latency. As noted in other comments here, I am not pushing more than 1k per socket server.
    - dfox 4y ago
      The issue with select(2)/poll(2) (and for that matter WaitForMultipleObjects()) is that you are passing huge datastructure across userspace/kernel boundary with each call. There are various alternative platform specific implementations of what in the end is still polling that place the set of interesting FDs on the kernel side and thus make the whole thing significantly faster. That is what libev/libevent/libuv abstracts away in portable manner. The programming style is essentially same for a program that uses select(2) directly. (If you do not count various anti-patterns that are common in GUI applications which do networking)
User23 4y ago
A well written C mux is absurdly performant. I was handling thousands of requests per second with efficient hand rolled binary protocols in the early oughts when I could make UDP work, which I could since it was an intra datacenter app and a super lightweight exponential backoff retry scheme on the client side was entirely adequate. “Modern” network programming is hilariously inefficient. I think games are the only mass market exception since that’s the one area where optimal network performance affects pleasantness. Gamers used to speak of netcode almost reverently.
- bob1029 4y ago
  I am not even planning to push any sort of limits here. There are architectural elements which coalesce requests towards the edges into fewer, larger pipes. At most, any one of my socket servers will be responsible for 1k connections.
- ur-whale 4y ago
  This. The waste of doing RPC over TCP is simply astounding.
chasil 4y ago
I've used NFS over stunnel and I don't see any performance problems, but I do have the perception that it is not a panacea. I've read that kerberos NFS encryption has performance problems that stunnel solves. I've never sunk down to ONC RPC, but it does feel like it's a hack compared to QUIC.

Karrot_Kream 4y ago

Do you mean the `select` call? libuv and friends can offer higher throughput than `select`.

User23 4y ago
It’s been many many years since I’ve done low level multiplexing but I think epoll() is the no longer quite so new hotness? Ah I see libuv is a cross platform lib. I’d probably use that for production code, but it’s still really valuable to work with the low level API enough to understand it.
- Karrot_Kream 4y ago
  Yeah epoll is what libuv calls on Linux. And agreed I think it's very valuable to try and use epoll or equivalent yourself at least once to understand what's happening.
bob1029 4y ago
Whatever OS primitive this API effectively calls is what I use: https://learn.microsoft.com/en-us/dotnet/api/system.net.sockets.socket.select?view=net-7.0#system-net-sockets-socket-select(system-collections-ilist-system-collections-ilist-system-collections-ilist-system-int32) https://learn.microsoft.com/en-us/dotnet/api/system.net.sock... Edit: This is the source: https://source.dot.net/#System.Net.Sockets/System/Net/Sockets/SocketPal.Unix.cs,a83ac8f2e1e985a6 https://source.dot.net/#System.Net.Sockets/System/Net/Socket...
- Karrot_Kream 4y ago
  I didn't dive past the links you dropped, but reading the doc comment makes it seem like the call is using `poll` (as an alternative to `select` for FD size reasons.) It's well recognized that the polling model of `select` and `poll` are slow and it was specifically this problem that led to the framing of the C10K problem. The solution is to use an async style of programming which is what libuv and other wrappers offer (piggybacking off `epoll` on Linux.)
  - dist1ll 4y ago
    Interestingly enough, polling is still preferred in some cases of low-latency networking, because the overhead of NIC interrupts can become significant under high utilization. Of course in those cases you're running on bare-metal with kernel bypass networking, so there's no user<->kernel switch like in poll
  - bob1029 4y ago
    Async style brings downsides with regard to real time latency. As noted in other comments here, I am not pushing more than 1k per socket server.
  - dfox 4y ago
    The issue with select(2)/poll(2) (and for that matter WaitForMultipleObjects()) is that you are passing huge datastructure across userspace/kernel boundary with each call. There are various alternative platform specific implementations of what in the end is still polling that place the set of interesting FDs on the kernel side and thus make the whole thing significantly faster. That is what libev/libevent/libuv abstracts away in portable manner. The programming style is essentially same for a program that uses select(2) directly. (If you do not count various anti-patterns that are common in GUI applications which do networking)

User23 4y ago

A well written C mux is absurdly performant. I was handling thousands of requests per second with efficient hand rolled binary protocols in the early oughts when I could make UDP work, which I could since it was an intra datacenter app and a super lightweight exponential backoff retry scheme on the client side was entirely adequate. “Modern” network programming is hilariously inefficient. I think games are the only mass market exception since that’s the one area where optimal network performance affects pleasantness. Gamers used to speak of netcode almost reverently.

bob1029 4y ago
I am not even planning to push any sort of limits here. There are architectural elements which coalesce requests towards the edges into fewer, larger pipes. At most, any one of my socket servers will be responsible for 1k connections.
ur-whale 4y ago
This. The waste of doing RPC over TCP is simply astounding.

chasil 4y ago

I've used NFS over stunnel and I don't see any performance problems, but I do have the perception that it is not a panacea. I've read that kerberos NFS encryption has performance problems that stunnel solves. I've never sunk down to ONC RPC, but it does feel like it's a hack compared to QUIC.