3 ms·

RPC vs. TCP (Redux)

67 points by signa11 4y ago

sjm 4y ago
Previous article on HN: https://news.ycombinator.com/item?id=34871710 https://news.ycombinator.com/item?id=34871710
- dang 4y ago
  Thanks! Macroexpanded: It's TCP vs. RPC All over Again - https://news.ycombinator.com/item?id=34871710 https://news.ycombinator.com/item?id=34871710 - Feb 2023 (112 comments)
- cryptonector 4y ago
  Excellent, now I don't have to make similar comments.
jerf 4y ago
I have a hard time wrapping my mind around this whole line of discussion because it seems to treat as a binary something that is a continuum. It is not the case that we have "TCP" and "RPC" as some sort of cleanly separable discussion, or at least, if they do mean it as some very cleanly definable term it doesn't seem to have ever gotten a discussion. RPC, to my mind, isn't really a thing of its own. It's a particular popular pattern for using messages. You can use that pattern poorly or well on TCP. You may need to use another underlying protocol for any of a variety of reasons to get another performance profile out of it. You may be in a situation where you need all the guarantees of TCP anyhow, in which case the room for improvement is somewhat marginal (although non-zero, especially if you want to be on the cell network). I'm not particularly convinced there's a need or even "room" for a pure RPC protocol, on the grounds that it'd probably end up much like SOAP in that it can only serve a particular set of needs and the benefit of a custom protocol for those needs could well be outweighed by the costs of the stampede of developers jamming everything into the new protocol regardless of their needs, most of which would mismatch. The field of "RPC" is already very diverse and to my eyes, people generally have satisfactory solutions available to them. It's not like it's even possible to build something that would both replace generalized RPC on the internet and service supercomputer mesh networks. If TCP was the only option, yes, many needs would be going unmet, but it already isn't. If I do an analysis and determine that TCP is my core problem, it seems to me I already have an abundance of choices serving many points on the continuum. I don't see what trying to lift up one of those points as some sort of standard would particularly accomplish that isn't already being accomplished in the real world. And it isn't even RPC qua RPC driving that per se; you can have all sorts of niche needs that drive you off of TCP, of which RPC doesn't even strike me as a particularly distinguished case. RPC when you get down to it just isn't that complicated or really even that interesting; you send a message, you expect a 1-to-1 message back in reply. Of all the message patterns, it's just about the simplest other than unilateral broadcast without any reply at all.
- candiddevmike 4y ago
  RPC over SCTP could be interesting, not sure of any libraries that support it out of the box. Could also be hell for enterprises/firewall admins.
- eklitzke 4y ago
  You can obviously build an RPC protocol on top of TCP, people have been doing it for decades. However there are things that are difficult to efficiently implement on top of TCP (multiplexing different logical streams over the same connection, message priorities) in the TCP model, and some things that are impossible to implement over TCP (for example, you don't really have any ability to influence what happens during congestion control scenarios). There are also a bunch of things about TCP that are fundamentally inefficient on modern hardware. Do these things matter if you're running some RPC service over the internet with a Python backend? Almost certainly not. But if you read John Ousterhout's paper or watch his Netdev talk^[1] it's also pretty clear that this isn't the use case he's been talking about. He's talking about RPC in the datacenter, where in theory modern hardware is capable of delivering RPC messages with microsecond scale latency but in practice no RPC systems built on TCP come anywhere close to that. The Netdev talk I've linked below is really good and John makes a really thorough presentation of his ideas and what he's actually trying to optimize for. [1]: https://www.youtube.com/watch?v=o2HBHckrdQc https://www.youtube.com/watch?v=o2HBHckrdQc
  - tveita 4y ago
    Interesting that gRPC is too slow for the latencies they're aiming at, maybe Cap'n Proto would be a better match. He doesn't go into details on how they handle packet loss, but the paper[1] says: Receivers use a simple timeout-based mechanism to detect lost packets. If a long time period (a few milliseconds) elapses without additional packets arriving for a message, the receiver sends a RESEND packet that identifies the first range of missing bytes; the sender will then retransmit those bytes. Their implementation uses 15 ms as a default, so even crossing regions e.g. between us-west-1 and us-west-2 would time out [2]. So they're really only targetting communication within a single datacenter, this is not a protocol for "the Internet". Maybe annoying if you need to switch protocols depending on where your target server is located at. [1] https://people.csail.mit.edu/alizadeh/papers/homa-sigcomm18.pdf https://people.csail.mit.edu/alizadeh/papers/homa-sigcomm18.... [2] https://www.cloudping.co/grid https://www.cloudping.co/grid
cryptonector 4y ago
> No one would argue that implementing a reliable byte stream service, bundled with a fair congestion control algorithm, should be left to the application. User-land TCP stacks exist. Sure, that's not "leaving it to the application", but still. Re: HPC and RDMA, yes, if you want every ounce of performance (HPC) then you want RDMA. Lustre was doing that, what, 20+ years ago? But the price you pay is that you get no congestion control, and you have to plan everything. That works great in certain data centers -- it works great if you're CERN recording petabytes of data from one particle physics experiment. It also works great in data centers in general if you don't exceed limits. > Saving the best for last, my third observation is the consequence of an exchange between David Reed and John Ousterhout. David argued that “RPC was one big reason for creating UDP as a datagram ‘process addressing’ support layer” and that “...argument passing and waiting for a return was way too high level for a transport protocol.” Er, UDP goes back at least to 1980, and as such it predates the RPCs of the 80s (Apollo Domain, Sun's ONC RPC, etc).

RPC vs. TCP (Redux)

67 points by signa11 4y ago

sjm 4y ago

Previous article on HN: https://news.ycombinator.com/item?id=34871710 https://news.ycombinator.com/item?id=34871710

dang 4y ago
Thanks! Macroexpanded: It's TCP vs. RPC All over Again - https://news.ycombinator.com/item?id=34871710 https://news.ycombinator.com/item?id=34871710 - Feb 2023 (112 comments)
cryptonector 4y ago
Excellent, now I don't have to make similar comments.

jerf 4y ago

I have a hard time wrapping my mind around this whole line of discussion because it seems to treat as a binary something that is a continuum. It is not the case that we have "TCP" and "RPC" as some sort of cleanly separable discussion, or at least, if they do mean it as some very cleanly definable term it doesn't seem to have ever gotten a discussion. RPC, to my mind, isn't really a thing of its own. It's a particular popular pattern for using messages. You can use that pattern poorly or well on TCP. You may need to use another underlying protocol for any of a variety of reasons to get another performance profile out of it. You may be in a situation where you need all the guarantees of TCP anyhow, in which case the room for improvement is somewhat marginal (although non-zero, especially if you want to be on the cell network). I'm not particularly convinced there's a need or even "room" for a pure RPC protocol, on the grounds that it'd probably end up much like SOAP in that it can only serve a particular set of needs and the benefit of a custom protocol for those needs could well be outweighed by the costs of the stampede of developers jamming everything into the new protocol regardless of their needs, most of which would mismatch. The field of "RPC" is already very diverse and to my eyes, people generally have satisfactory solutions available to them. It's not like it's even possible to build something that would both replace generalized RPC on the internet and service supercomputer mesh networks. If TCP was the only option, yes, many needs would be going unmet, but it already isn't. If I do an analysis and determine that TCP is my core problem, it seems to me I already have an abundance of choices serving many points on the continuum. I don't see what trying to lift up one of those points as some sort of standard would particularly accomplish that isn't already being accomplished in the real world. And it isn't even RPC qua RPC driving that per se; you can have all sorts of niche needs that drive you off of TCP, of which RPC doesn't even strike me as a particularly distinguished case. RPC when you get down to it just isn't that complicated or really even that interesting; you send a message, you expect a 1-to-1 message back in reply. Of all the message patterns, it's just about the simplest other than unilateral broadcast without any reply at all.

candiddevmike 4y ago
RPC over SCTP could be interesting, not sure of any libraries that support it out of the box. Could also be hell for enterprises/firewall admins.
eklitzke 4y ago
You can obviously build an RPC protocol on top of TCP, people have been doing it for decades. However there are things that are difficult to efficiently implement on top of TCP (multiplexing different logical streams over the same connection, message priorities) in the TCP model, and some things that are impossible to implement over TCP (for example, you don't really have any ability to influence what happens during congestion control scenarios). There are also a bunch of things about TCP that are fundamentally inefficient on modern hardware. Do these things matter if you're running some RPC service over the internet with a Python backend? Almost certainly not. But if you read John Ousterhout's paper or watch his Netdev talk^[1] it's also pretty clear that this isn't the use case he's been talking about. He's talking about RPC in the datacenter, where in theory modern hardware is capable of delivering RPC messages with microsecond scale latency but in practice no RPC systems built on TCP come anywhere close to that. The Netdev talk I've linked below is really good and John makes a really thorough presentation of his ideas and what he's actually trying to optimize for. [1]: https://www.youtube.com/watch?v=o2HBHckrdQc https://www.youtube.com/watch?v=o2HBHckrdQc
- tveita 4y ago
  Interesting that gRPC is too slow for the latencies they're aiming at, maybe Cap'n Proto would be a better match. He doesn't go into details on how they handle packet loss, but the paper[1] says: Receivers use a simple timeout-based mechanism to detect lost packets. If a long time period (a few milliseconds) elapses without additional packets arriving for a message, the receiver sends a RESEND packet that identifies the first range of missing bytes; the sender will then retransmit those bytes. Their implementation uses 15 ms as a default, so even crossing regions e.g. between us-west-1 and us-west-2 would time out [2]. So they're really only targetting communication within a single datacenter, this is not a protocol for "the Internet". Maybe annoying if you need to switch protocols depending on where your target server is located at. [1] https://people.csail.mit.edu/alizadeh/papers/homa-sigcomm18.pdf https://people.csail.mit.edu/alizadeh/papers/homa-sigcomm18.... [2] https://www.cloudping.co/grid https://www.cloudping.co/grid

cryptonector 4y ago

> No one would argue that implementing a reliable byte stream service, bundled with a fair congestion control algorithm, should be left to the application. User-land TCP stacks exist. Sure, that's not "leaving it to the application", but still. Re: HPC and RDMA, yes, if you want every ounce of performance (HPC) then you want RDMA. Lustre was doing that, what, 20+ years ago? But the price you pay is that you get no congestion control, and you have to plan everything. That works great in certain data centers -- it works great if you're CERN recording petabytes of data from one particle physics experiment. It also works great in data centers in general if you don't exceed limits. > Saving the best for last, my third observation is the consequence of an exchange between David Reed and John Ousterhout. David argued that “RPC was one big reason for creating UDP as a datagram ‘process addressing’ support layer” and that “...argument passing and waiting for a return was way too high level for a transport protocol.” Er, UDP goes back at least to 1980, and as such it predates the RPCs of the 80s (Apollo Domain, Sun's ONC RPC, etc).