4 ms·

You can obviously build an RPC protocol on top of TCP, people have been doing it for decades. However there are things that are difficult to efficiently impleme

You can obviously build an RPC protocol on top of TCP, people have been doing it for decades. However there are things that are difficult to efficiently implement on top of TCP (multiplexing different logical streams over the same connection, message priorities) in the TCP model, and some things that are impossible to implement over TCP (for example, you don't really have any ability to influence what happens during congestion control scenarios). There are also a bunch of things about TCP that are fundamentally inefficient on modern hardware. Do these things matter if you're running some RPC service over the internet with a Python backend? Almost certainly not. But if you read John Ousterhout's paper or watch his Netdev talk^[1] it's also pretty clear that this isn't the use case he's been talking about. He's talking about RPC in the datacenter, where in theory modern hardware is capable of delivering RPC messages with microsecond scale latency but in practice no RPC systems built on TCP come anywhere close to that. The Netdev talk I've linked below is really good and John makes a really thorough presentation of his ideas and what he's actually trying to optimize for. [1]: https://www.youtube.com/watch?v=o2HBHckrdQc https://www.youtube.com/watch?v=o2HBHckrdQc

tveita 4y ago
Interesting that gRPC is too slow for the latencies they're aiming at, maybe Cap'n Proto would be a better match. He doesn't go into details on how they handle packet loss, but the paper[1] says: Receivers use a simple timeout-based mechanism to detect lost packets. If a long time period (a few milliseconds) elapses without additional packets arriving for a message, the receiver sends a RESEND packet that identifies the first range of missing bytes; the sender will then retransmit those bytes. Their implementation uses 15 ms as a default, so even crossing regions e.g. between us-west-1 and us-west-2 would time out [2]. So they're really only targetting communication within a single datacenter, this is not a protocol for "the Internet". Maybe annoying if you need to switch protocols depending on where your target server is located at. [1] https://people.csail.mit.edu/alizadeh/papers/homa-sigcomm18.pdf https://people.csail.mit.edu/alizadeh/papers/homa-sigcomm18.... [2] https://www.cloudping.co/grid https://www.cloudping.co/grid
jmholla 4y ago
Not the GP, but I think the confusion stems from comparing TCP to RPC as if they are at odds with each other. TCP (Transmission Control Protocol) is a delivery protocol (i.e. how can we have a conversation between two places). RPC (Remote Procedure Call) describes how the messages should be treated (i.e. how each side should talk). So, RPC vs TCP doesn't make sense, even just looking at their names; a new protocol designed for RPC (maybe an RPCP) vs RPC over TCP does make sense.
- sophacles 4y ago
  This is just a name collision. Rpc in this context (that is transport layer protocols) just means a request /response pattern. A better category name would be nice. (Compare middleware - it's a concept that is pretty broad and has a lot of meanings over time but has been coopted to a subset that tends to mean "parse http headers and do things before the main endpoint handler")
  - cryptonector 4y ago
    It's a misnomer, really. HTTP is request/response. If you want a non-TCP request/response protocol, QUIC is quite good. > A better category name would be nice. Sure, but if QUIC is good enough, then we don't need a new category name yet.
    - deleted 4y ago
      [deleted]
- cryptonector 4y ago
  In earlier posts on this topic (HOMA) we saw that TFA wants a new upper layer protocol to replace TCP and which is strictly a request/response protocol. They call that an RPC. RPC is a misnomer that invites all sorts of quibbling, so they would do better to not call it an RPC. New ULPs have trouble getting deployed due to bad middleboxen. It can work in a data center, but you have to deal with provisioning carefully when you opt to toss the congestion control belt and suspenders. More importantly, QUIC exists and is probably good enough, and without evidence that it's not I find HOMA and similar proposals to be possibly premature optimization.
cryptonector 4y ago
You should check out Lustre RPC. It's basically RPC over RDMA, and based on Portals. It uses RDMA for bulk data. The actual RPC call/reply headers are all essentially a flavor of FlatBuffers before that existed, using C and recipient-makes-right for byte endianness. If you want an RPC that is not over TCP and which takes advantage of RDMA, then Lustre RPC is what you want to study first. I wouldn't necessarily copy Lustre RPC but only because one might want to use a better encoding for headers and a better language than C. But I would copy Lustre RPC in every other way unless a different choice of RDMA technology made that impossible.

tveita 4y ago

Interesting that gRPC is too slow for the latencies they're aiming at, maybe Cap'n Proto would be a better match. He doesn't go into details on how they handle packet loss, but the paper[1] says: Receivers use a simple timeout-based mechanism to detect lost packets. If a long time period (a few milliseconds) elapses without additional packets arriving for a message, the receiver sends a RESEND packet that identifies the first range of missing bytes; the sender will then retransmit those bytes. Their implementation uses 15 ms as a default, so even crossing regions e.g. between us-west-1 and us-west-2 would time out [2]. So they're really only targetting communication within a single datacenter, this is not a protocol for "the Internet". Maybe annoying if you need to switch protocols depending on where your target server is located at. [1] https://people.csail.mit.edu/alizadeh/papers/homa-sigcomm18.pdf https://people.csail.mit.edu/alizadeh/papers/homa-sigcomm18.... [2] https://www.cloudping.co/grid https://www.cloudping.co/grid

jmholla 4y ago

Not the GP, but I think the confusion stems from comparing TCP to RPC as if they are at odds with each other. TCP (Transmission Control Protocol) is a delivery protocol (i.e. how can we have a conversation between two places). RPC (Remote Procedure Call) describes how the messages should be treated (i.e. how each side should talk). So, RPC vs TCP doesn't make sense, even just looking at their names; a new protocol designed for RPC (maybe an RPCP) vs RPC over TCP does make sense.

sophacles 4y ago
This is just a name collision. Rpc in this context (that is transport layer protocols) just means a request /response pattern. A better category name would be nice. (Compare middleware - it's a concept that is pretty broad and has a lot of meanings over time but has been coopted to a subset that tends to mean "parse http headers and do things before the main endpoint handler")
- cryptonector 4y ago
  It's a misnomer, really. HTTP is request/response. If you want a non-TCP request/response protocol, QUIC is quite good. > A better category name would be nice. Sure, but if QUIC is good enough, then we don't need a new category name yet.
  - deleted 4y ago
    [deleted]
cryptonector 4y ago
In earlier posts on this topic (HOMA) we saw that TFA wants a new upper layer protocol to replace TCP and which is strictly a request/response protocol. They call that an RPC. RPC is a misnomer that invites all sorts of quibbling, so they would do better to not call it an RPC. New ULPs have trouble getting deployed due to bad middleboxen. It can work in a data center, but you have to deal with provisioning carefully when you opt to toss the congestion control belt and suspenders. More importantly, QUIC exists and is probably good enough, and without evidence that it's not I find HOMA and similar proposals to be possibly premature optimization.

cryptonector 4y ago

You should check out Lustre RPC. It's basically RPC over RDMA, and based on Portals. It uses RDMA for bulk data. The actual RPC call/reply headers are all essentially a flavor of FlatBuffers before that existed, using C and recipient-makes-right for byte endianness. If you want an RPC that is not over TCP and which takes advantage of RDMA, then Lustre RPC is what you want to study first. I wouldn't necessarily copy Lustre RPC but only because one might want to use a better encoding for headers and a better language than C. But I would copy Lustre RPC in every other way unless a different choice of RDMA technology made that impossible.