3 ms·

gRPC-style streaming can be implemented in terms of Cap'n Proto object references: When starting a streaming call, the client creates a callback object to be ca

by kentonv 10y ago

gRPC-style streaming can be implemented in terms of Cap'n Proto object references: When starting a streaming call, the client creates a callback object to be called by the server every time a new message is available, and then sends a reference to that object as part of the request. Or for client->server streaming, the server returns such an object reference. The drawbacks of callbacks that you mention don't apply in this scenario: The same network connection is utilized for the callback, solving the routing question. The callback object receives a notification when the caller is done with it (or disconnects), so it can clean up, solving the lifecycle question. Flow control / backpressure can be achieved by pausing calls to the callback if too many previous calls have not returned (I actually plan to bake this pattern into the library in the future to make it dead simple). So, it seems to me streaming in gRPC is actually a narrow subset of what Cap'n Proto can express. For web stack compatibility purposes, it's straightforward to implement Cap'n-Proto-over-WebSocket. I'm not sure that HTTP/2 buys much on top of that.

Matthias247 10y ago
Thanks for the explanation. If it works like you describe it seems to be a reasonable way to obtain streaming behavior. Will take a look at it if I find some time. Regarding backpressure and HTTP/2: You get some different kind of backpressure behavior with your approach (application level flow control) and the grpc approach (transport level flow control). Let's say you have 2 functions which are called in parallel. For one the arguments are very big (let's say 100kB), for the other one they are small (some bytes). With HTTP/2 and transport level flow control the small function could get the bandwidth as the big one, which means more requests/s for the small function. With pure application level flow control the small function needs to wait until the big one is fully sent before it can be put on the wire. Which means if I have 2 tasks that execute concurrently and look like A: while (true) { await callBigFunction(); } B: while (true) { await callSmallFunction(); } then with grpc I get more calls for B and without Cap'N'Proto I get the same for both (correct me if I'm wrong). I don't think it's a huge disadvantage in practice, because if you have large data chunks you should probably design your API different - and if you want realtime behavior then probably both protocols are not optimal. But it's still something to keep in mind.
- kentonv 10y ago
  You can achieve what you describe under my scheme by counting the total size of the parameters of all calls in flight rather than just counting the calls. This is what sophisticated systems are doing in practice and is what I plan to do when incorporating the pattern into the library.
  - Matthias247 10y ago
    The described approach sounds very reasonable! Besides building more support in the library it would maybe also be very worthwhile to have these patterns described together with examples on the website. E.g. how to achieve callback interfaces which don't user another connection, server->client streaming, etc. For me as a potential user information about these kind of features would make Cap'n'Proto immediately look more interesting. When I look at the website I only see Promise Pipelining described as a main feature - which is novel for sure, but not something that attracts my interest if I look for the other features we discussed about.

by kentonv 10y ago

Matthias247 10y ago

Thanks for the explanation. If it works like you describe it seems to be a reasonable way to obtain streaming behavior. Will take a look at it if I find some time. Regarding backpressure and HTTP/2: You get some different kind of backpressure behavior with your approach (application level flow control) and the grpc approach (transport level flow control). Let's say you have 2 functions which are called in parallel. For one the arguments are very big (let's say 100kB), for the other one they are small (some bytes). With HTTP/2 and transport level flow control the small function could get the bandwidth as the big one, which means more requests/s for the small function. With pure application level flow control the small function needs to wait until the big one is fully sent before it can be put on the wire. Which means if I have 2 tasks that execute concurrently and look like A: while (true) { await callBigFunction(); } B: while (true) { await callSmallFunction(); } then with grpc I get more calls for B and without Cap'N'Proto I get the same for both (correct me if I'm wrong). I don't think it's a huge disadvantage in practice, because if you have large data chunks you should probably design your API different - and if you want realtime behavior then probably both protocols are not optimal. But it's still something to keep in mind.

kentonv 10y ago
You can achieve what you describe under my scheme by counting the total size of the parameters of all calls in flight rather than just counting the calls. This is what sophisticated systems are doing in practice and is what I plan to do when incorporating the pattern into the library.
- Matthias247 10y ago
  The described approach sounds very reasonable! Besides building more support in the library it would maybe also be very worthwhile to have these patterns described together with examples on the website. E.g. how to achieve callback interfaces which don't user another connection, server->client streaming, etc. For me as a potential user information about these kind of features would make Cap'n'Proto immediately look more interesting. When I look at the website I only see Promise Pipelining described as a main feature - which is novel for sure, but not something that attracts my interest if I look for the other features we discussed about.