10 ms·

Cap'n Proto 0.8: Streaming flow control, HTTP-over-RPC, fibers

259 points by kentonv 7y ago

subhobroto 7y ago
Kenton, THANK YOU for not only designing Cap’n Proto but also continuing to work on it. Before today, I had slight concern that in the face of Sandstorm, Cap’n Proto would fall in priority. Perhaps Cloudflare Workers are now making a great case for it? My usecase has been Cap’n Proto w/ Python. Being able to do RPC is icing on the cake. I will be looking into streaming fields of entities (classes) soon - the idea being decentralized microservices have local knowledge on how to transform and output a field and other modules/microservices just reach out to them and ask for the same. A silly question: do you forsee any issues getting Cap’n Proto 0.8 to work over a completely locked down Docker environment that only allows HTTPS proxy as the connection between nodes that use Cap’n Proto RPC? PS: Thoughts on why Cap’n Proto did not win over MessagePack? MessagePack had a better JS implementation? Is it the convenience of not needing to define a schema when using MessagePack vs Cap’n Proto schema requirement?
- anderspitman 7y ago
  Not sure a straight comparison to MessagePack is really fair, since capnproto provides a lot more functionality. Unless you know what it's gaining you the syntax probably comes across as more complicated than necessary.
- kentonv 7y ago
  > A silly question: do you forsee any issues getting Cap’n Proto 0.8 to work over a completely locked down Docker environment that only allows HTTPS proxy as the connection between nodes that use Cap’n Proto RPC? If it supports WebSocket, it should be relatively easy to layer Cap'n Proto RPC on top of that. Alternatively, some proxies allow full-duplex HTTP (request and response bodies streaming simultaneously), which could also be enough to bootstrap a connection on top of -- but in practice that tends to run into a lot of problems. Otherwise, that's tough. HTTP is fundamentally a one-way, FIFO request-response protocol, whereas Cap'n Proto is multi-directional and asynchronous. Starting a separate HTTP connection for each call -- with connections initiated in both directions -- would be pretty ugly and have lots of issues with synchronization and routing. > PS: Thoughts on why Cap’n Proto did not win over MessagePack? MessagePack had a better JS implementation? Is it the convenience of not needing to define a schema when using MessagePack vs Cap’n Proto schema requirement? I don't really consider MessagePack a direct competitor to Cap'n Proto. It's more of a competitor to JSON. Schema-driven vs. non-schema-driven changes everything about how you use a serialization. A more apples-to-apples comparison in Protobuf. Protobuf is much more popular for a simple reason: It has had a lot more engineering investment, leading to mature implementations in more languages and lots of great tooling that capnp doesn't have (yet). No amount of clever design can beat that.
  - anderspitman 7y ago
    > If it supports WebSocket, it should be relatively easy to layer Cap'n Proto RPC on top of that. How would the new streaming functionality be implemented over WebSockets? WS has no flow control. You can check bufferedAmount but I found it to be fairly useless[0]. Maybe it's improved in the last 1.5 years or I was using it wrong. > HTTP is fundamentally a one-way, FIFO request-response protocol, whereas Cap'n Proto is multi-directional and asynchronous. Starting a separate HTTP connection for each call -- with connections initiated in both directions -- would be pretty ugly and have lots of issues with synchronization and routing. I think you could get a long way using HTTP/2 and server-sent events. [0]: https://github.com/websockets/ws/issues/492 https://github.com/websockets/ws/issues/492 EDIT: added link to bufferedAmount issue
    - kentonv 7y ago
      > How would the new streaming functionality be implemented over WebSockets? WS has no flow control. Sure it does. WS is just a framing protocol on top of a regular TCP connection. Though it sounds like you're not talking about the protocol so much as the JavaScript API, which perhaps doesn't give you enough visibility into the underlying TCP socket state. But a BBR-like algorithm could still work. Basically (massively oversimplified): 1) Determine the connection latency based on the fastest response you've seen. 2) Determine the connection throughput by tracking the highest throughput you've ever seen. 3) Set your window size to be a little bit more than latency * throughput. This "should" saturate the connection with just a little bit of buffering. If it doesn't saturate the connection, then your measured throughput will go up until it does. This way there is no need to ask the OS or browser to tell you how much is buffered... Disclaimer: I have yet to actually implement something like this. Obviously, it gets tricky in the details.
      - anderspitman 7y ago
        > Sure it does. WS is just a framing protocol on top of a regular TCP connection. Though it sounds like you're not talking about the protocol so much as the JavaScript API, which perhaps doesn't give you enough visibility into the underlying TCP socket state. Every WS implementation I've ever seen is nonblocking on both sides. Are you aware of any that aren't? I'm not even sure the spec allows for that. But yes you are correct that you can implement more advanced algos on top. EDIT: Sounds like blocking implementations do exist, but unfortunately I'm constrained to a browser environment, and all the browsers are nonblocking.
anderspitman 7y ago
This is awesome. The new streaming stuff is particularly interesting to me, and it's very impressive that you managed to implement it with no protocol changes. I have a couple questions. Please forgive any misconceptions as I've never used capnproto myself, since all the streaming I've done has to work in the browser, and as far as I know capnproto doesn't work over WebSocket or WebRTC transports. But I've long been impressed with and inspired by capnproto and sandstorm. Main question: is there a reason you opted for traditional window flow control a la TCP, as opposed to "request-N" style like in reactive streams[0] (see rsocket[1] for a great implementation)? So with request-N, a server->client stream would look something like this: interface MyInterface { streamingCall @0 (callback :Callback) -> (requester :Requester); interface Callback { sendChunk @0 (chunk :Data) -> (); } interface Requester { request @0 (n :int) -> (); } } And the server will only ever send as much data as has been requested by the client with requester calls. This results in really elegant flow control that takes into account both the network, and the client's capacity to consume, without the necessity of tracking windows. The receiver simply calls request(1) each time it processes a message. If you want a buffer you can just start with an assumed N=10, 100 etc. I've found this worked really well when implementing omnistreams[2], which is basically a very thin streaming/multiplexing layer for WebSockets, since WS doesn't have any flow control. (fibridge[3] is a good example of it in action). I started with a window-style but once I learned about reactive streams the request model was much easier to reason about for me. [0]: https://github.com/reactive-streams/reactive-streams-jvm https://github.com/reactive-streams/reactive-streams-jvm [1]: https://github.com/rsocket/rsocket https://github.com/rsocket/rsocket [2]: https://github.com/omnistreams/omnistreams-spec https://github.com/omnistreams/omnistreams-spec [3]: http://iobio.io/2019/06/12/introducing-fibridge/ http://iobio.io/2019/06/12/introducing-fibridge/
- kentonv 7y ago
  Hmm, to me, what you describe still sounds window-based, it's just that the receiver chooses the window size. The question then is: how does the receiver decide on a good size? If it chooses a window that is too small, it won't fully utilize the available bandwidth. If it chooses one too big, it'll create queuing delay. This is a very hard question to answer and many academic papers have been written on the subject. But the strategies I thought about seemed easy enough to compute on the sender side, and the sender is the one that ultimately needs to know the window size in order to decide when to send more data. But I can totally imagine that there are applications where the receiver knows better how much data it wants to request at a time. You can, of course, use a pattern like you suggest to accomplish that, without any help from the RPC system. Regarding WebSockets, you could totally make Cap'n Proto RPC run over WebSocket. It wouldn't even be much work to hook up the C++ RPC implementation to KJ's HTTP library which supports WebSocket. The harder problem is that there isn't currently a JavaScript implementation of capnp RPC... :/
malkia 7y ago
I'm using grpc/protobuf for the fact that it supports plenty of languages: C++, C#, Python, Dart, even Rust and others (Node, PHP, etc.). What is the state of CapnProto when comes to this? I've tried to look at the C# project, but the page was missing on github.
- kentonv 7y ago
  Admittedly, this is a huge weakness of Cap'n Proto. The C++ implementation (which is the one I use and work on personally) is mature. There are pretty solid implementations in Rust and Go, too. But it falls off after that, with most implementations being serialization-only and at various levels of (im)maturity. There's not a lot I can do about this. Cap'n Proto adoption doesn't directly drive revenue for anyone in particular, so I can't hire an army of engineers to throw at it... People who want better Cap'n Proto support in each language need to step up to help make it happen. One thing I am looking at doing is making it easier for per-language serialization implementations to bind to the C++ RPC implementation. This might make a lot of sense, since the serialization implementations have wide APIs but shallow implementation details, while the RPC implementation is a pretty narrow API with very complex implementation. And it turns out Cap'n Proto messages are super-easy to pass between languages since the in-memory format is by design the same across languages -- passing around byte buffers tends to be pretty easy.
  - anderspitman 7y ago
    > One thing I am looking at doing is making it easier for per-language serialization implementations to bind to the C++ RPC implementation Depending on how much more mature the C++ implementation is, you might consider using the Rust version for this instead. I've toyed around with exposing Rust to C (for a TCP-based message protocol[0], as chance would have it), and it worked pretty great. [0]: https://github.com/anderspitman/messend-rs https://github.com/anderspitman/messend-rs
    - kentonv 7y ago
      While the Rust implementation of Cap'n Proto is one of the better ones, it's still received only a tiny fraction of the engineering investment that the C++ implementation has.
omginternets 7y ago
Does anybody know if the Go implementation is still maintained? The author is unresponsive and there haveNot been any updates in quite a while. https://github.com/capnproto/go-capnproto2 https://github.com/capnproto/go-capnproto2
- anderspitman 7y ago
  There was a commit yesterday? But it does seem to have slowed down.
  - omginternets 7y ago
    Go figure, just as soon as I post my concern to HN! Oh well, here’s to hoping development picks up again :) Edit: unfortunately the commit seems to be to a funding.yml. While I really can’t afford to contribute myself, I hope someone does.
kortex 7y ago
I just want to say, I've been following Capn for a few years now, excellent work, keep it up! I feel it's a better architecture overall than protobuf and wish I had more time to contribute to the project.
- kentonv 7y ago
  Thanks!
xyz-x 7y ago
For me, new to cap'n'proto, this blog post doesn't cut the mustard because of multiple red flags: - to start, it's self-congratulatory at stating that streaming already exists "[via] promise pipelining" — but that has a name, it's called polling, not streaming. Making asynchronicity explicit doesn't make a protocol streaming. - in the same paragraph an "object-capability model" is introduced as a concept, but not explained - second paragraph: "think of this like providing a callback function in an object-oriented language", when it should be "in a functional programming language" (callbacks aren't OOP, they are per definition functional programming) - second paragraph, vocab: what's a "temporary RPC object"? Contrary to the precise, albeit unexplained vocabulary, in the first pragraph, this is vague. - creating examples with "MyInterface" being the service shows a lack of creativity and a rather low ability to communicate; is this service on the server side, or the client side? Noone knows, and "Callback" is not a good name for a callback, it should be "SendEmailWithData" or something that makes sense. - `sendChunk @0 (chunk :Data)` doesn't make sense to a beginner without explaination; what's `@0` and why do I care? Here's what made me write this comment despite the threshold annoyance in commenting: - Why name a message, `Data` when it's clearly NOT a chunk of data in layer-3/layer-4, but rather a layer-7 artifact with retries and checksumming implemented to ensure complete and accurate message delivery? Finally, the articles goes on and discusses control flow via a proxy variable; your OS'es TCP send buffer size. But the linked Wikipedia article states: > because the protocol can only achieve optimum throughput if a sender sends a sufficiently large quantity of data before being required to stop and wait until a confirming message is received from the receiver, acknowledging successful receipt of that data Which is not the case for Cap'n'Proto (admittedly it states it uses a hack). And there's no discussion of end-to-end problems like BufferBloat, which are very hard to solve by only looking at your own buffer https://en.wikipedia.org/wiki/Bufferbloat#Solutions_and_mitigations https://en.wikipedia.org/wiki/Bufferbloat#Solutions_and_miti... — or even the semantics of "blocking on server's return value" (Is it enough for the receiving process to have the message in memory? The type system showcased seems to tell that story) The article also doesn't state how a simple RPC call works. Going to https://capnproto.org/rpc.html https://capnproto.org/rpc.html immediately puts me off by inventing "time travel", calling it "promise pipelining" and showing an impossible trace diagram (you can't have messages go backwards in time). But when explaining it, it's really RPC message coalescing and compile-time reference indirection, from the promise to the underlying object instance as it is after executing the coalesced message pipeline. However, even when using the example of a file system (which is about as exception-intense as you can imagine), exceptions are ignored. Looking through the Calculator example (https://github.com/capnproto/capnproto/blob/master/c++/samples/calculator-client.c++ https://github.com/capnproto/capnproto/blob/master/c++/sampl...) it turns out they haven't actually performed the compile-time indirection, but actually block the calling thread like any random do-it-yourself-RPC framework out there. What a RPC framework should do is give you: - an extremely clear serialisation model that is outside of the framework - a clear API - clear garantuees / invariants on how it manages the complexities of network programming In short: it must be very clear in what it promises. Cap'n'proto is not.
_pmf_ 7y ago
I'd really like to read an experience report of using the promise pipelined / time-travelling RPC mechanism in production.
malkia 7y ago
Kenton, have you looked into what Fuchsia is doing with FIDL? https://fuchsia.googlesource.com/docs/+/ea2fce2874556205204d3ef70c60e25074dc7ffd/development/languages/fidl/tutorial.md https://fuchsia.googlesource.com/docs/+/ea2fce2874556205204d... (not sure if recent page, but good enough) Just wondering about your opinion... Thanks!
- kentonv 7y ago
  I've heard it mentioned but haven't had the chance to look closely.
IshKebab 7y ago
Impressive work! I just wish it had a less awkward API for serialisation and deserialisation. Compare the Rust example for Capnp with Protobuf: https://github.com/capnproto/capnproto-rust/blob/master/example/addressbook_send/addressbook_send.rs https://github.com/capnproto/capnproto-rust/blob/master/exam... https://docs.rs/prost/0.6.1/prost/trait.Message.html#method.encode https://docs.rs/prost/0.6.1/prost/trait.Message.html#method.... (Ok I couldn't actually find an example for Prost because all you do is create a normal Rust `struct` and call `encode()` on it.)
- zenhack 7y ago
  Yeah, this is kinda the cost of not having an encode/decode step. The Haskell implementation (of which I am the primary author) provides a higher-level API with "normal" data types for cases where performance requirements aren't stringent enough to merit the extra burden on the developer. I'm mostly interested in RPC, so I rarely use the low-level API myself...
beagle3 7y ago
kenton, I admire your work on Cap'n Proto and Sandstorm. Question about time traveling promises - I read the documentation, and it sounds to me like - taken to the logical extreme, that would effectively mean an interpreter - because at some point, you don't just want to use return values from one call as parameters in the other - you'd also want e.g. to build a one-network-roundtrip "create-if-not-exists" call from "exists(name)" and "create(name)", which would require the second call's activation to be dependent on the first call's result (rather than its parameters) - which is just a small change. But if you consider that, and error handling, and a few other relatively simple cases, you quickly end up with an informally specified, bug ridden, incomplete implementation of Emacs Lisp inside the RPC implementation. So, my question is - where do you draw the line, and how do you decide to draw it? I don't believe there's a right answer, but I wonder about your philosophy.
- kentonv 7y ago
  Indeed, the comments allude at this possibility (note the TODO): https://github.com/capnproto/capnproto/blob/77f20b4652e51b5a7ebda414e979e059a6c7c27c/c++/src/capnp/rpc.capnp#L1050-L1093 https://github.com/capnproto/capnproto/blob/77f20b4652e51b5a... But in practice, the only kind of "script" we support currently is following a chain of named fields followed by invoking a new RPC method, unconditionally, as in: fooResult = cap.foo(); quxResult = fooResult.bar.baz.qux(); This seems to satisfy the vast majority of real-world use cases. I would only add other operations if I identified some use case where it turns out to be a really big performance win. So far I haven't seen any.

259 points by kentonv 7y ago

Kenton, THANK YOU for not only designing Cap’n Proto but also continuing to work on it. Before today, I had slight concern that in the face of Sandstorm, Cap’n Proto would fall in priority. Perhaps Cloudflare Workers are now making a great case for it? My usecase has been Cap’n Proto w/ Python. Being able to do RPC is icing on the cake. I will be looking into streaming fields of entities (classes) soon - the idea being decentralized microservices have local knowledge on how to transform and output a field and other modules/microservices just reach out to them and ask for the same. A silly question: do you forsee any issues getting Cap’n Proto 0.8 to work over a completely locked down Docker environment that only allows HTTPS proxy as the connection between nodes that use Cap’n Proto RPC? PS: Thoughts on why Cap’n Proto did not win over MessagePack? MessagePack had a better JS implementation? Is it the convenience of not needing to define a schema when using MessagePack vs Cap’n Proto schema requirement?

anderspitman 7y ago
Not sure a straight comparison to MessagePack is really fair, since capnproto provides a lot more functionality. Unless you know what it's gaining you the syntax probably comes across as more complicated than necessary.
kentonv 7y ago
> A silly question: do you forsee any issues getting Cap’n Proto 0.8 to work over a completely locked down Docker environment that only allows HTTPS proxy as the connection between nodes that use Cap’n Proto RPC? If it supports WebSocket, it should be relatively easy to layer Cap'n Proto RPC on top of that. Alternatively, some proxies allow full-duplex HTTP (request and response bodies streaming simultaneously), which could also be enough to bootstrap a connection on top of -- but in practice that tends to run into a lot of problems. Otherwise, that's tough. HTTP is fundamentally a one-way, FIFO request-response protocol, whereas Cap'n Proto is multi-directional and asynchronous. Starting a separate HTTP connection for each call -- with connections initiated in both directions -- would be pretty ugly and have lots of issues with synchronization and routing. > PS: Thoughts on why Cap’n Proto did not win over MessagePack? MessagePack had a better JS implementation? Is it the convenience of not needing to define a schema when using MessagePack vs Cap’n Proto schema requirement? I don't really consider MessagePack a direct competitor to Cap'n Proto. It's more of a competitor to JSON. Schema-driven vs. non-schema-driven changes everything about how you use a serialization. A more apples-to-apples comparison in Protobuf. Protobuf is much more popular for a simple reason: It has had a lot more engineering investment, leading to mature implementations in more languages and lots of great tooling that capnp doesn't have (yet). No amount of clever design can beat that.
- anderspitman 7y ago
  > If it supports WebSocket, it should be relatively easy to layer Cap'n Proto RPC on top of that. How would the new streaming functionality be implemented over WebSockets? WS has no flow control. You can check bufferedAmount but I found it to be fairly useless[0]. Maybe it's improved in the last 1.5 years or I was using it wrong. > HTTP is fundamentally a one-way, FIFO request-response protocol, whereas Cap'n Proto is multi-directional and asynchronous. Starting a separate HTTP connection for each call -- with connections initiated in both directions -- would be pretty ugly and have lots of issues with synchronization and routing. I think you could get a long way using HTTP/2 and server-sent events. [0]: https://github.com/websockets/ws/issues/492 https://github.com/websockets/ws/issues/492 EDIT: added link to bufferedAmount issue
  - kentonv 7y ago
    > How would the new streaming functionality be implemented over WebSockets? WS has no flow control. Sure it does. WS is just a framing protocol on top of a regular TCP connection. Though it sounds like you're not talking about the protocol so much as the JavaScript API, which perhaps doesn't give you enough visibility into the underlying TCP socket state. But a BBR-like algorithm could still work. Basically (massively oversimplified): 1) Determine the connection latency based on the fastest response you've seen. 2) Determine the connection throughput by tracking the highest throughput you've ever seen. 3) Set your window size to be a little bit more than latency * throughput. This "should" saturate the connection with just a little bit of buffering. If it doesn't saturate the connection, then your measured throughput will go up until it does. This way there is no need to ask the OS or browser to tell you how much is buffered... Disclaimer: I have yet to actually implement something like this. Obviously, it gets tricky in the details.
    - anderspitman 7y ago
      > Sure it does. WS is just a framing protocol on top of a regular TCP connection. Though it sounds like you're not talking about the protocol so much as the JavaScript API, which perhaps doesn't give you enough visibility into the underlying TCP socket state. Every WS implementation I've ever seen is nonblocking on both sides. Are you aware of any that aren't? I'm not even sure the spec allows for that. But yes you are correct that you can implement more advanced algos on top. EDIT: Sounds like blocking implementations do exist, but unfortunately I'm constrained to a browser environment, and all the browsers are nonblocking.

anderspitman 7y ago

This is awesome. The new streaming stuff is particularly interesting to me, and it's very impressive that you managed to implement it with no protocol changes. I have a couple questions. Please forgive any misconceptions as I've never used capnproto myself, since all the streaming I've done has to work in the browser, and as far as I know capnproto doesn't work over WebSocket or WebRTC transports. But I've long been impressed with and inspired by capnproto and sandstorm. Main question: is there a reason you opted for traditional window flow control a la TCP, as opposed to "request-N" style like in reactive streams[0] (see rsocket[1] for a great implementation)? So with request-N, a server->client stream would look something like this: interface MyInterface { streamingCall @0 (callback :Callback) -> (requester :Requester); interface Callback { sendChunk @0 (chunk :Data) -> (); } interface Requester { request @0 (n :int) -> (); } } And the server will only ever send as much data as has been requested by the client with requester calls. This results in really elegant flow control that takes into account both the network, and the client's capacity to consume, without the necessity of tracking windows. The receiver simply calls request(1) each time it processes a message. If you want a buffer you can just start with an assumed N=10, 100 etc. I've found this worked really well when implementing omnistreams[2], which is basically a very thin streaming/multiplexing layer for WebSockets, since WS doesn't have any flow control. (fibridge[3] is a good example of it in action). I started with a window-style but once I learned about reactive streams the request model was much easier to reason about for me. [0]: https://github.com/reactive-streams/reactive-streams-jvm https://github.com/reactive-streams/reactive-streams-jvm [1]: https://github.com/rsocket/rsocket https://github.com/rsocket/rsocket [2]: https://github.com/omnistreams/omnistreams-spec https://github.com/omnistreams/omnistreams-spec [3]: http://iobio.io/2019/06/12/introducing-fibridge/ http://iobio.io/2019/06/12/introducing-fibridge/

kentonv 7y ago
Hmm, to me, what you describe still sounds window-based, it's just that the receiver chooses the window size. The question then is: how does the receiver decide on a good size? If it chooses a window that is too small, it won't fully utilize the available bandwidth. If it chooses one too big, it'll create queuing delay. This is a very hard question to answer and many academic papers have been written on the subject. But the strategies I thought about seemed easy enough to compute on the sender side, and the sender is the one that ultimately needs to know the window size in order to decide when to send more data. But I can totally imagine that there are applications where the receiver knows better how much data it wants to request at a time. You can, of course, use a pattern like you suggest to accomplish that, without any help from the RPC system. Regarding WebSockets, you could totally make Cap'n Proto RPC run over WebSocket. It wouldn't even be much work to hook up the C++ RPC implementation to KJ's HTTP library which supports WebSocket. The harder problem is that there isn't currently a JavaScript implementation of capnp RPC... :/

malkia 7y ago

I'm using grpc/protobuf for the fact that it supports plenty of languages: C++, C#, Python, Dart, even Rust and others (Node, PHP, etc.). What is the state of CapnProto when comes to this? I've tried to look at the C# project, but the page was missing on github.

kentonv 7y ago
Admittedly, this is a huge weakness of Cap'n Proto. The C++ implementation (which is the one I use and work on personally) is mature. There are pretty solid implementations in Rust and Go, too. But it falls off after that, with most implementations being serialization-only and at various levels of (im)maturity. There's not a lot I can do about this. Cap'n Proto adoption doesn't directly drive revenue for anyone in particular, so I can't hire an army of engineers to throw at it... People who want better Cap'n Proto support in each language need to step up to help make it happen. One thing I am looking at doing is making it easier for per-language serialization implementations to bind to the C++ RPC implementation. This might make a lot of sense, since the serialization implementations have wide APIs but shallow implementation details, while the RPC implementation is a pretty narrow API with very complex implementation. And it turns out Cap'n Proto messages are super-easy to pass between languages since the in-memory format is by design the same across languages -- passing around byte buffers tends to be pretty easy.
- anderspitman 7y ago
  > One thing I am looking at doing is making it easier for per-language serialization implementations to bind to the C++ RPC implementation Depending on how much more mature the C++ implementation is, you might consider using the Rust version for this instead. I've toyed around with exposing Rust to C (for a TCP-based message protocol[0], as chance would have it), and it worked pretty great. [0]: https://github.com/anderspitman/messend-rs https://github.com/anderspitman/messend-rs
  - kentonv 7y ago
    While the Rust implementation of Cap'n Proto is one of the better ones, it's still received only a tiny fraction of the engineering investment that the C++ implementation has.

omginternets 7y ago

Does anybody know if the Go implementation is still maintained? The author is unresponsive and there haveNot been any updates in quite a while. https://github.com/capnproto/go-capnproto2 https://github.com/capnproto/go-capnproto2

anderspitman 7y ago
There was a commit yesterday? But it does seem to have slowed down.
- omginternets 7y ago
  Go figure, just as soon as I post my concern to HN! Oh well, here’s to hoping development picks up again :) Edit: unfortunately the commit seems to be to a funding.yml. While I really can’t afford to contribute myself, I hope someone does.

kortex 7y ago

I just want to say, I've been following Capn for a few years now, excellent work, keep it up! I feel it's a better architecture overall than protobuf and wish I had more time to contribute to the project.

kentonv 7y ago
Thanks!

xyz-x 7y ago

For me, new to cap'n'proto, this blog post doesn't cut the mustard because of multiple red flags: - to start, it's self-congratulatory at stating that streaming already exists "[via] promise pipelining" — but that has a name, it's called polling, not streaming. Making asynchronicity explicit doesn't make a protocol streaming. - in the same paragraph an "object-capability model" is introduced as a concept, but not explained - second paragraph: "think of this like providing a callback function in an object-oriented language", when it should be "in a functional programming language" (callbacks aren't OOP, they are per definition functional programming) - second paragraph, vocab: what's a "temporary RPC object"? Contrary to the precise, albeit unexplained vocabulary, in the first pragraph, this is vague. - creating examples with "MyInterface" being the service shows a lack of creativity and a rather low ability to communicate; is this service on the server side, or the client side? Noone knows, and "Callback" is not a good name for a callback, it should be "SendEmailWithData" or something that makes sense. - `sendChunk @0 (chunk :Data)` doesn't make sense to a beginner without explaination; what's `@0` and why do I care? Here's what made me write this comment despite the threshold annoyance in commenting: - Why name a message, `Data` when it's clearly NOT a chunk of data in layer-3/layer-4, but rather a layer-7 artifact with retries and checksumming implemented to ensure complete and accurate message delivery? Finally, the articles goes on and discusses control flow via a proxy variable; your OS'es TCP send buffer size. But the linked Wikipedia article states: > because the protocol can only achieve optimum throughput if a sender sends a sufficiently large quantity of data before being required to stop and wait until a confirming message is received from the receiver, acknowledging successful receipt of that data Which is not the case for Cap'n'Proto (admittedly it states it uses a hack). And there's no discussion of end-to-end problems like BufferBloat, which are very hard to solve by only looking at your own buffer https://en.wikipedia.org/wiki/Bufferbloat#Solutions_and_mitigations https://en.wikipedia.org/wiki/Bufferbloat#Solutions_and_miti... — or even the semantics of "blocking on server's return value" (Is it enough for the receiving process to have the message in memory? The type system showcased seems to tell that story) The article also doesn't state how a simple RPC call works. Going to https://capnproto.org/rpc.html https://capnproto.org/rpc.html immediately puts me off by inventing "time travel", calling it "promise pipelining" and showing an impossible trace diagram (you can't have messages go backwards in time). But when explaining it, it's really RPC message coalescing and compile-time reference indirection, from the promise to the underlying object instance as it is after executing the coalesced message pipeline. However, even when using the example of a file system (which is about as exception-intense as you can imagine), exceptions are ignored. Looking through the Calculator example (https://github.com/capnproto/capnproto/blob/master/c++/samples/calculator-client.c++ https://github.com/capnproto/capnproto/blob/master/c++/sampl...) it turns out they haven't actually performed the compile-time indirection, but actually block the calling thread like any random do-it-yourself-RPC framework out there. What a RPC framework should do is give you: - an extremely clear serialisation model that is outside of the framework - a clear API - clear garantuees / invariants on how it manages the complexities of network programming In short: it must be very clear in what it promises. Cap'n'proto is not.

_pmf_ 7y ago

I'd really like to read an experience report of using the promise pipelined / time-travelling RPC mechanism in production.

malkia 7y ago

Kenton, have you looked into what Fuchsia is doing with FIDL? https://fuchsia.googlesource.com/docs/+/ea2fce2874556205204d3ef70c60e25074dc7ffd/development/languages/fidl/tutorial.md https://fuchsia.googlesource.com/docs/+/ea2fce2874556205204d... (not sure if recent page, but good enough) Just wondering about your opinion... Thanks!

kentonv 7y ago
I've heard it mentioned but haven't had the chance to look closely.

IshKebab 7y ago

Impressive work! I just wish it had a less awkward API for serialisation and deserialisation. Compare the Rust example for Capnp with Protobuf: https://github.com/capnproto/capnproto-rust/blob/master/example/addressbook_send/addressbook_send.rs https://github.com/capnproto/capnproto-rust/blob/master/exam... https://docs.rs/prost/0.6.1/prost/trait.Message.html#method.encode https://docs.rs/prost/0.6.1/prost/trait.Message.html#method.... (Ok I couldn't actually find an example for Prost because all you do is create a normal Rust `struct` and call `encode()` on it.)

zenhack 7y ago
Yeah, this is kinda the cost of not having an encode/decode step. The Haskell implementation (of which I am the primary author) provides a higher-level API with "normal" data types for cases where performance requirements aren't stringent enough to merit the extra burden on the developer. I'm mostly interested in RPC, so I rarely use the low-level API myself...

beagle3 7y ago

kenton, I admire your work on Cap'n Proto and Sandstorm. Question about time traveling promises - I read the documentation, and it sounds to me like - taken to the logical extreme, that would effectively mean an interpreter - because at some point, you don't just want to use return values from one call as parameters in the other - you'd also want e.g. to build a one-network-roundtrip "create-if-not-exists" call from "exists(name)" and "create(name)", which would require the second call's activation to be dependent on the first call's result (rather than its parameters) - which is just a small change. But if you consider that, and error handling, and a few other relatively simple cases, you quickly end up with an informally specified, bug ridden, incomplete implementation of Emacs Lisp inside the RPC implementation. So, my question is - where do you draw the line, and how do you decide to draw it? I don't believe there's a right answer, but I wonder about your philosophy.

kentonv 7y ago
Indeed, the comments allude at this possibility (note the TODO): https://github.com/capnproto/capnproto/blob/77f20b4652e51b5a7ebda414e979e059a6c7c27c/c++/src/capnp/rpc.capnp#L1050-L1093 https://github.com/capnproto/capnproto/blob/77f20b4652e51b5a... But in practice, the only kind of "script" we support currently is following a chain of named fields followed by invoking a new RPC method, unconditionally, as in: fooResult = cap.foo(); quxResult = fooResult.bar.baz.qux(); This seems to satisfy the vast majority of real-world use cases. I would only add other operations if I identified some use case where it turns out to be a really big performance win. So far I haven't seen any.