5 ms·

> If it supports WebSocket, it should be relatively easy to layer Cap'n Proto RPC on top of that. How would the new streaming functionality be implemented over

> If it supports WebSocket, it should be relatively easy to layer Cap'n Proto RPC on top of that. How would the new streaming functionality be implemented over WebSockets? WS has no flow control. You can check bufferedAmount but I found it to be fairly useless[0]. Maybe it's improved in the last 1.5 years or I was using it wrong. > HTTP is fundamentally a one-way, FIFO request-response protocol, whereas Cap'n Proto is multi-directional and asynchronous. Starting a separate HTTP connection for each call -- with connections initiated in both directions -- would be pretty ugly and have lots of issues with synchronization and routing. I think you could get a long way using HTTP/2 and server-sent events. [0]: https://github.com/websockets/ws/issues/492 https://github.com/websockets/ws/issues/492 EDIT: added link to bufferedAmount issue

kentonv 7y ago
> How would the new streaming functionality be implemented over WebSockets? WS has no flow control. Sure it does. WS is just a framing protocol on top of a regular TCP connection. Though it sounds like you're not talking about the protocol so much as the JavaScript API, which perhaps doesn't give you enough visibility into the underlying TCP socket state. But a BBR-like algorithm could still work. Basically (massively oversimplified): 1) Determine the connection latency based on the fastest response you've seen. 2) Determine the connection throughput by tracking the highest throughput you've ever seen. 3) Set your window size to be a little bit more than latency * throughput. This "should" saturate the connection with just a little bit of buffering. If it doesn't saturate the connection, then your measured throughput will go up until it does. This way there is no need to ask the OS or browser to tell you how much is buffered... Disclaimer: I have yet to actually implement something like this. Obviously, it gets tricky in the details.
- anderspitman 7y ago
  > Sure it does. WS is just a framing protocol on top of a regular TCP connection. Though it sounds like you're not talking about the protocol so much as the JavaScript API, which perhaps doesn't give you enough visibility into the underlying TCP socket state. Every WS implementation I've ever seen is nonblocking on both sides. Are you aware of any that aren't? I'm not even sure the spec allows for that. But yes you are correct that you can implement more advanced algos on top. EDIT: Sounds like blocking implementations do exist, but unfortunately I'm constrained to a browser environment, and all the browsers are nonblocking.
  - zenhack 7y ago
    off the top of my head https://godoc.org/golang.org/x/net/websocket https://godoc.org/golang.org/x/net/websocket has a synchronous read/write api
    - anderspitman 7y ago
      Does that block until messages are actually transferred, or just shove them in a golang buffer and return? Now that you mention it I feel like maybe I have seen blocking implementations in golang and Python a while back, which would make sense.
      - zenhack 7y ago
        Skimming the implementation, there is buffering, but it looks like it's bounded, so tight looping Write() you should still end up blocking on the OS at some point.
        anderspitman 7y ago
        Makes sense. Thanks for correcting my misconception!
  - kentonv 7y ago
    Heh. The WebSocket implementation I wrote for KJ does in fact provide backpressure (the send() method returns a promise that resolves when it's a good time to send the next message). I guess I'm surprised to hear that most don't...
    - anderspitman 7y ago
      I think it's more likely I've just been too deep in JS-land. Does KJ send() do any internal buffering, or wait for the OS to tell it to send more?
      - kentonv 7y ago
brabel 7y ago
> HTTP is fundamentally a one-way, FIFO request-response protocol... I have to disagree partially with this... HTTP allows a server to start streaming a response back as soon as it has received all header fields from the client, and since HTTP/1.1, it can stream back an infinite sequence of chunks, which the client can be doing at the same time - effectively resulting in a two-way communication channel as long as the underlying TCP connection allows that (which normally is true, unless you run into bad proxy implementations in the middle). Server-sent events is a incomplete implementation of this idea. I have used this in the past , before websockets, and I fail to see the advantage of using websockets over this.
- kentonv 7y ago
  You're quoting me from two posts up, but I also specifically called out full-duplex HTTP in that post. HTTP/1.1 does indeed support full duplex. However, most proxies don't. E.g. if you put nginx in front of your application server, you might lose the ability to do full duplex streaming right there. Or, if your client is behind a normal forward proxy, they might not be able to get full duplex. But, it can be pretty hard to figure out what's wrong. So, in practice I don't recommend relying on HTTP/1.1 full duplex. To be honest, I don't recommend HTTP/2 full duplex either, because HTTP/2 is just an optimization applying to individual network hops, and you still have the same proxy software in between screwing things up (and maybe even downgrading the request to HTTP/1.1 for the next hop). Personally I recommend using WebSocket when you expect full duplex. That way, all intervening proxy servers either know that you are expecting full duplex (because they recognize WebSocket), or connection setup will fail fast.
  - brabel 7y ago
    I realized I replied to the wrong comment :) but I had some issue with the suggestion regarding server-sent events... anyway, thanks for answering, I really appreciate your work and it's nice to hear your input on this.

kentonv 7y ago

> How would the new streaming functionality be implemented over WebSockets? WS has no flow control. Sure it does. WS is just a framing protocol on top of a regular TCP connection. Though it sounds like you're not talking about the protocol so much as the JavaScript API, which perhaps doesn't give you enough visibility into the underlying TCP socket state. But a BBR-like algorithm could still work. Basically (massively oversimplified): 1) Determine the connection latency based on the fastest response you've seen. 2) Determine the connection throughput by tracking the highest throughput you've ever seen. 3) Set your window size to be a little bit more than latency * throughput. This "should" saturate the connection with just a little bit of buffering. If it doesn't saturate the connection, then your measured throughput will go up until it does. This way there is no need to ask the OS or browser to tell you how much is buffered... Disclaimer: I have yet to actually implement something like this. Obviously, it gets tricky in the details.

anderspitman 7y ago
> Sure it does. WS is just a framing protocol on top of a regular TCP connection. Though it sounds like you're not talking about the protocol so much as the JavaScript API, which perhaps doesn't give you enough visibility into the underlying TCP socket state. Every WS implementation I've ever seen is nonblocking on both sides. Are you aware of any that aren't? I'm not even sure the spec allows for that. But yes you are correct that you can implement more advanced algos on top. EDIT: Sounds like blocking implementations do exist, but unfortunately I'm constrained to a browser environment, and all the browsers are nonblocking.
- zenhack 7y ago
  off the top of my head https://godoc.org/golang.org/x/net/websocket https://godoc.org/golang.org/x/net/websocket has a synchronous read/write api
  - anderspitman 7y ago
    Does that block until messages are actually transferred, or just shove them in a golang buffer and return? Now that you mention it I feel like maybe I have seen blocking implementations in golang and Python a while back, which would make sense.
    - zenhack 7y ago
      Skimming the implementation, there is buffering, but it looks like it's bounded, so tight looping Write() you should still end up blocking on the OS at some point.
      - anderspitman 7y ago
        Makes sense. Thanks for correcting my misconception!
- kentonv 7y ago
  Heh. The WebSocket implementation I wrote for KJ does in fact provide backpressure (the send() method returns a promise that resolves when it's a good time to send the next message). I guess I'm surprised to hear that most don't...
  - anderspitman 7y ago
    I think it's more likely I've just been too deep in JS-land. Does KJ send() do any internal buffering, or wait for the OS to tell it to send more?
    - kentonv 7y ago

brabel 7y ago

> HTTP is fundamentally a one-way, FIFO request-response protocol... I have to disagree partially with this... HTTP allows a server to start streaming a response back as soon as it has received all header fields from the client, and since HTTP/1.1, it can stream back an infinite sequence of chunks, which the client can be doing at the same time - effectively resulting in a two-way communication channel as long as the underlying TCP connection allows that (which normally is true, unless you run into bad proxy implementations in the middle). Server-sent events is a incomplete implementation of this idea. I have used this in the past , before websockets, and I fail to see the advantage of using websockets over this.

kentonv 7y ago
You're quoting me from two posts up, but I also specifically called out full-duplex HTTP in that post. HTTP/1.1 does indeed support full duplex. However, most proxies don't. E.g. if you put nginx in front of your application server, you might lose the ability to do full duplex streaming right there. Or, if your client is behind a normal forward proxy, they might not be able to get full duplex. But, it can be pretty hard to figure out what's wrong. So, in practice I don't recommend relying on HTTP/1.1 full duplex. To be honest, I don't recommend HTTP/2 full duplex either, because HTTP/2 is just an optimization applying to individual network hops, and you still have the same proxy software in between screwing things up (and maybe even downgrading the request to HTTP/1.1 for the next hop). Personally I recommend using WebSocket when you expect full duplex. That way, all intervening proxy servers either know that you are expecting full duplex (because they recognize WebSocket), or connection setup will fail fast.
- brabel 7y ago
  I realized I replied to the wrong comment :) but I had some issue with the suggestion regarding server-sent events... anyway, thanks for answering, I really appreciate your work and it's nice to hear your input on this.