5 ms·
Cool article, thanks for sharing. You don't mention which version of HTTP you were using, which makes some of my comments here conjecture, but if I had to gues
by KingMob 3y ago
Cool article, thanks for sharing.
You don't mention which version of HTTP you were using, which makes some of my comments here conjecture, but if I had to guess, I'd say you were using HTTP/1.1. Is that right?
It would really help if you could break down where the time is being spent. I'd love to know what fraction of the time is spent in processing, transit, querying the db, etc.
Did you consider HTTP/2 or 3? I would expect them to have lower setup overhead, and be more competitive with Websockets.
In particular, if you were using HTTP/2+, did you look into CONNECT proxying? It could replace Websockets while having lower connection setup overhead, albeit at the cost of a little payload unwrapping overhead in the proxy. But theoretically, this could offer both the faster startup of HTTP/2+, combined with the lower processing overhead for the db of a TCP stream. Comparing the graphs of 10 queries, it _appears_ that processing HTTP imposes a ~15 ms overhead over raw bytes. (Theoretically, the CONNECT proxy has to obey the host and port in the :authority pseudoheader, but in your case, the proxy would just ignore it, and hook up the edge function to whatever compute node it pleased.)
- deleted 3y ago[deleted]
- raoufchebri 3y ago> I'd love to know what fraction of the time is spent in processing, transit, querying the db, etc. We experimented with SELECT 1, so most of the latency is transit
- KingMob 3y agoThat's good to rule out db differences, but that doesn't explain where in transit the 15 ms difference is between HTTP and WS for established connections. I'm assuming the most likely difference is in Pg processing an HTTP request over processing raw bytes, but there are other explanations, too. (Older/slower code pathway for HTTP in proxy, different intermediate node behavior for HTTP vs TCP from proxy to db, etc. Much less likely, but not impossible.)
- kelvich 3y agoYes, exploring http/3 is on the roadmap. One of the learnings for us was the fact that v8 is quite good with caching open TCP connection; For consecutive queries the same TCP+TLS session is being reused. Even when queries are sent from unrelated V8 isolates.
- VWWHFSfQ 3y agoAre you saying that V8 is reusing TCP connections between different browser tabs and websites? Seems like a timing/side-channel attack could be done with that.
- KingMob 3y agoIn that case, HTTP/3 might actually perform worse for you than HTTP/2, unless v8 caches QUIC conns as well. The major advantage you'd get with HTTP/3 and QUIC is fewer round trips to start a conn, and no head-of-line blocking like with TCP (which might be irrelevant if the Pg protocol doesn't use simultaneous streams, dunno). The disadvantage is you'd have to support QUIC in your proxy (and gmac said your server lib doesn't support it yet.) Don't get hung up on HTTP/3 as the latest and greatest, HTTP/2 is probably still an improvement for you. > Even when queries are sent from unrelated V8 isolates. Heh, I've definitely written a few AWS Lambdas that exploited this sort of behavior. "Sure, this resource might not exist. But I'm going to check, just in case..."
- gmac 3y agoYes, we're currently on HTTP/1.1, but I'm sure we'll be looking at later versions. Unfortunately the Rust web server library we're using doesn't currently support HTTP/3, but we could potentially e.g. put an nginx proxy in front of it for this purpose. A key point of our WebSocket and http transports is to support environments where TCP connections aren't available. Are there environments that would let us send and receive binary data following an http CONNECT that wouldn't simply allow an ordinary TCP connection?
- toomim 3y agoTry HTTP/2. You don't need /3.
- KingMob 3y agoAs toomim also replied, HTTP/2 is fine, and honestly, is probably preferable for your use case, since you won't also have to deal with QUIC/UDP, and can keep using TCP. (HTTP/3 is ~90% the same as HTTP/2, just over a different transport layer, and ditching HTTP/2 prioritization mechanisms which you probably won't care about.) > A key point of our WebSocket and http transports is to support environments where TCP connections aren't available. Are there environments that would let us send and receive binary data following an http CONNECT that wouldn't simply allow an ordinary TCP connection? Based on the network diagram in your article, it looks like the edge functions speak HTTP to the proxy, upgrade that to WS, and then the proxy routes raw TCP to the conn pool/db. If so, CONNECT would be the same. The edge function sends an HTTP CONNECT request, the proxy would set it up, and afterwards, both hops are using the same transports as with websockets. The edge function would send HTTP/2 DATA frames wrapping the raw bytes, the proxy would unwrap those bytes, and then route the bytes over TCP to the conn pool/db. If I understand your setup correctly, HTTP/2 CONNECT would be applicable everywhere WS is.