7 ms·
> Once trusted, each worker executes its local query through DuckDB and streams intermediate Arrow IPC datasets back to the server over secure WebSockets. The s
by MobiusHorizons 1y ago
> Once trusted, each worker executes its local query through DuckDB and streams intermediate Arrow IPC datasets back to the server over secure WebSockets. The server merges and aggregates all results in parallel to produce the final SQL result—often in seconds.
Can someone explain why you would use websockets in an application where neither end is a browser? Why not just use regular sockets and cut the overhead of the http layer? Is there a real benefit I’m missing?
- philbe77 1y agoHi MobiusHorizons, I happened to use websockets b/c it was the technology I was familiar with. I will try to learn more about normal sockets to see if I could perhaps make them work with the app. Thanks for the suggestion...
- DanielHB 1y agoif you really want maximum performance maybe consider using CoAP for node-communication: https://en.wikipedia.org/wiki/Constrained_Application_Protocol https://en.wikipedia.org/wiki/Constrained_Application_Protoc... It is UDP-based but adds handshakes and retransmissions. But I am guessing for your benchmark transmission overhead isn't a major concern. Websockets are not that bad, only the initial connection is HTTP. As long as you don't create a ton of connections all the time it shouldn't be much slower than a TCP-based socket (purely theoretical assumption on my part, I never tested).
- gopalv 1y ago> will try to learn more about normal sockets to see if I could perhaps make them work with the app. There's a whole skit in the vein of "What have the Romans ever done for us?" about ZeroMQ[1] which has probably lost to the search index now. As someone who has held a socket wrench before, fought tcp_cork and dsack, Websockets isn't a bad abstraction to be on top of, especially if you are intending to throw TLS in there anyway. Low level sockets is like assembly, you can use it but it is a whole box of complexity (you might use it completely raw sometimes like a tickle ack in the ctdb[2] implementation). [1] - https://news.ycombinator.com/item?id=32242238 https://news.ycombinator.com/item?id=32242238 [2] - https://linux.die.net/man/1/ctdb https://linux.die.net/man/1/ctdb
- sureglymop 1y agoWait but websockets aren't over http right? Just the initiation and then there is a protocol upgrade or am I wrong? What overhead is there otherwise?
- tsimionescu 1y agoYou're right, WebSockets aren't over HTTP, they just use HTTP for the connection initiation. They do add some overhead in two places: one, when opening a new connection, since you go TCP -> TLS -> HTTP -> WebSockets -> Your protocol ; and two, they do add some per packet overhead, since there is a WebSocket encapsulation of your data - but this is much smaller than typical HTTP request/response overhead.
- kevincox 1y ago> the overhead of the http layer There isn't much overhead here other than connection setup. For HTTP/1 the connection is just "upgraded" to websockets. For HTTP/2 I think the HTTP layer still lives on a bit so that you can use connection multiplexing (which maybe be overhead if you have no use for it here) but that is still a very thin layer. So I think the question isn't so much HTTP overhead but WebSocket overhead. WebSockets add a bit of message framing and whatnot that may be overhead if you don't need it. In 99% of applications if you need encryption, authentication and message framing you would be hard-pressed to find a significantly more efficient option.
- deleted 1y ago[deleted]
- toast0 1y ago> In 99% of applications if you need encryption, authentication and message framing you would be hard-pressed to find a significantly more efficient option. AFAIK, websockets doesn't do authentication? And the encryption it does is minimal, optional xor with a key disclosed in the handshake. It does do framing. It's not super common, but if all your messages have a 16-bit length, you can just use TLS framing. I would argue that TLS framing is ineffecient (multiple length terms), but using it by itself is better than adding a redundant framing layer. But IMHO, there is significant benefit from removing a layer where it'd unneeded.
- LunaSea 1y ago> AFAIK, websockets doesn't do authentication? Websocket allows for custom header and query parameters which make it possible to run a basic authentication scheme and later on additional autorisation in the message themselves if really necessary. > And the encryption it does is minimal, optional xor with a key disclosed in the handshake. It does do framing. Web Secure Socket (WSS) is the TLS encrypted version of Websockets (WS) (similar to HTTP vs. HTTPS).
- fyrn_ 1y agoWorth noting that wbesockets in the browser don't allow custom headers and custom header support is spotty accross sever impls. It's just not exposed in the javascript API. There has been an open chrome bug for that for like 15 years
- simonw 1y agoIf you're using sockets you still need to come up with some kind of protocol on top of those sockets for the data that's being transferred - message delimiter, a data format etc. Then you have to build client libraries for that protocol. WebSockets solve a bunch of those low level problems for you, in a well specified way with plenty of existing libraries.
- HumblyTossed 1y agoASCII table codes 1,2,3 & 4 pretty simple to use.
- dns_snek 1y agoSure, in principle. Someone already mentioned binary data, then you come up with a framing scheme and get to write protocol documentation, but why? What's the benefit?
- HumblyTossed 1y agoSimplicity.
- ryanjshaw 1y agoYou misspelled “bugs and maintenance nightmare”
- jcheng 1y agoNot if you're passing binary data
- woodruffw 1y agoEven beyond that: the ASCII delimiter control codes are perfectly valid UTF-8 (despite not being printable), so using them for in-band signaling is a recipe for pain on arbitrary UTF-8 data.
- lucideer 1y ago> overhead of the http layer Detail of this well-covered in sibling comments, but at a higher-level, two thoughts on this: 1. I see a lot of backlash lately against everything being HTTP-ified, with little justification other than a presumption that it necessarily adds overhead. Perf-wise, HTTP has come a long way & modern HTTP is a very efficient protocol. I think this has cleared the way for it to be a foundation for many more things than in the past. HTTP/3 being over UDP might clear the way for more of this (albeit I think the overhead of TCP/IP is also often overstated - see e.g. MQTT). 2. Overhead can be defined in two ways: perf. & maintenance complexity. Modern HTTP does add a bit of the latter, so in that context it may be a fair concern, but I think the large range of competing implementations probably obviates any concern here & the alternative usually involves doing something custom (albeit simpler), so you run into inconsistency, re-invented wheels & bus factor issues there.
- fweimer 1y agoOne reason comes to my mind: HTTP is no longer a stable protocol with well-understood security properties. If you deploy it today, people expect interoperability with clients and servers that implement future protocol upgrades, resulting in an ongoing maintenance burden that a different protocol choice would avoid.
- lucideer 1y agoI'm absolutely not an expert of any kind on protocol details, so pardon my ignorance here but this surprises me: is this true? High-level spec changes have been infrequent, with long dual support periods, & generally seen pretty slow gradual client & server adoption. 1.1 was 1997 & continues to have widespread support today. 2 & 3 were proposed in 2015 & 2016 - almost 2 decades later - & 2 is only really starting to see wide support today, with 3 still broadly unsupported. I'm likely missing a lot of nuance in between versioned releases though - I know e.g. 2 saw at least two major additions/updates, though I thought those were mostly additive security features rather than changes to existing protocol features.
- tsimionescu 1y ago
- nurettin 1y agohttp 101 upgrade isn't much of an overhead and there are tried and tested websocket/ssl libraries with pretty callback interfaces versus your custom binary protocol. I would still choose the latter but I wouldn't recommend it.
- immibis 1y agoyou can apply this reasoning to a lot of protocols, like why not use Nostr over websockets? I mean, I don't see any reason to do this with Nostr over websockets, but also, why not? it's not much overhead right?
- stevenhuang 1y agoComparing ws to nostr shows you might not understand how ws actually works. You realize after connection setup it's just a tcp socket? It's not framed by http headers if that's what you're wondering. The ws frame is like 6 bytes.
- immibis 1y agoThat connection setup has a huge amount of complexity in it. The fact the complexity is front-loaded doesn't negate the complexity. You have to include an HTTP parser and a copy of SHA1 for basically no good reason.
- lantastic 1y agoOthers pointed plenty of arguments, but the ones I find most compelling (not necessarily useful in this context) are: - you can serve any number of disjoint websocket services via same port via HTTP routing - this also means you can do TLS termination in one place, so downstream websocket service doesn't have to deal with the nitty-gritty of certificates. Sure, it adds a hop compared to socket passing, and there are ways to get similar fanout with TCP with a custom protocol. But you need to add this to every stack that interacting components use, while websockets libraries exist for most languages that are likely to be used in such an endeavor.
- rcxdude 1y agoI've done this. It's a reasonably straightforward way to multiplex multiple endpoints over a single TCP socket, and it also gives you a framing protocol. It doesn't have a particularly high overhead (past the initial headers and such it's just a socket with a pretty lightweight frame header). You can find a library in basically every language and/or framework and they often deal with a bunch of other details for you.