11 ms·

Back to basics: Why we chose long-polling over websockets

258 points by lunarcave 2y ago

CharlieDigital 2y ago
Would there be any technical benefit to this over using server sent events (SSE)? Both are similar in that they hold the HTTP connection open and have the benefit of being simply HTTP (the big plus here). SSE (at least to me) feels like it's more suitable for some use cases where updates/results could be streamed in. A fitting use case might be where you're monitoring all job IDs on behalf of a given client. Then you could move the job monitoring loop to the server side and continuously yield results to the client.
- lunarcave 2y ago
  Good point! We did consider SSE, but ultimately decided against it due to the way we have to re-implement response payloads (one for application/json and one for text/event-stream). I've not personally witnessed this, but people on the internets have said that _some_ proxies/LBs have problems with SSE due to the way it does buffering.
  - gorjusborg 2y ago
    > we have to re-implement response payloads (one for application/json and one for text/event-stream) I am curious about what you mean here. The 'text/event-stream' allows for abitrary event formats, it just provides structure for EventSource to be able to parse. You should only need one 'text/event-stream' and should be able send the same JSON via normal or SSE response.
    - josephg 2y ago
      What the GP commenter might have meant is that websockets support binary message bodies. SSE does not.
- vindex10 2y ago
  I got interested and found this nice thread on SO: https://stackoverflow.com/a/5326159 https://stackoverflow.com/a/5326159 One of the drawbacks, as I learned - SSE have limit on number of up to ~6 open connections (browser + domain name). This can quickly become a limiting factor when you open the same web page in multiple tabs.
  - CharlieDigital 2y ago
    As the other two comments mentioned, this is a restriction with HTTP/1.1 and it would apply also to long polling connections as well.
  - arresin 2y ago
    …if you’re using http/1.1. It’s not an issue with 2+
  - Klonoar 2y ago
    Not an issue if you’re using HTTP/2 due to how multiplexing happens.
  - _heimdall 2y ago
    Syncing state across multiple tabs and windows is always a bit tricky. For SSE, I'd probably reach for the BroadcastChannel API. Open the SSE connection in the first tab and have it broadcast events to any other open tab or window.
  - bioneuralnet 2y ago
    I've gotten around this by using the Page Visibility API - https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibi.... Close the SSE connection when the page is hidden, and re-open when it becomes visible again.
- csumtin 2y ago
  I tried using SSE and found it didn't work for my use case, it was broken on mobile. When users switched from the browser to an app and back, the SSE connection was broken and they wouldn't receive state updates. Was easier to do long polling
  - CharlieDigital 2y ago
    Not sure about the default `EventSource` object in JavaScript, but the Microsoft polyfill that I use (https://github.com/Azure/fetch-event-source https://github.com/Azure/fetch-event-source) supports `POST` and there's an option `openWhenHidden` which controls how it reacts to users tabbing away.
  - josephg 2y ago
    The standard way to fix that is to send ping messages every ~15 seconds or something over the SSE stream. If the client doesn’t get a ping in any 20 second window, assume the sse stream is broken somehow and restart it. It’s complex but it works. The big downside of sse in mobile safari - at least a few years ago - is you got a constant loading spinner on the page. Thats bad UX.
- bythreads 2y ago
  SSE are being removed from various stacks and implementations
  - sudodevnull 2y ago
    That's the dumbest thing I've ever heard. SSE is just a response type for normal HTTP. Explain exactly what you mean cause that's like saying that are removing response types from HTTP.
    - CharlieDigital 2y ago
      Yup; would also like a citation on that.
bartvk 2y ago
Refreshing to be reminded of a relatively simple alternative to websockets. For a short time, I worked at a now-defunct startup which had made the decision for websockets. It was an app that would often be used on holiday so testing was done on hotel and restaurant wifi. Websockets made that difficult.
- ipnon 2y ago
  I feel like WebSockets are already as simple as it gets. It's "just" an HTTP request with an indeterminate body. Just make an HTTP request and don't close the connection. That's a WebSocket.
  - bluepizza 2y ago
    It's surprisingly complex. Connections are dropped all the time, and then your code, on both client and server, need to account for retries (will the reconnection use a cached DNS entry? how will load balancing affect long term connections?), potentially missed events (now you need a delta between pings), DDoS protections (is this the same client connecting from 7 IPs in a row or is this a botnet), and so on. Regular polling great reduces complexity on some of these points.
    - slau 2y ago
      Long polling has nearly all the same disadvantages. Disconnections are harder to track, DNS works exactly the same for both techniques, as does load balancing, and DDoS is specifically about different IPs trying to DoS your system, not the same IP creating multiple connections, so irrelevant to this discussion. Yes, WS is complex. Long polling is not much better. I can’t help but think that if front end connections are destroying your database, then your code is not structured correctly. You can accept both WS and long polls without touching your DB, having a single dispatcher then send the jobs to the waiting connections.
      - bluepizza 2y ago
        My understanding is that long polling has these issues handled by assuming the connection will be regularly dropped. Clients using mobile phones tend to have their IPs rapidly changed in sequence. I didn't mention databases, so I can't comment on that point.
valenterry 2y ago
Using websockets with graphql, I feel like a lot of the challenges are then already solved. From the post: - Observability: WebSockets are more stateful, so you need to implement additional logging and monitoring for persistent connections: solved with graphql if the existing monitoring is already sufficient. - Authentication: You need to implement a new authentication mechanism for incoming WebSocket connections: solved with graphql. - Infrastructure: You need to configure your infrastructure to support WebSockets, including load balancers and firewalls: True, firewalls need to be updated. - Operations: You need to manage WebSocket connections and reconnections, including handling connection timeouts and errors: normally already solved by the graphql library. For errors, it's basically the same though. - Client Implementation: You need to implement a client-side WebSocket library, including handling reconnections and state management: Just have to use a graphql library that comes with websocket support (I think most of them do) and configure it accordingly.
- anonzzzies 2y ago
  I hope (never needed this) client implementations that do this all for you and pick the best implementation based on what the client supports? Not sure why the transport is interesting when/if you have freedom to choose.
  - josephg 2y ago
    Yeah there’s plenty of high quality websocket client libraries in all languages now. Support and feature are excellent. And they’ve been supported in all browsers for a decade or something at this point. I vomit in my mouth a bit whenever people reach for socket.io or junk like that. You don’t want or need the complexity and bugs these libraries bring. They’re obsolete.
- _heimdall 2y ago
  Using graphql comes with IRS own list of challenges and issues though. Its a good solution for some situations, but it isn't so universal that you can just switch to it without a problem.
feverzsj 2y ago
Why not just use chunked encoding and get rid of extra requests.
justinl33 2y ago
yeah, authentication complexity with WebSockets is severely underappreciated. We ran into major RBAC headaches when clients needed to switch between different privilege contexts mid-session. Long polling with standard HTTP auth patterns eliminates this entire class of problems.
- watermelon0 2y ago
  Couldn't you just disconnect and reconnect websocket if privileges change, since the same needs to be done with the long polling?
  - josephg 2y ago
    Yeah, and you can send cookies in the websocket connection headers. This used to be a problem in some browsers iirc - they wouldn’t send cookies properly over websocket connection requests. As a workaround in one project I wrote JavaScript code which manually sent cookies in the first websocket message from the client as soon as a connection opened. But I think this problem is now solved in all browsers.
baumschubser 2y ago
I like long polling, it’s easy to understand from start to finish and from client perspective it just works like a very slow connection. You have to keep track of retries and client-side cancelled connections to have one but only one (and the right one) of requests at hand to answer to. One thing that seems clumsy in the code example is the loop that queries the data again and again. Would be nicer if the data update could also resolve the promise of the response directly.
- moribvndvs 2y ago
  You could have your job status update push an update into an in-memory or distributed cache and check that in your long poll rather than a DB lookup, but that may require adding a bunch of complexity to wire the completion of the task to updating said cache. If your database is tuned well and you don’t have any other restrictions (e.g. serverless where you pay by the IO), it may be good enough and come out in the wash.
- josephg 2y ago
  Hard disagree. Long polling can have complex message ordering problems. You have completely different mechanisms for message passing from client-to-server and server-to-client. And middle boxes can and will stall long polled connections, stopping incremental message delivery. (Or you can use one http query per message - but that’s insanely inefficient over the wire). Websockets are simply a better technology. With long polling, the devil is in the details and it’s insanely hard to get those details right in every case.
  - _nalply 2y ago
    One of them 2001 was that Netscape didn't render correctly if the connection is still open. Hah. I am sure this issue has been fixed a long, long time ago, but perhaps there are other issues. Nowadays I prefer SSE to long polling and websockets. The idea is: the client doesn't know that the server has new data before it makes a request. With a very simple SSE the client is told that new data is there then it can request new data separately if it wants. This said, SSE has a few quirks, one of them that on HTTP/1 the connection counts to the maximum limit of 6 concurrent connections per browser and domain, so if you have several tabs, you need a SharedWorker to share the connection between the tabs. But probably this quirk also appllies to long polling and websockets. Another quirk, SSE can't transmit binary data and has some limitations in the textual data it represents. But for this use case this doesn't matter. I would use websockets only if you have a real bidirectional data flow or need to transmit complex data.
peheje 2y ago
What about HTTP/2 Multiplexing, how does it hold up against long-polling and websockets? I have only tried it briefly when we use gRPC: https://grpc.io/docs/what-is-grpc/core-concepts/#server-streaming-rpc https://grpc.io/docs/what-is-grpc/core-concepts/#server-stre... Here it's easy to specify that a endpoint is a "stream", and then the code-generation tool gives all tools really to just keep serving the client with multiple responses. It looks deceptively simple. We already have setup auth, logging and metrics for gRPC, so I hope it just works off of that maybe with minor adjustments. But I'm guessing you don't need the gRPC layer to use HTTP/2 Multiplexing?
- toast0 2y ago
  At least in a browser context, HTTP/2 doesn't address server to client unsolicitied messages. So you'd still need a polling request open from the client. HTTP/2 does specify a server push mechanism (PUSH_PROMISE), but afaik, browsers don't accept them and even if they did, (again afaik) there's no mechanism for a page to listen for them. But if you control the client and the server, you could use it.
- yencabulator 2y ago
  gRPC as specced to ride directly on top of HTTP/2 doesn't work from browsers, the sandboxed JS isn't allowed that level of control over the protocol. And often is too low level to implement as part of a pre-existing HTTP server, too. gRPC is a server-to-server protocol that is not part of the usual Web, but happens to repurpose HTTP/2. Outside of gRPC, just HTTP POST cannot at this time replace websockets because the in-browser `fetch` API doesn't support streaming request body. For now, websockets is the only thing that can natively provide an ordered stream of messages from browser to server.
- BitPirate 2y ago
  With RFC8441 websockets are just HTTP/2 streams.
bigbones 2y ago
I don't know how meaningful it is any more, but with long polling with a short timeout and a gracefully ended request (i.e. chunked encoding with an eof chunk sent rather than disconnection), the browser would always end up with one spare idle connection to the server, making subsequent HTTP requests for other parts of the UI far more likely to be snappier, even if the app has been left otherwise idle for half the day I guess at least this trick is still meaningful where HTTP/2 or QUIC aren't in use
ipnon 2y ago
Articles like this make me happy to use Phoenix and LiveView every day. My app uses WebSockets and I don’t think about them at all.
- leansensei 2y ago
  Same here. It truly is a godsend.
- zacksiri 2y ago
  I was thinking this exact thing as I was reading the article.
- tzumby 2y ago
  I came here to say exactly this! Elixir and OTP (and by extension LiveView) are such a good match for the problem described in the post.
  - j45 2y ago
    I was kind of wondering how something hadn't solve this at all, compared to a solution not readily being on one's path.
- cultofmetatron 2y ago
  hah seriously. my app uses web sockets extensively but since we are also using Phoenix, its never been source of conflict in development. it really was just drop it and scale to thousands of users.
  - arrty88 2y ago
    Why couldn’t nodejs with uWS library or golang + gorilla handle 10s of thousands of connections?
    - apitman 2y ago
      I think GP's point is that they feel Phoenix is simpler to use than alternatives, not necessarily that it scales better.
sgarland 2y ago
The full schema isn’t listed, but the indices don’t make sense to me. (id, cluster_id) sounds like it could / should be the PK If the jobs are cleared once they’ve succeeded, and presumably retried if they’ve failed or stalled, then the table should be quite small; so small, that a. The query planner is unlikely to use the partial index on (status) b. The bloat from the rapidity of DELETEs likely overshadows the live tuple size.
DougN7 2y ago
I implemented a long polling solution in desktop software over 20 years ago and it’s still working great. It can even be used as a tunnel to stream RDP sessions, through which YouTube can play without a hiccup. Big fan of long polling, though I admit I didn’t get a chance to try web sockets back then.
- jclarkcom 2y ago
  I did the same, were you at VMware by any chance? At the time it was the only way to get comparability with older browsers.
deleted 2y ago
[deleted]
mojuba 2y ago
Can someone explain why TTL = 60s is a good choice? Why not more, or less?
- notatoad 2y ago
  i can't speak for why the author chose it, but if you're operating behind AWS cloudfront then http requests have a maximum timeout of 60s - if you don't close the request within 60s, cloudfront will close it for you. i suspect other firewalls, cdns, or reverse proxy products will all do something similar. for me, this is one of the biggest benefits of websockets over long-polling: it's a standard way to communicate to proxies and firewalls "this connection is supposed to stay open, don't close it on me"
k__ 2y ago
Half-OT: What's the most resource efficient way to push data to clients over HTTP? I can send data to a server via HTTP request, I just need a way to notify a client about a change and would like to avoid polling for it. I heard talk about SSE, WebSockets, and now long-polling. Is there something else? What requires the least resources on the server?
- mojuba 2y ago
  I don't think any of the methods give any significant advantage since in the end you need to maintain a connection per each client. The difference between the methods boils down to complexity of implementation and reliability. If you want to reduce server load then you'd have to sacrifice responsiveness, e.g. you perform short polls at certain intervals, say 10s.
  - k__ 2y ago
    Okay, thanks. What's the least complex to implement then?
    - mojuba 2y ago
      For the browser and if you need only server-to-client sends, I assume SSE would be the best option. For other clients, such as mobile apps, I think long poll would be the simplest.
amelius 2y ago
> Corporate firewalls blocking WebSocket connections was one of our other worries. Some of our users are behind firewalls, and we don't need the IT headache of getting them to open up WebSockets. Don't websockets look like ordinary https connections?
- doublerabbit 2y ago
  It does. However DPI firewalls look at and block the upgrade handshake. Connection: Upgrade Upgrade: websocket
- toast0 2y ago
  Some corporate firewalls MITM all https connections. Websocket does not look normal once you've terminated TLS.
  - amelius 2y ago
    Can websites detect this?
    - toast0 2y ago
      AFAIK, only by symptoms. If https fetches work and websockets don't, that's a sign. HSTS and assorted reporting can help a bit in aggregate, but not if the corporate MITM CA has been inserted into the browser's trusted CA list. I don't think there's an API to get certificate details from the browser side to compare. A proxy may have a different TLS handshake than a real browser would, depending on how good the MITM is, but the better they are, the more likely it is that websockets work.
yuliyp 2y ago
I think this article is tying a lot of unrelated decisions to "Websocket" vs "Long-polling" when they're actually independent. A long-polling server could handle a websocket client with just a bit of extra work to handle keep-alive. For the other direction, to support long-polling clients if your existing architecture is websockets which get data pushed to them by other parts of the system, just have two layers of servers: one which maintains the "state" of the connection, and then the HTTP server which receives the long polling request can connect to the server that has the connection state and wait for data that way.
- harrall 2y ago
  It sounded like the author(s) just had existing request-oriented code and didn’t want to rewrite it to be connection-oriented. Personally I would enjoyed solving that problem instead of hacking around it but that’s me.
- lunarcave 2y ago
  Author here. Having done this, I don't think I'd reduce it to "just a little bit of work" to make it hum in production. Everything in between your UI components and the database layer needs to be reworked to work in the connection-oriented (Websockets) model of the world vs request-oriented world.
  - bythreads 2y ago
    It's a good pattern to have a vanilla js network manager / layer in fe for this exact reason - makes swapping network technologies a lot simpler. Only that knows url for endpoints, protocols and connections - and proxies between them and your app / components
  - yuliyp 2y ago
    > Everything in between your UI components and the database layer needs to be reworked to work in the connection-oriented (Websockets) model of the world vs request-oriented world. How so? As a minimal change, the thing on the server end of the websocket could just do the polling of your database on its own while the connection is open (using authorization credentials supplied as the websocket is being opened). If the connection dies, stop polling. This has the nice property that you're in full control of the refresh rate, can implement coordinated backoffs if the database is overloaded, etc.
vitus 2y ago
I would appreciate if the article spent more time actually discussing the benefits of websockets (and/or more modern approaches to pushing data from server -> browser) and why the team decided those benefits were not worth the purported downsides. I could see the same simplicity argument being applied to using unencrypted HTTP/1.1 instead of HTTP/2, or TCP Reno instead of CUBIC. The section at the end talking about "A Case for Websockets" really only rehashes the arguments made in "Hidden Benefits of Long-Polling" stating that you need to reimplement these various mechanisms (or just use a library for it). My experience in this space is from 2011, when websockets were just coming onto the scene. Tooling / libraries were much more nascent, websockets had much lower penetration (we still had to support IE6 in those days!), and the API was far less stable prior to IETF standardization. But we still wanted to use them when possible, since they provided much better user experience (lower latency, etc) and lower server load.
tguvot 2y ago
Another reason: there is a patent troll suing companies over usage of websockets.
vouwfietsman 2y ago
The points mentioned against websockets are mostly fud, I've used websockets in production for a very heavy global data streaming application, and I would respond the following to the "upsides" of not using websockets: > Observability Remains Unchanged Actually it doesn't, many standard interesting metrics will break because long-polling is not a standard request either. > Authentication Simplicity Sure, auth is different than with http, but not more difficult. You can easily pass a token. > Infrastructure Compatibility I'm sure you can find firewalls out there where websockets are blocked, however for my use case I have never seen this reported. I think this is outdated, for sure you don't need "special proxy configurations or complex infrastructure setups". > Operational Simplicity Restarts will drop any persistent connection, state can be both or neither in WS or in LP, it doesn't matter what you use. > Client implementation It mentions "no special WebSocket libraries needed" and also "It works with any HTTP client". Guess what, websockets will work with any websocket client! Who knew! Finally, in the conclusion: > For us, staying close to the metal with a simple HTTP long polling implementation was the right choice Calling simple HTTP long polling "close to the metal" in comparison to websockets is weird. I wouldn't be surprised if websockets scale much better and give much more control depending on the type of data, but that's besides the point. If you want to use long polling because you prefer it, go ahead. Its a great way to stick to request/response style semantics that web devs are familiar with. Its not necessary to regurgitate a bunch of random hearsay arguments that may influence people in the wrong way. Try to actually leave the reader with some notion of when to use long polling vs when to use websockets, not a post-hoc justification of your decision based on generalized arguments that do not apply.
- amatuer_sodapop 2y ago
  > > Observability Remains Unchanged > Actually it doesn't, many standard interesting metrics will break because long-polling is not a standard request either. As a person who works in a large company handling millions of websockets, I fundamentally disagree with discounting the observability challenges. WebSockets completely transform your observability stack - they require different logging patterns, new debugging approaches, different connection tracking, and change how you monitor system health at scale. Observability is far more than metrics, and handwaving away these architectural differences doesn't make the implementation easier.
Cort3z 2y ago
I think they are mixing some problems here. They could probably have used their original setup with Postgres NOTIFY+triggers in stead of polling, and only have one "pickup poller" to catch any missed events/jobs. In my opinion transaction medium should not be linked to how the data is manage internally, but I know from experience that this separation is often hard to achieve in practice.
emilio1337 2y ago
The article does discuss a lot of mixed concepts. I would prefer one process polling new jobs/state and one process handling http connections/websockets. Hence no flooding the database and completely scalable from the client side. The database process pushes everything downstream via some queue while the other process/server handles those and sends them to respective clients
imglorp 2y ago
Since the article mentioned Postgres by name, isn't this a case for using its asynchronous notification features? Servers can LISTEN to a channel and PG can TRIGGER and NOTIFY them when the data changes. No polling needed, regardless of the frontend channel.
- lunarcave 2y ago
  Yes, but the problems of detecting that changeset and delivering it to the right connection remains to be solved in the app layer.
- cluckindan 2y ago
  It would be easier to run Hypermode’s Dgraph as the database and use GraphQL subscriptions from the frontend. But nobody ever got fired for choosing postgres.
  - j45 2y ago
    I have relatively recently taken steps towards Postgres from it's abiality to be at the center of so much until a project outgrows it. In terms of not getting fired - Postgres is a lot more innovative than most databases, and the insinuation of IBM. By innovative I mean uniquely putting in performance related items for the last 10-20 years.
wereHamster 2y ago
Unrelated to the topic in the article… await new Promise(resolve => setTimeout(resolve, 500)); In Node.js context, it's easier to: import { setTimeout } from "node:timers/promises"; await setTimeout(500);
- treve 2y ago
  Is that easier? The first snippet is shorter and works on any runtime.
  - joshmanders 2y ago
    In the context of Node.js, where op said, yes it is easier. But it's a new thing and most people don't realize timers in Node are awaitable yet, so the other way is less about "works everywhere" and more "this is just what I know"
    - wereHamster 2y ago
      I guess most Node.js developers also don't realize that there's "node:fs/promises" so you don't have to use callbacks or manually wrap functions from "node:fs" with util.promisify(). Doesn't mean need to stick with old patterns forever. When I said 'in the context of Node.js' I meant if you are in a JS module where you already import other node: modules, ie. when it's clear that code runs in a Node.js runtime and not in a browser. Of course when you are writing code that's supposed to be portable, don't use it. Or don't use setTimeout at all because it's not guaranteed to be available in all runtimes - it's not part of the ECMA-262 language specification after all.
- hombre_fatal 2y ago
  I haven't used that once since I found out that it exists. I just don't see the point. It doesn't work in the browser and it shadows global.setTimeout which is confusing. Meanwhile the idiom works everywhere.
  - joshmanders 2y ago
bob1029 2y ago
I think we could have the best of both worlds. E.g.: https://socket.io/docs/v4/how-it-works/#upgrade-mechanism https://socket.io/docs/v4/how-it-works/#upgrade-mechanism
- j45 2y ago
  I wonder if people only look for solutions in their language when a tech like websockets is language independent.
rednafi 2y ago
Neither Server-Sent Events nor WebSockets have replaced all use cases of long polling reliably. The connection limit of SSE comes up a lot, even if you’re using HTTP/2. WebSockets, on the other hand, are unreliable as hell in most environments. Also, WS is hard to debug, and many of our prod issues with WS couldn’t even be reproduced locally. Detecting changes in the backend and propagating them to the right client is still an unsolved problem. Until then, long polling is surprisingly simple and a robust solution that works.
- pas 2y ago
  Robust WS solutions need a fallback anyway, and unless you are doing something like Discord long polling is a reasonable option.
- infamia 2y ago
  > The connection limit of SSE comes up a lot, even if you’re using HTTP/2. I'm considering using SSE for an app. I'm curious, what problems you've run into? At least the docs say you get 100 connections between the server and a client, but it can be negotiated higher if needed it seems? https://developer.mozilla.org/en-US/docs/Web/API/EventSource https://developer.mozilla.org/en-US/docs/Web/API/EventSource
  - rednafi 2y ago
    SSEs are great and more reliable than websockets in a smaller scale. So I'd reach for it despite the issues. But that being said, some websevers don't play well with SSE and you'll need to fiddle with it. If you control the webserver, then it's not much of a problem.
Animats 2y ago
Long polling has some problems of its own. Second Life has an HTTPS long polling channel between client and server. It's used for some data that's too bulky for the UDP connection, not too time sensitive, or needs encryption. This has caused much grief. On the client side, the poller uses libcurl. Libcurl has timeouts. If the server has nothing to send for a while, libcurl times out. The client then makes the request again. This results in a race condition if the server wants to send something between timeout and next request. Messages get lost. On top of that, the real server is front-ended by an Apache server. This just passes through relevant requests, blocking the endless flood of junk HTTP requests from scrapers, attacks, and search engines. Apache has a timeout, and may close a connection that's in a long poll and not doing anything. Additional trouble can come from middle boxes and proxy servers that don't like long polling. There are a lot of things out there that just don't like holding an HTTP connection open. Years ago, a connection idle for a minute was fine. Today, hold a connection open for ten seconds without sending any data and something is likely to disconnect it. The end result is an unreliable message channel. It has to have sequence numbers to detect duplicates, and can lose messages. For a long time, nobody had discovered that, and there were intermittent failures that were not understood. In the original article, the chart section labelled "loop" doesn't mention timeout handling. That's not good. If you do long polling, you probably need to send something every few seconds to keep the connection alive. Not clear what a safe number is.
- rednafi 2y ago
  Yeah, some servers close connections when there’s no data transfer. When the backend holds the connection while polling the database until a timeout occurs or the database returns data, it needs to send something back to the client to keep the connection alive. I wonder what could be sent in this case and whether it would require special client-side logic.
  - mhitza 2y ago
    In the HTTP model, technically status code 102 Processing would fit best. Though, no longer part of the HTTP specification. https://http.dev/102 https://http.dev/102 100 Continue could be usable as a workaround. Would probably require at a bare minimum some extra integration code on the client side.
sneak 2y ago
Given long polling, I have never ever understood why websockets became a thing. I’ve never implemented them and never will, it’s a protocol extension where none is necessary.
- ekkeke 2y ago
  Websockets can operate outside the request/response model used in this long polling example, and allow you to stream data continuously. They're also a lot more efficient in terms of framing and connections if there are a lot of individual pieces of data to push as you don't need to spin a up a connection + request for each bit.
LeicaLatte 2y ago
Long polling is my choice for simple, reliable and plug and play like interfaces. HTTP requests tend to be standard and simplify authentication as well. Systems with frequent but not constant updates are ideal. Text yes. Voice maybe not. Personal Case Study: I built mobile apps which used Flowise assistants for RAG and found websockets compeletely out of line with the rest of my system and interactions. Suddenly I was fitting a round peg in a square hole. I switched to OpenAI assistants and their polling system felt completely "natural" to integrate.
gloosx 2y ago
>Our system handles hundreds of worker nodes constantly polling our PostgreSQL-backed control plane for new jobs Does everybody poll their PosgreSQL to get new rows in real-time? This is really weird, there are trigger functions and notifications.
- lunarcave 2y ago
  In pg, each unique `listen(channel)` takes up server resources, and if you don't reliably clean them up, everything comes to a screeching halt. There's also `max_notify_queue_pages` >Specifies the maximum amount of allocated pages for NOTIFY / LISTEN queue. The default value is 1048576.

258 points by lunarcave 2y ago

Would there be any technical benefit to this over using server sent events (SSE)? Both are similar in that they hold the HTTP connection open and have the benefit of being simply HTTP (the big plus here). SSE (at least to me) feels like it's more suitable for some use cases where updates/results could be streamed in. A fitting use case might be where you're monitoring all job IDs on behalf of a given client. Then you could move the job monitoring loop to the server side and continuously yield results to the client.

lunarcave 2y ago
Good point! We did consider SSE, but ultimately decided against it due to the way we have to re-implement response payloads (one for application/json and one for text/event-stream). I've not personally witnessed this, but people on the internets have said that _some_ proxies/LBs have problems with SSE due to the way it does buffering.
- gorjusborg 2y ago
  > we have to re-implement response payloads (one for application/json and one for text/event-stream) I am curious about what you mean here. The 'text/event-stream' allows for abitrary event formats, it just provides structure for EventSource to be able to parse. You should only need one 'text/event-stream' and should be able send the same JSON via normal or SSE response.
  - josephg 2y ago
    What the GP commenter might have meant is that websockets support binary message bodies. SSE does not.
vindex10 2y ago
I got interested and found this nice thread on SO: https://stackoverflow.com/a/5326159 https://stackoverflow.com/a/5326159 One of the drawbacks, as I learned - SSE have limit on number of up to ~6 open connections (browser + domain name). This can quickly become a limiting factor when you open the same web page in multiple tabs.
- CharlieDigital 2y ago
  As the other two comments mentioned, this is a restriction with HTTP/1.1 and it would apply also to long polling connections as well.
- arresin 2y ago
  …if you’re using http/1.1. It’s not an issue with 2+
- Klonoar 2y ago
  Not an issue if you’re using HTTP/2 due to how multiplexing happens.
- _heimdall 2y ago
  Syncing state across multiple tabs and windows is always a bit tricky. For SSE, I'd probably reach for the BroadcastChannel API. Open the SSE connection in the first tab and have it broadcast events to any other open tab or window.
- bioneuralnet 2y ago
  I've gotten around this by using the Page Visibility API - https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibility_API https://developer.mozilla.org/en-US/docs/Web/API/Page_Visibi.... Close the SSE connection when the page is hidden, and re-open when it becomes visible again.
csumtin 2y ago
I tried using SSE and found it didn't work for my use case, it was broken on mobile. When users switched from the browser to an app and back, the SSE connection was broken and they wouldn't receive state updates. Was easier to do long polling
- CharlieDigital 2y ago
  Not sure about the default `EventSource` object in JavaScript, but the Microsoft polyfill that I use (https://github.com/Azure/fetch-event-source https://github.com/Azure/fetch-event-source) supports `POST` and there's an option `openWhenHidden` which controls how it reacts to users tabbing away.
- josephg 2y ago
  The standard way to fix that is to send ping messages every ~15 seconds or something over the SSE stream. If the client doesn’t get a ping in any 20 second window, assume the sse stream is broken somehow and restart it. It’s complex but it works. The big downside of sse in mobile safari - at least a few years ago - is you got a constant loading spinner on the page. Thats bad UX.
bythreads 2y ago
SSE are being removed from various stacks and implementations
- sudodevnull 2y ago
  That's the dumbest thing I've ever heard. SSE is just a response type for normal HTTP. Explain exactly what you mean cause that's like saying that are removing response types from HTTP.
  - CharlieDigital 2y ago
    Yup; would also like a citation on that.

bartvk 2y ago

Refreshing to be reminded of a relatively simple alternative to websockets. For a short time, I worked at a now-defunct startup which had made the decision for websockets. It was an app that would often be used on holiday so testing was done on hotel and restaurant wifi. Websockets made that difficult.

ipnon 2y ago
I feel like WebSockets are already as simple as it gets. It's "just" an HTTP request with an indeterminate body. Just make an HTTP request and don't close the connection. That's a WebSocket.
- bluepizza 2y ago
  It's surprisingly complex. Connections are dropped all the time, and then your code, on both client and server, need to account for retries (will the reconnection use a cached DNS entry? how will load balancing affect long term connections?), potentially missed events (now you need a delta between pings), DDoS protections (is this the same client connecting from 7 IPs in a row or is this a botnet), and so on. Regular polling great reduces complexity on some of these points.
  - slau 2y ago
    Long polling has nearly all the same disadvantages. Disconnections are harder to track, DNS works exactly the same for both techniques, as does load balancing, and DDoS is specifically about different IPs trying to DoS your system, not the same IP creating multiple connections, so irrelevant to this discussion. Yes, WS is complex. Long polling is not much better. I can’t help but think that if front end connections are destroying your database, then your code is not structured correctly. You can accept both WS and long polls without touching your DB, having a single dispatcher then send the jobs to the waiting connections.
    - bluepizza 2y ago
      My understanding is that long polling has these issues handled by assuming the connection will be regularly dropped. Clients using mobile phones tend to have their IPs rapidly changed in sequence. I didn't mention databases, so I can't comment on that point.

valenterry 2y ago

Using websockets with graphql, I feel like a lot of the challenges are then already solved. From the post: - Observability: WebSockets are more stateful, so you need to implement additional logging and monitoring for persistent connections: solved with graphql if the existing monitoring is already sufficient. - Authentication: You need to implement a new authentication mechanism for incoming WebSocket connections: solved with graphql. - Infrastructure: You need to configure your infrastructure to support WebSockets, including load balancers and firewalls: True, firewalls need to be updated. - Operations: You need to manage WebSocket connections and reconnections, including handling connection timeouts and errors: normally already solved by the graphql library. For errors, it's basically the same though. - Client Implementation: You need to implement a client-side WebSocket library, including handling reconnections and state management: Just have to use a graphql library that comes with websocket support (I think most of them do) and configure it accordingly.

anonzzzies 2y ago
I hope (never needed this) client implementations that do this all for you and pick the best implementation based on what the client supports? Not sure why the transport is interesting when/if you have freedom to choose.
- josephg 2y ago
  Yeah there’s plenty of high quality websocket client libraries in all languages now. Support and feature are excellent. And they’ve been supported in all browsers for a decade or something at this point. I vomit in my mouth a bit whenever people reach for socket.io or junk like that. You don’t want or need the complexity and bugs these libraries bring. They’re obsolete.
_heimdall 2y ago
Using graphql comes with IRS own list of challenges and issues though. Its a good solution for some situations, but it isn't so universal that you can just switch to it without a problem.

feverzsj 2y ago

Why not just use chunked encoding and get rid of extra requests.

justinl33 2y ago

yeah, authentication complexity with WebSockets is severely underappreciated. We ran into major RBAC headaches when clients needed to switch between different privilege contexts mid-session. Long polling with standard HTTP auth patterns eliminates this entire class of problems.

watermelon0 2y ago
Couldn't you just disconnect and reconnect websocket if privileges change, since the same needs to be done with the long polling?
- josephg 2y ago
  Yeah, and you can send cookies in the websocket connection headers. This used to be a problem in some browsers iirc - they wouldn’t send cookies properly over websocket connection requests. As a workaround in one project I wrote JavaScript code which manually sent cookies in the first websocket message from the client as soon as a connection opened. But I think this problem is now solved in all browsers.

baumschubser 2y ago

I like long polling, it’s easy to understand from start to finish and from client perspective it just works like a very slow connection. You have to keep track of retries and client-side cancelled connections to have one but only one (and the right one) of requests at hand to answer to. One thing that seems clumsy in the code example is the loop that queries the data again and again. Would be nicer if the data update could also resolve the promise of the response directly.

moribvndvs 2y ago
You could have your job status update push an update into an in-memory or distributed cache and check that in your long poll rather than a DB lookup, but that may require adding a bunch of complexity to wire the completion of the task to updating said cache. If your database is tuned well and you don’t have any other restrictions (e.g. serverless where you pay by the IO), it may be good enough and come out in the wash.
josephg 2y ago
Hard disagree. Long polling can have complex message ordering problems. You have completely different mechanisms for message passing from client-to-server and server-to-client. And middle boxes can and will stall long polled connections, stopping incremental message delivery. (Or you can use one http query per message - but that’s insanely inefficient over the wire). Websockets are simply a better technology. With long polling, the devil is in the details and it’s insanely hard to get those details right in every case.
- _nalply 2y ago
  One of them 2001 was that Netscape didn't render correctly if the connection is still open. Hah. I am sure this issue has been fixed a long, long time ago, but perhaps there are other issues. Nowadays I prefer SSE to long polling and websockets. The idea is: the client doesn't know that the server has new data before it makes a request. With a very simple SSE the client is told that new data is there then it can request new data separately if it wants. This said, SSE has a few quirks, one of them that on HTTP/1 the connection counts to the maximum limit of 6 concurrent connections per browser and domain, so if you have several tabs, you need a SharedWorker to share the connection between the tabs. But probably this quirk also appllies to long polling and websockets. Another quirk, SSE can't transmit binary data and has some limitations in the textual data it represents. But for this use case this doesn't matter. I would use websockets only if you have a real bidirectional data flow or need to transmit complex data.

peheje 2y ago

What about HTTP/2 Multiplexing, how does it hold up against long-polling and websockets? I have only tried it briefly when we use gRPC: https://grpc.io/docs/what-is-grpc/core-concepts/#server-streaming-rpc https://grpc.io/docs/what-is-grpc/core-concepts/#server-stre... Here it's easy to specify that a endpoint is a "stream", and then the code-generation tool gives all tools really to just keep serving the client with multiple responses. It looks deceptively simple. We already have setup auth, logging and metrics for gRPC, so I hope it just works off of that maybe with minor adjustments. But I'm guessing you don't need the gRPC layer to use HTTP/2 Multiplexing?

toast0 2y ago
At least in a browser context, HTTP/2 doesn't address server to client unsolicitied messages. So you'd still need a polling request open from the client. HTTP/2 does specify a server push mechanism (PUSH_PROMISE), but afaik, browsers don't accept them and even if they did, (again afaik) there's no mechanism for a page to listen for them. But if you control the client and the server, you could use it.
yencabulator 2y ago
gRPC as specced to ride directly on top of HTTP/2 doesn't work from browsers, the sandboxed JS isn't allowed that level of control over the protocol. And often is too low level to implement as part of a pre-existing HTTP server, too. gRPC is a server-to-server protocol that is not part of the usual Web, but happens to repurpose HTTP/2. Outside of gRPC, just HTTP POST cannot at this time replace websockets because the in-browser `fetch` API doesn't support streaming request body. For now, websockets is the only thing that can natively provide an ordered stream of messages from browser to server.
BitPirate 2y ago
With RFC8441 websockets are just HTTP/2 streams.

bigbones 2y ago

I don't know how meaningful it is any more, but with long polling with a short timeout and a gracefully ended request (i.e. chunked encoding with an eof chunk sent rather than disconnection), the browser would always end up with one spare idle connection to the server, making subsequent HTTP requests for other parts of the UI far more likely to be snappier, even if the app has been left otherwise idle for half the day I guess at least this trick is still meaningful where HTTP/2 or QUIC aren't in use

ipnon 2y ago

Articles like this make me happy to use Phoenix and LiveView every day. My app uses WebSockets and I don’t think about them at all.

leansensei 2y ago
Same here. It truly is a godsend.
zacksiri 2y ago
I was thinking this exact thing as I was reading the article.
tzumby 2y ago
I came here to say exactly this! Elixir and OTP (and by extension LiveView) are such a good match for the problem described in the post.
- j45 2y ago
  I was kind of wondering how something hadn't solve this at all, compared to a solution not readily being on one's path.
cultofmetatron 2y ago
hah seriously. my app uses web sockets extensively but since we are also using Phoenix, its never been source of conflict in development. it really was just drop it and scale to thousands of users.
- arrty88 2y ago
  Why couldn’t nodejs with uWS library or golang + gorilla handle 10s of thousands of connections?
  - apitman 2y ago
    I think GP's point is that they feel Phoenix is simpler to use than alternatives, not necessarily that it scales better.

sgarland 2y ago

The full schema isn’t listed, but the indices don’t make sense to me. (id, cluster_id) sounds like it could / should be the PK If the jobs are cleared once they’ve succeeded, and presumably retried if they’ve failed or stalled, then the table should be quite small; so small, that a. The query planner is unlikely to use the partial index on (status) b. The bloat from the rapidity of DELETEs likely overshadows the live tuple size.

DougN7 2y ago

I implemented a long polling solution in desktop software over 20 years ago and it’s still working great. It can even be used as a tunnel to stream RDP sessions, through which YouTube can play without a hiccup. Big fan of long polling, though I admit I didn’t get a chance to try web sockets back then.

jclarkcom 2y ago
I did the same, were you at VMware by any chance? At the time it was the only way to get comparability with older browsers.

deleted 2y ago

[deleted]

mojuba 2y ago

Can someone explain why TTL = 60s is a good choice? Why not more, or less?

notatoad 2y ago
i can't speak for why the author chose it, but if you're operating behind AWS cloudfront then http requests have a maximum timeout of 60s - if you don't close the request within 60s, cloudfront will close it for you. i suspect other firewalls, cdns, or reverse proxy products will all do something similar. for me, this is one of the biggest benefits of websockets over long-polling: it's a standard way to communicate to proxies and firewalls "this connection is supposed to stay open, don't close it on me"

k__ 2y ago

Half-OT: What's the most resource efficient way to push data to clients over HTTP? I can send data to a server via HTTP request, I just need a way to notify a client about a change and would like to avoid polling for it. I heard talk about SSE, WebSockets, and now long-polling. Is there something else? What requires the least resources on the server?

mojuba 2y ago
I don't think any of the methods give any significant advantage since in the end you need to maintain a connection per each client. The difference between the methods boils down to complexity of implementation and reliability. If you want to reduce server load then you'd have to sacrifice responsiveness, e.g. you perform short polls at certain intervals, say 10s.
- k__ 2y ago
  Okay, thanks. What's the least complex to implement then?
  - mojuba 2y ago
    For the browser and if you need only server-to-client sends, I assume SSE would be the best option. For other clients, such as mobile apps, I think long poll would be the simplest.

amelius 2y ago

> Corporate firewalls blocking WebSocket connections was one of our other worries. Some of our users are behind firewalls, and we don't need the IT headache of getting them to open up WebSockets. Don't websockets look like ordinary https connections?

doublerabbit 2y ago
It does. However DPI firewalls look at and block the upgrade handshake. Connection: Upgrade Upgrade: websocket
toast0 2y ago
Some corporate firewalls MITM all https connections. Websocket does not look normal once you've terminated TLS.
- amelius 2y ago
  Can websites detect this?
  - toast0 2y ago
    AFAIK, only by symptoms. If https fetches work and websockets don't, that's a sign. HSTS and assorted reporting can help a bit in aggregate, but not if the corporate MITM CA has been inserted into the browser's trusted CA list. I don't think there's an API to get certificate details from the browser side to compare. A proxy may have a different TLS handshake than a real browser would, depending on how good the MITM is, but the better they are, the more likely it is that websockets work.

yuliyp 2y ago

I think this article is tying a lot of unrelated decisions to "Websocket" vs "Long-polling" when they're actually independent. A long-polling server could handle a websocket client with just a bit of extra work to handle keep-alive. For the other direction, to support long-polling clients if your existing architecture is websockets which get data pushed to them by other parts of the system, just have two layers of servers: one which maintains the "state" of the connection, and then the HTTP server which receives the long polling request can connect to the server that has the connection state and wait for data that way.

harrall 2y ago
It sounded like the author(s) just had existing request-oriented code and didn’t want to rewrite it to be connection-oriented. Personally I would enjoyed solving that problem instead of hacking around it but that’s me.
lunarcave 2y ago
Author here. Having done this, I don't think I'd reduce it to "just a little bit of work" to make it hum in production. Everything in between your UI components and the database layer needs to be reworked to work in the connection-oriented (Websockets) model of the world vs request-oriented world.
- bythreads 2y ago
  It's a good pattern to have a vanilla js network manager / layer in fe for this exact reason - makes swapping network technologies a lot simpler. Only that knows url for endpoints, protocols and connections - and proxies between them and your app / components
- yuliyp 2y ago
  > Everything in between your UI components and the database layer needs to be reworked to work in the connection-oriented (Websockets) model of the world vs request-oriented world. How so? As a minimal change, the thing on the server end of the websocket could just do the polling of your database on its own while the connection is open (using authorization credentials supplied as the websocket is being opened). If the connection dies, stop polling. This has the nice property that you're in full control of the refresh rate, can implement coordinated backoffs if the database is overloaded, etc.

vitus 2y ago

I would appreciate if the article spent more time actually discussing the benefits of websockets (and/or more modern approaches to pushing data from server -> browser) and why the team decided those benefits were not worth the purported downsides. I could see the same simplicity argument being applied to using unencrypted HTTP/1.1 instead of HTTP/2, or TCP Reno instead of CUBIC. The section at the end talking about "A Case for Websockets" really only rehashes the arguments made in "Hidden Benefits of Long-Polling" stating that you need to reimplement these various mechanisms (or just use a library for it). My experience in this space is from 2011, when websockets were just coming onto the scene. Tooling / libraries were much more nascent, websockets had much lower penetration (we still had to support IE6 in those days!), and the API was far less stable prior to IETF standardization. But we still wanted to use them when possible, since they provided much better user experience (lower latency, etc) and lower server load.

tguvot 2y ago

Another reason: there is a patent troll suing companies over usage of websockets.

vouwfietsman 2y ago

The points mentioned against websockets are mostly fud, I've used websockets in production for a very heavy global data streaming application, and I would respond the following to the "upsides" of not using websockets: > Observability Remains Unchanged Actually it doesn't, many standard interesting metrics will break because long-polling is not a standard request either. > Authentication Simplicity Sure, auth is different than with http, but not more difficult. You can easily pass a token. > Infrastructure Compatibility I'm sure you can find firewalls out there where websockets are blocked, however for my use case I have never seen this reported. I think this is outdated, for sure you don't need "special proxy configurations or complex infrastructure setups". > Operational Simplicity Restarts will drop any persistent connection, state can be both or neither in WS or in LP, it doesn't matter what you use. > Client implementation It mentions "no special WebSocket libraries needed" and also "It works with any HTTP client". Guess what, websockets will work with any websocket client! Who knew! Finally, in the conclusion: > For us, staying close to the metal with a simple HTTP long polling implementation was the right choice Calling simple HTTP long polling "close to the metal" in comparison to websockets is weird. I wouldn't be surprised if websockets scale much better and give much more control depending on the type of data, but that's besides the point. If you want to use long polling because you prefer it, go ahead. Its a great way to stick to request/response style semantics that web devs are familiar with. Its not necessary to regurgitate a bunch of random hearsay arguments that may influence people in the wrong way. Try to actually leave the reader with some notion of when to use long polling vs when to use websockets, not a post-hoc justification of your decision based on generalized arguments that do not apply.

amatuer_sodapop 2y ago
> > Observability Remains Unchanged > Actually it doesn't, many standard interesting metrics will break because long-polling is not a standard request either. As a person who works in a large company handling millions of websockets, I fundamentally disagree with discounting the observability challenges. WebSockets completely transform your observability stack - they require different logging patterns, new debugging approaches, different connection tracking, and change how you monitor system health at scale. Observability is far more than metrics, and handwaving away these architectural differences doesn't make the implementation easier.

Cort3z 2y ago

I think they are mixing some problems here. They could probably have used their original setup with Postgres NOTIFY+triggers in stead of polling, and only have one "pickup poller" to catch any missed events/jobs. In my opinion transaction medium should not be linked to how the data is manage internally, but I know from experience that this separation is often hard to achieve in practice.

emilio1337 2y ago

The article does discuss a lot of mixed concepts. I would prefer one process polling new jobs/state and one process handling http connections/websockets. Hence no flooding the database and completely scalable from the client side. The database process pushes everything downstream via some queue while the other process/server handles those and sends them to respective clients

imglorp 2y ago

Since the article mentioned Postgres by name, isn't this a case for using its asynchronous notification features? Servers can LISTEN to a channel and PG can TRIGGER and NOTIFY them when the data changes. No polling needed, regardless of the frontend channel.

lunarcave 2y ago
Yes, but the problems of detecting that changeset and delivering it to the right connection remains to be solved in the app layer.
cluckindan 2y ago
It would be easier to run Hypermode’s Dgraph as the database and use GraphQL subscriptions from the frontend. But nobody ever got fired for choosing postgres.
- j45 2y ago
  I have relatively recently taken steps towards Postgres from it's abiality to be at the center of so much until a project outgrows it. In terms of not getting fired - Postgres is a lot more innovative than most databases, and the insinuation of IBM. By innovative I mean uniquely putting in performance related items for the last 10-20 years.

wereHamster 2y ago

Unrelated to the topic in the article… await new Promise(resolve => setTimeout(resolve, 500)); In Node.js context, it's easier to: import { setTimeout } from "node:timers/promises"; await setTimeout(500);

treve 2y ago
Is that easier? The first snippet is shorter and works on any runtime.
- joshmanders 2y ago
  In the context of Node.js, where op said, yes it is easier. But it's a new thing and most people don't realize timers in Node are awaitable yet, so the other way is less about "works everywhere" and more "this is just what I know"
  - wereHamster 2y ago
    I guess most Node.js developers also don't realize that there's "node:fs/promises" so you don't have to use callbacks or manually wrap functions from "node:fs" with util.promisify(). Doesn't mean need to stick with old patterns forever. When I said 'in the context of Node.js' I meant if you are in a JS module where you already import other node: modules, ie. when it's clear that code runs in a Node.js runtime and not in a browser. Of course when you are writing code that's supposed to be portable, don't use it. Or don't use setTimeout at all because it's not guaranteed to be available in all runtimes - it's not part of the ECMA-262 language specification after all.
hombre_fatal 2y ago
I haven't used that once since I found out that it exists. I just don't see the point. It doesn't work in the browser and it shadows global.setTimeout which is confusing. Meanwhile the idiom works everywhere.
- joshmanders 2y ago

bob1029 2y ago

I think we could have the best of both worlds. E.g.: https://socket.io/docs/v4/how-it-works/#upgrade-mechanism https://socket.io/docs/v4/how-it-works/#upgrade-mechanism

j45 2y ago
I wonder if people only look for solutions in their language when a tech like websockets is language independent.

rednafi 2y ago

Neither Server-Sent Events nor WebSockets have replaced all use cases of long polling reliably. The connection limit of SSE comes up a lot, even if you’re using HTTP/2. WebSockets, on the other hand, are unreliable as hell in most environments. Also, WS is hard to debug, and many of our prod issues with WS couldn’t even be reproduced locally. Detecting changes in the backend and propagating them to the right client is still an unsolved problem. Until then, long polling is surprisingly simple and a robust solution that works.

pas 2y ago
Robust WS solutions need a fallback anyway, and unless you are doing something like Discord long polling is a reasonable option.
infamia 2y ago
> The connection limit of SSE comes up a lot, even if you’re using HTTP/2. I'm considering using SSE for an app. I'm curious, what problems you've run into? At least the docs say you get 100 connections between the server and a client, but it can be negotiated higher if needed it seems? https://developer.mozilla.org/en-US/docs/Web/API/EventSource https://developer.mozilla.org/en-US/docs/Web/API/EventSource
- rednafi 2y ago
  SSEs are great and more reliable than websockets in a smaller scale. So I'd reach for it despite the issues. But that being said, some websevers don't play well with SSE and you'll need to fiddle with it. If you control the webserver, then it's not much of a problem.

Animats 2y ago

Long polling has some problems of its own. Second Life has an HTTPS long polling channel between client and server. It's used for some data that's too bulky for the UDP connection, not too time sensitive, or needs encryption. This has caused much grief. On the client side, the poller uses libcurl. Libcurl has timeouts. If the server has nothing to send for a while, libcurl times out. The client then makes the request again. This results in a race condition if the server wants to send something between timeout and next request. Messages get lost. On top of that, the real server is front-ended by an Apache server. This just passes through relevant requests, blocking the endless flood of junk HTTP requests from scrapers, attacks, and search engines. Apache has a timeout, and may close a connection that's in a long poll and not doing anything. Additional trouble can come from middle boxes and proxy servers that don't like long polling. There are a lot of things out there that just don't like holding an HTTP connection open. Years ago, a connection idle for a minute was fine. Today, hold a connection open for ten seconds without sending any data and something is likely to disconnect it. The end result is an unreliable message channel. It has to have sequence numbers to detect duplicates, and can lose messages. For a long time, nobody had discovered that, and there were intermittent failures that were not understood. In the original article, the chart section labelled "loop" doesn't mention timeout handling. That's not good. If you do long polling, you probably need to send something every few seconds to keep the connection alive. Not clear what a safe number is.

rednafi 2y ago
Yeah, some servers close connections when there’s no data transfer. When the backend holds the connection while polling the database until a timeout occurs or the database returns data, it needs to send something back to the client to keep the connection alive. I wonder what could be sent in this case and whether it would require special client-side logic.
- mhitza 2y ago
  In the HTTP model, technically status code 102 Processing would fit best. Though, no longer part of the HTTP specification. https://http.dev/102 https://http.dev/102 100 Continue could be usable as a workaround. Would probably require at a bare minimum some extra integration code on the client side.

sneak 2y ago

Given long polling, I have never ever understood why websockets became a thing. I’ve never implemented them and never will, it’s a protocol extension where none is necessary.

ekkeke 2y ago
Websockets can operate outside the request/response model used in this long polling example, and allow you to stream data continuously. They're also a lot more efficient in terms of framing and connections if there are a lot of individual pieces of data to push as you don't need to spin a up a connection + request for each bit.

LeicaLatte 2y ago

Long polling is my choice for simple, reliable and plug and play like interfaces. HTTP requests tend to be standard and simplify authentication as well. Systems with frequent but not constant updates are ideal. Text yes. Voice maybe not. Personal Case Study: I built mobile apps which used Flowise assistants for RAG and found websockets compeletely out of line with the rest of my system and interactions. Suddenly I was fitting a round peg in a square hole. I switched to OpenAI assistants and their polling system felt completely "natural" to integrate.

gloosx 2y ago

>Our system handles hundreds of worker nodes constantly polling our PostgreSQL-backed control plane for new jobs Does everybody poll their PosgreSQL to get new rows in real-time? This is really weird, there are trigger functions and notifications.

lunarcave 2y ago
In pg, each unique `listen(channel)` takes up server resources, and if you don't reliably clean them up, everything comes to a screeching halt. There's also `max_notify_queue_pages` >Specifies the maximum amount of allocated pages for NOTIFY / LISTEN queue. The default value is 1048576.