10 ms·
Server-Sent Events (SSE) Are Underrated
- recursivedoubts 2y agohttps://data-star.dev https://data-star.dev is a hypermedia-oriented front end library built entirely around the idea of streaming hypermedia responses via SSE. It was developed using Go & NATS as backend technologies, but works with any SSE implementation. Worth checking out if you want to explore SSE and what can be achieved w/it more deeply. Here is an interview with the author: https://www.youtube.com/watch?v=HbTFlUqELVc https://www.youtube.com/watch?v=HbTFlUqELVc
- sudodevnull 2y agoDatastar author here, happy to answer any questions!
- andersmurphy 2y ago+1 for recommending data-star. The combination of idiomorph (thank you), SSE and signals is fantastic for making push based and/or multiplayer hypermedia apps.
- ramon156 2y agoThey're underrated when they work™ Currently at work I'm having issues because - Auth between an embedded app and javascript's EventSource is not working, so I have to resort to a Microsoft package which doesn't always work. - Not every tunnel is fond of keep-alive (Cloudflare), so I had to switch to ngrok (until I found out they have a limit of 20k requests). I know this isn't the protocol's fault, and I'm sure there's something I'm missing, but my god is it frustrating.
- nchmy 2y agoTry this sse client https://github.com/rexxars/eventsource-client https://github.com/rexxars/eventsource-client
- condiment 2y agoSo it’s websockets, only instead of the Web server needing to handle the protocol upgrade, you just piggyback on HTTP with an in-band protocol. I’m not sure this makes sense in 2024. Pretty much every web server supports websockets at this point, and so do all of the browsers. You can easily impose the constraint on your code that communication through a websocket is mono-directional. And the capability to broadcast a message to all subscribers is going to be deceptively complex, no matter how you broadcast it.
- deleted 2y ago[deleted]
- realPubkey 2y agoYes most servers support websockets. But unfortunately most proxies and firewalls do not, especially in big company networks. Suggesting my users to use SSEs for my database replication stream solved most of their problems. Also setting up a SSE endpoint is like 5 lines of code. WebSockets instead require much more and you also have to do things like pings etc to ensure that it automatically reconnects. SEEs with the JavaScript EventSource API have all you need build in: https://rxdb.info/articles/websockets-sse-polling-webrtc-webtransport.html https://rxdb.info/articles/websockets-sse-polling-webrtc-web...
- the_mitsuhiko 2y agoSSE also works well on HTTP/3 whereas web sockets still don’t.
- apitman 2y agoI don't see much point in WebSockets for HTTP/3. WebTransport will cover everything you would need it for an more.
- the_mitsuhiko 2y agoThat might very well be but the future is not today.
- programmarchy 2y agoGreat post. I discovered SSE when building a chatbot and found out it’s what OpenAI used rather than WebSockets. The batteries-included automatic reconnection is huge, and the format is surprisingly human readable.
- dugmartin 2y agoIt doesn’t mention the big drawback of SSE as spelled out in the MDN docs: “Warning: When not used over HTTP/2, SSE suffers from a limitation to the maximum number of open connections, which can be especially painful when opening multiple tabs, as the limit is per browser and is set to a very low number (6).”
- k__ 2y agoAnd over HTTP/2 and 3 they are efficient?
- RadiozRadioz 2y agoThat is a very low number. I can think of many reasons why one would end up with more. Does anyone know why it is so low?
- apitman 2y agoHistorical reasons. The HTTP/1.1 spec actually recommends limited to 2 connections per domain. That said, I'm not sure why it's still so low. I would guess mostly to avoid unintended side effects of changing it.
- gsnedders 2y ago> The HTTP/1.1 spec actually recommends limited to 2 connections per domain. This is no longer true. From RFC 9112 § 9.4 (https://httpwg.org/specs/rfc9112.html#rfc.section.9.4 https://httpwg.org/specs/rfc9112.html#rfc.section.9.4): > Previous revisions of HTTP gave a specific number of connections as a ceiling, but this was found to be impractical for many applications. As a result, this specification does not mandate a particular maximum number of connections but, instead, encourages clients to be conservative when opening multiple connections.
- apitman 2y ago> Perceived Limitations: The unidirectional nature might seem restrictive, though it's often sufficient for many use cases For my use cases the main limitations of SSE are: 1. Text-only, so if you want to do binary you need to do something like base64 2. Browser connection limits for HTTP/1.1, ie you can only have ~6 connections per domain[0] Connection limits aren't a problem as long as you use HTTP/2+. Even so, I don't think I would reach for SSE these days. For less latency-sensitive and data-use sensitive applications, I would just use long polling. For things that are more performance-sensitive, I would probably use fetch with ReadableStream body responses. On the server side I would prefix each message with a 32bit integer (or maybe a variable length int of some sort) that gives the size of the message. This is far more flexible (by allowing binary data), and has less overhead compared to SSE, which requires 7 bytes ("data:" + "\n\n") of overhead for each message. [0]: https://stackoverflow.com/a/985704 https://stackoverflow.com/a/985704
- nchmy 2y agoYou can do fetch and readable stream with SSE - here's an excellent client library for that https://github.com/rexxars/eventsource-client https://github.com/rexxars/eventsource-client
- nhumrich 2y agoReadableStream appears to be SSE without any defined standards for chunk separation. In practice, how is it any different from using SSE? It appears to use the same concept.
- tomsmeding 2y agoPresumably, ReadableStream does not auto-reconnect.
- kdunglas 2y agoA while ago I created Mercure: an open pub-sub protocol built on top of SSE that is a replacement for WebSockets-based solutions such as Pusher. Mercure is now used by hundreds of apps in production. At the core of Mercure is the hub. It is a standalone component that maintains persistent SSE (HTTP) connections to the clients, and it exposes a very simple HTTP API that server apps and clients can use to publish. POSTed updates are broadcasted to all connected clients using SSE. This makes SSE usable even with technologies not able to maintain persistent connections such as PHP and many serverless providers. Mercure also adds nice features to SSE such as a JWT-based authorization mechanism, the ability to subscribe to several topics using a single connection, events history, automatic state reconciliation in case of network issue… I maintain an open-source hub written in Go (technically, a module for the Caddy web server) and a SaaS version is also available. Docs and code are available on https://mercure.rocks https://mercure.rocks
- apitman 2y agoThe site mentions battery-efficiency specifically. I'm curious what features does Mercure provide in that direction?
- kdunglas 2y agoSSE/Mercure (as WebSockets) is much battery-efficient than polling (push vs poll, less bandwidth used). Additionally, on controlled environnements, SSE can use a « push proxy » to wake up the device only when necessary: https://html.spec.whatwg.org/multipage/server-sent-events.html#eventsource-push https://html.spec.whatwg.org/multipage/server-sent-events.ht...
- pests 2y agoIt comes down to all the extra bytes sent and processed (local and remote, and in flight) by long polling. SSE events are small while other methods might require multiple packets and all the needless headers throughout the stack, for example.
- Dren95 2y agoCool didn’t know this. I used a similar solution called Centrifugo for a while. It allows you to choose which transport to use (ws, sse, others) https://github.com/centrifugal/centrifugo https://github.com/centrifugal/centrifugo
- piccirello 2y agoI utilized SSE when building automatic restart functionality[0] into Doppler's CLI. Our api server would send down an event whenever an application's secrets changed. The CLI would then fetch the latest secrets to inject into the application process. (I opted not to directly send the changed secrets via SSE as that would necessitate rechecking the access token that was used to establish the connection, lest we send changed secrets to a recently deauthorized client). I chose SSE over websockets because the latter required pulling in additional dependencies into our Golang application, and we truly only needed server->client communication. One issue we ran into that hasn't been discussed is HTTP timeouts. Some load balancers close an HTTP connection after a certain timeout (e.g. 1 hour) to prevent connection exhaustion. You can usually extend this timeout, but it has to be explicitly configured. We also found that our server had to send intermittent "ping" events to prevent either Cloudflare or Google Cloud Load Balancing from closing the connection, though I don't remember how frequently these were sent. Otherwise, SSE worked great for our use case. [0] https://docs.doppler.com/docs/automatic-restart https://docs.doppler.com/docs/automatic-restart
- Xenoamorphous 2y agoI also used SSE 6 or so years ago, and had the same issue with out load balancer; a bit hacky but what I did was to set a timer that would send a single colon character (which is the comment delimiter IIRC) periodically to the client. Is that what you meant by “ping”?
- apitman 2y agoGenerally you're going to want to send ping events pretty regularly (I'd default to every 15-30 seconds depending on application) whether you're using SSE, WebSockets, or something else. Otherwise if the server crashes the client might not know the connection is no longer live.
- sabareesh 2y agoYeah with cloudflare you need to do it every 30 seconds as the timeout is is 60 seconds
- yu3zhou4 2y agoI've had no idea they exist until I began to use APIs serving LLM outputs. They work pretty well for this purpose from my experience. An alternative to SSE is web sockets for this purpose I suppose
- sbergjohansen 2y agoPrevious related discussion (2022): https://news.ycombinator.com/item?id=30403438 https://news.ycombinator.com/item?id=30403438 (100 comments)
- schmichael 2y agoI’ve never understood the use of SSE over ndjson. Builtin browser support for SSE might be nice, but it seems fairly easy to handle ndjson? For non-browser consumers ndjson is almost assuredly easier to handle. ndjson works over any transport from HTTP/0.9 to HTTP/3 to raw TCP or unix sockets or any reliable transport protocol.
- apitman 2y agoManually streaming a XHR and parsing the messages is significantly more work, and you lose the built-in browser API. But if you use a fetch ReadableStream with TLV messages I'm sold.
- nchmy 2y agoHere's SSE with fetch and streams https://github.com/rexxars/eventsource-client https://github.com/rexxars/eventsource-client
- Tiberium 2y agoOne thing I dislike regards to SSE, which is not its fault but probably a side effect of the perceived simplicity: lots of developers do not actually use proper implementations and instead just parse the data chunks with regex, or something of the sorts! This is bad because SSE, for example, supports comments (": text") in streams, which most of those hand-rolled implementations don't support. For example, my friend used an LLM proxy that sends keepalive/queue data as SSE comments (just for debugging mainly), but it didn't work for Gemini, because someone at Google decided to parse SSE with a regex: https://github.com/google-gemini/generative-ai-js/blob/main/src/requests/stream-reader.ts#L28 https://github.com/google-gemini/generative-ai-js/blob/main/... (and yes, if the regex doesn't match the complete line, the library will just throw an error)
- Tiberium 2y agoAlso, another day, another mostly AI-written article on HN's top page :)
- emmanueloga_ 2y agoIt’s funny how HN has a mix of people who think AGI is just around the corner, people trying to build/sell stuff that uses LLMs, and others who can’t stand LLM-generated content. Makes me wonder how much overlap there is between these groups.
- Tiberium 2y agoI don't have anything against LLMs, I use them daily myself, but publishing content that's largely AI-generated without a disclaimer just feels dishonest to me. Oh, and also when people don't spend at least some effort to make the style more natural, not those bullet point lists in the article that e.g. Claude loves so much.
- remram 2y agoThose are not incompatible positions at all. You can think great AI is around the corner and still dislike today's not-great AI writing.
- slow_typist 2y agoWhat makes you think the article is AI-written?
- Tiberium 2y agoI've just spent too much time with different LLMs, and for example Claude really loves such bullet point lists. The article is full of them. The whole structure with numbered sections really gives it away, humans don't write blog posts like that.
- abnercoimbre 2y agoI remember reading the post and feeling it was inorganic. After your comment I re-read everything and you're correct: this is a very common template for Claude type answers. LLM-generated comments on HN are against the guidelines. Would the same apply to blog post submissions?
- whatever1 2y agoCan Django with vanilla gunicorn do this ?
- mfalcao 2y agoYes, I’ve done it using StreamingHttpResponse. You’ll want to use an asynchronous worker type though.
- upghost 2y agoDoes anyone have a good trick for figuring out when the client side connection is closed? I just kill the connection on the server every N minutes and force the client to reconnect, but it's not exactly graceful. Secondly, on iOS mobile, I've noticed that the EventSource seems to fall asleep at some point and not wake up when you switch back to the PWA. Does anyone know what's up with that?
- jesprenj 2y agoSend a dummy event and see if you get an ACK in response. Depends on the library you're using.
- upghost 2y agoThere's no ack on a raw SSE stream, unfortunately -- unless you mean send an event and expect the client to issue an HTTP request to the server like a keepalive?
- jauco 2y agoThere should be an ACK on the tcp packet (IIRC it’s not a lateral ACK but something like it) and the server should handle a timeout on that as the connection being “closed” which can be returned to the connection opener. You might want to look into timeouts or error callbacks on your connection library/framework.
- upghost 2y agoInteresting, hadn't checked at the TCP level. Will need to look into that.
- jauco 2y agoI remembered wrong. In most circumstances a tcp connection will be gracefully terminated by sending a FIN message. The timeout I talked about is on an ACK for a keepalive message. So after x time of not receiving a keepalive message the connection is closed. This handles cases where a connection is ungracefully dropped. All this is done at the kernel level, so at the application level you should be able to just verify if the connection is open by trying a read from the socket.
- henning 2y agoThey are handy for implementing simple ad-hoc hot reloading systems as well. E.g. you can have whatever file watcher you are using call an API when a file of interest changes that sends an event to listening clients on the frontend. You can also trigger an event after restarting the backend if you make an API change by triggering the event at boot time. Then you can just add a dev-only snippet to your base template that reloads the page or whatever. Better than nothing if your stack doesn't support it out of the box and doesn't take very much code or require adding any additional project dependencies. Not as sophisticated as React environments that will only reload a component that changed and only do a full page refresh if needed, but it still gives a nice, responsive feeling when paired with tools that recompile your backend when it changes.
- hamandcheese 2y agoI tried implementing SSE in a web project of mine recently, and was very surprised when my website totally stopped working when I had more than 6 tabs open. It turns out, Firefox counts SSE connections against the 6 host max connections limit, and gives absolutely no useful feedback that it's blocking the subsequent requests due to this limit (I don't remember the precise error code and message anymore, but it left me very clueless for a while). It was only when I stared at the lack of corresponding server side logs that it clicked. I don't know if this same problem happens with websockets or not.
- uncomplexity_ 2y agowait let's check this https://news.ycombinator.com/item?id=42511562 https://news.ycombinator.com/item?id=42511562 at https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events https://developer.mozilla.org/en-US/docs/Web/API/Server-sent... it says "Warning: When not used over HTTP/2, SSE suffers from a limitation to the maximum number of open connections, which can be especially painful when opening multiple tabs, as the limit is per browser and is set to a very low number (6). The issue has been marked as "Won't fix" in Chrome and Firefox. This limit is per browser + domain, which means that you can open 6 SSE connections across all of the tabs to www.example1.com and another 6 SSE connections to www.example2.com (per Stack Overflow). When using HTTP/2, the maximum number of simultaneous HTTP streams is negotiated between the server and the client (defaults to 100)." so the fix is just use http/2 on server-side?
- remram 2y agoOr a SharedWorker that creates a single SSE connection for all your tabs. SharedWorker is not very complicated but it's another component to add. It would be cool if this was built into SSE instead.
- uncomplexity_ 2y agookay wtf this is amazing, seems usable with websockets too. usage of Shared Web Workers https://dev.to/ayushgp/scaling-websocket-connections-using-shared-workers-14mj https://dev.to/ayushgp/scaling-websocket-connections-using-s... caniuse Shared Web Workers 45% https://caniuse.com/sharedworkers https://caniuse.com/sharedworkers caniuse BroadcastChannel 96% https://caniuse.com/broadcastchannel https://caniuse.com/broadcastchannel
- _caw 2y ago> SSE works seamlessly with existing HTTP infrastructure This is false. SSE is not supported on many proxies, and isn't even supported on some common local proxy tooling.
- ksajadi 2y agoI'm curious as to how everyone deals with HTTP/2 requirements between the backend servers and the load balancer? By default, HTTP/2 requires TLS which means either no SSL termination at the load balancer or a SSL cert generated per server with a different one for the front end load balancer. This all seems very inefficient.
- kcb 2y agoNot sure how widespread this is but AWS load balancers don't validate the backend cert in any way. So I just generate some random self signed cert and use it everywhere.
- nhumrich 2y agoYou don't need http2 on the actual backend. All limitations for SSE/http1 are browser level. Just downgrade to http1 from the LB to backend, even without SSL. As long as LB to browser is http2 you should be fine.
- ksajadi 2y agoIsn't that going to affect the whole multiplexing / multiple connection of SSEs?
- seabrookmx 2y agoI don't think so? HTTP 1.1 still supports Keep-Alive.
- nhumrich 2y agoNo. That's all handled in the browser and load balancer.
- est 2y agoI built several internal tool to tail logs using SSE with Flask/FastAPI. Easy to implement and maintain. For FastAPI if you want some hooks when client disconnects aka nginx 499 errors, follow this simple tip https://github.com/encode/starlette/discussions/1776#discussioncomment-3207518 https://github.com/encode/starlette/discussions/1776#discuss...
- fitsumbelay 2y agoFinding use cases for SSE and reading about others doing the same brings me great joy. Very easy to set up -- you just set 2 or 3 response headers and off you go. I have a hard time imagining the tech's limits outside of testing scenarios so some of the examples brought up here are interesting
- crowdyriver 2y agohttp streaming is even more underrated.
- anshumankmr 2y agoSSE is not underrated. In fact it's being used by Open AI for streaming completions. It's just not always needed unlike the very obvious use cases for normal REST APIs and Websockets. It was a pain to figure out how to get it to work in a ReactJS codebase I was working on then and from what I remember Axios didn't support it then so I had to use native fetch to get it to work.
- jpc0 2y agoHow long ago was this? I seem to remember not having too many issue with useEffect and context on this. Maybe the issue is you wanted to implement it in a singular react component when in reality you should be treating it like an other state library since it is something long lived that should live outside of react and pass data into react.
- anshumankmr 2y agoPretty much a year and a half back (I think it was March 2023). We had a real complicated set up back then, since I couldn't put our Open AI client key on the client side so I wrote an end point to to call Open AI's GPT3.5 API and then send that back to the front end to get the "typewriter" effect that they had on the frontend. It was quite broken back then cause sometimes random artifacts used to pop up in response, and some chunks came along with one another requiring me to write some really convoluted deserializing logic for it.
- lakomen 2y agoNo they're not. They're limited to 6 clients per browser per domain on http/1.1 Which is important because nginx can't reverse proxy http/2 or higher, so you end up with very weird functionality, essentially you can't use nginx with SSE. https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#sect1 https://developer.mozilla.org/en-US/docs/Web/API/Server-sent... Edit: I see someone already posted about that
- benterix 2y agoThe topic is interesting but the ChatGPT style of presenting information as bullet points is tiring.
- aniketchopade 2y agoI had some trouble implementing when server has to wait for another endpoint (webhook) to feed output to browser . During request processing (thread1) had to store the sse context in native Java object which will be retrieved later when webhook(thread2) is called But then with multiple service instance you wouldn't know which service had stored it so webhook had to publish something which others has to subscribe.
- RevEng 2y agoOpenAI's own documentation makes note of how difficult it is to work with SSE and to just use their library instead. My team wrote our own parser for these streaming events from an OpenAI compatible LLM server. The streaming format is awful. The double newline block separator also shows up in a bunch of our text, making parsing a nightmare. The "data:" signifier is slightly better, but when working with scientific software, it still occurs too often. Instead we've had to rely on the totally-not-reliable fact that the server returns each as a separate packet and the receiving end can be set up to return each packet in the stream. The suggestions I've found online for how to deal with the newline issue are to fold together consecutive newlines, but this loses formatting of some documents and otherwise means there is no way to transmit data verbatim. That might be fine for HTML or other text formats where newlines are pretty much optional, but it sucks for other data types. I'm happy to have something like SSE but the protocol needs more time to cook.
- skue 2y ago> I'm happy to have something like SSE but the protocol needs more time to cook. Just how well done do you like your protocols? SSE has been part of the WHATWG standard for almost 20 years. Every protocol requires some sort of data encoding. For SSE you need to either restrict yourself to payloads that can never conflict with the message structure (e.g. an enumeration of short strings to indicate different sorts of events), or you need to encode the data. It sounds like you are trying to send raw, unencoded data and are surprised that it sometimes conflicts with the message structure. Well of course it does! You can’t blame the protocol for that.
- RevEng 2y agoEvery other protocol I've used has a standard way to encode arbitrary data, but especially text data, usually using some kind of escape sequence. SSE does not. Just because it has been around for a long time does not mean it is well thought out or complete.
- deaf_coder 2y agoThe part where it says: > SSE works seamlessly with existing HTTP infrastructure: I'd be careful with that assumption. I have tried using SSE through some 3rd party load balancer at my work and it doesn't work that well. Because SSE is long-lived and doesn't normally close immediately, this load balancer will keep collecting and collecting bytes from the server and not forward it until server closes the connection, effectively making SSEs useless. I had to use WebSockets instead to get around this limitation with the load balancer.
- FZambia 2y agoYep, and in addition to that the ephemeral ports problem will araise at some scale with long-lived connections and infrastructure balancer/reverse proxy chain. So it's still required to tune.
- jpc0 2y agoI had a similar issue at one point but if I remember correctly I just had to have my webserver send the header section without closing the connection. Usually things would just get streamed through but for some reason until the full header was sent the proxy didn't forward and didn't acknowledge the connection. Not saying that is your issue but definitely was mine.
- sudhirj 2y agoNot entirely. If a load balancer is set to buffer say 4kb of data all the time, your SSE is stuck until you close the connection. I think there is a HTTP/2 flush instruction, but no load balancer is obligated to handle it and your SSE library might not be flushing anyway.
- deaf_coder 2y agoIn my case with this load balancer, I think it's just badly written. I think it is set to hold ALL data until the server ends the connection. I have tried leaving my SSE open to send over a few megabytes worth of data and the load balancer never forwarded it at all until I commanded the server to close the connection. The dev who wrote that code probably didn't think too much about memory efficiency of proxying HTTP connections or case of streaming HTTP connections like SSE.
- cle 2y ago> Automatic Reconnection I wouldn't characterize this as "automatic", you have to do a lot of manual work to support reconnection in most cases. You need to have a meaningful "event id" that can be resumed on a new connection/host somewhere else with the Last-Event-Id header. The plumbing for this event id is the trivial part IMO. The hard part is the server-side data synchronization, which is left as an exercise for the reader. Also, God help you if your SSE APIs have side effects. If the API call is involved in a sequence of side-effecting steps then you'll enter a world of pain by using SSE. Use regular HTTP calls or WebSockets. (Mostly b/c there's no cancellation ack, so retries are often racy.)
- junto 2y agoThis is why SignalR in .NET land is so popular in that ecosystem. It automatically falls back to SSE if web sockets aren’t available. In essence giving totally transparent real time communication.
- wlonkly 2y agoI love the format of this. I can tuck it away and whenever I need to think about SSE, refer back to it, ah, okay, now I'm all caught up. Also, surprised nobody's brought up PointCast[1] yet. Dotcom bubble, or ahead of their time? (Aside: while looking for a good reference link for Pointcast, I found an All Things Considered episode[2] about it from 1996!) [1] https://www.ecommerce-digest.com/early-dot-com-failure-case-studies.html#:~:text=Pointcast.Com https://www.ecommerce-digest.com/early-dot-com-failure-case-... [2] https://www.npr.org/1996/02/13/1042379/pointcast https://www.npr.org/1996/02/13/1042379/pointcast