5 ms·

The Journey from WebSockets to HTTP Streams

66 points by owulveryck 3y ago

freedomben 3y ago
This is a really great write-up! On a side note, the Remarkable Tablet is such a cool and hackable device. It surprises me that the company doesn't open source more of their client software to get help from the community, which I bet would do a large chunk of the work for them for free as it's mutually beneficial. Particularly as the market they are in is highly competitive, that could be a great way to get a great featureset, lower costs, while also providing something of immense value to the open source community. They could even offer a third-party app store of some sort and become the de-facto standard platform for e-ink devices! The rest are so locked down that this would be a huge competitive advantage.
- azeirah 3y ago
  ReMarkable recently got a new CEO, so who knows what's next? https://goodereader.com/blog/remarkable-news/remarkable-is-getting-a-new-ceo-to-take-digital-paper-to-the-next-level https://goodereader.com/blog/remarkable-news/remarkable-is-g...
  - stonogo 3y ago
    I guess things didn't work out at Signifier? https://signifiermedical.com/2020/09/02/signifier-medical-technologies-appoints-philip-hess-former-president-ceo-of-bose-corporation-as-chief-operating-officer/ https://signifiermedical.com/2020/09/02/signifier-medical-te...
MuffinFlavored 3y ago
Does "HTTP streams" mean "Server-Sent Events" here: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events https://developer.mozilla.org/en-US/docs/Web/API/Server-sent... ? I don't see it spelled out in the article. Edit: yes I do. > Internet and ChatGPT gave it a name: Server Sent Events Interesting callout: websockets can be "two way" (server -> client, client -> server). SSE, you just get one way (server -> client) and you have to do your typical POST to achieve the other way (client -> server)
- euroderf 3y ago
  Definite asymmetry in any actual deployment. htmx seems to be popular for the opposite direction. And fwiw http/2 mandates https but there is a workaround involving configuring a custom http.Transport .
voidwtf 3y ago
I would advise anyone finding inspiration in this post against using a similar solution in the real world. While this may work in a local only environment, once you encounter the real world consisting of Cloudflare, DPI/MITM proxies, load balancers, zscaler, etc... it would most likely fail to work in this manner. Standards are at least somewhat supported among the former, often not even correctly. Any solution you choose I'd suggest implementing a fall-back to the most basic of http methods (standard get/post and polling for instance). You'll find all types of problems once you start encountering middleware that just absolutely thrashes your expectations of a well functioning internet.
- lantastic 3y ago
  Interesting. Can you expand a bit on the kinds of problems that websockets would encounter with such middleware? I've seen a fair share of surprises on just TCP from DPI/MitM and high bandwidth-delay products, so I'm curious about ws analogues.
  - voidwtf 3y ago
    This article was not about using web sockets, it was about using a standard HTTP stream in a somewhat unorthodox way by never completing the request. Instead just beginning the response and occasionally "flush"ing data in the buffer to the client. The problem is that you'll often run into middleware that will do one of the following. I've personally encountered WAF proxies that will wait for the request to complete before sending any data to the client, that means that the client would appear to receive no data until the request timed out or the server completed the response. I've also encountered MITM/DPI and load-balancing proxies that will do the same, waiting until a buffer is filled and/or the response is completed.
- Matthias247 3y ago
  HTTP Streams (and Server Sent Events - which is a predefined body format that is understood among web browsers) are supported just fine by most infrastructure - probably even better than websockets. The advantage of them is that "they are just HTTP requests", so as long as your proxy actually can forward request and response bodies in a streaming fashion it will work. There's no need to understand the contents. It will not work if the proxy is implemented by waiting for the whole HTTP response to complete before forwarding the body. But that wouldn't work very well for a whole lot of other use-cases like file transfers, where buffering the whole body is not practical. A caveat is that most proxies won't allow for indefinite body body streaming, since they want to avoid all the TCP connections being tied up, so it's likely that the streaming would be interrupted at some time. In that case a reconnect logic would be necessary. But same applies to websockets.
riverdweller 3y ago
This approach was what we had to use back in the late 90s. We called it "iframe streaming", or "forever iframe" (and years later, as an industry term emerged, "comet"). It worked surprisingly well, except in cases where a client sat behind a greedily buffering proxy. We would send JavaScript statements that invoked callbacks on the client, rather than just JSON, as this avoided the need to parse data to determine which business logic to use on the client. This has the limitation of being "non-cross domain" (i.e. the web page containing the callback functions have to be served from the same domain as the infinite document). To get around buffering proxies, we optionally allowed our clients to use a JSONP long polling approach instead, whereby the client would dynamically generate a form inside a hidden iframe, and POST a request for JSONP data (JSON delivered wrapped in a callback), and the server would return data as soon as any was available. The client would immediately repeat the process to request more data, an infinitum. Eventually, the emergence of the XMLHTTPRequest object in IE (and subsequently in other browsers) allowed us to implement cleaner long-poll-style methods, holding the connection open until data was available (and automatically reconnecting on error). This was later enhanced with CORS for delivery of data from arbitrary domains. As support for detecting updates to an in-progress response became available (via XMLHTTPRequest's "progress" event, which for a long time was horribly buggy in IE ) our payloads became infinite streams too (of JavaScript callback invocations). Early versions of approach also required us to reload the entire page from time to time, as IE's underlying implementations of these browser objects appeared to have memory leaks (that we did not see in Firefox, for example). When IE8 was released, we allowed clients to optionally use its XDomainRequest object to stream a response instead. Years later, the much cleaner Server Sent Events (SSE) and WebSocket options became possible. Intermediate proxy support was initially troublesome however, and while both of these were our preferred choices from a performance and API perspective, it took several years before we could consider removing support for our earlier approaches. Even today, there are network environments where an infinite sequence of long polls is the only reliable option... My preference today? The JavaScript fetch API for sending commands, with a simple ack as a response, and an async flow of events over a persistent SSE connection, that feed into a simple JS message bus (implemented using the browser's native event API) for delivery to vanilla JavaScript web components. Simple, clean, and consistent.

66 points by owulveryck 3y ago

This is a really great write-up! On a side note, the Remarkable Tablet is such a cool and hackable device. It surprises me that the company doesn't open source more of their client software to get help from the community, which I bet would do a large chunk of the work for them for free as it's mutually beneficial. Particularly as the market they are in is highly competitive, that could be a great way to get a great featureset, lower costs, while also providing something of immense value to the open source community. They could even offer a third-party app store of some sort and become the de-facto standard platform for e-ink devices! The rest are so locked down that this would be a huge competitive advantage.

azeirah 3y ago
ReMarkable recently got a new CEO, so who knows what's next? https://goodereader.com/blog/remarkable-news/remarkable-is-getting-a-new-ceo-to-take-digital-paper-to-the-next-level https://goodereader.com/blog/remarkable-news/remarkable-is-g...
- stonogo 3y ago
  I guess things didn't work out at Signifier? https://signifiermedical.com/2020/09/02/signifier-medical-technologies-appoints-philip-hess-former-president-ceo-of-bose-corporation-as-chief-operating-officer/ https://signifiermedical.com/2020/09/02/signifier-medical-te...

MuffinFlavored 3y ago

Does "HTTP streams" mean "Server-Sent Events" here: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events https://developer.mozilla.org/en-US/docs/Web/API/Server-sent... ? I don't see it spelled out in the article. Edit: yes I do. > Internet and ChatGPT gave it a name: Server Sent Events Interesting callout: websockets can be "two way" (server -> client, client -> server). SSE, you just get one way (server -> client) and you have to do your typical POST to achieve the other way (client -> server)

euroderf 3y ago
Definite asymmetry in any actual deployment. htmx seems to be popular for the opposite direction. And fwiw http/2 mandates https but there is a workaround involving configuring a custom http.Transport .

voidwtf 3y ago

I would advise anyone finding inspiration in this post against using a similar solution in the real world. While this may work in a local only environment, once you encounter the real world consisting of Cloudflare, DPI/MITM proxies, load balancers, zscaler, etc... it would most likely fail to work in this manner. Standards are at least somewhat supported among the former, often not even correctly. Any solution you choose I'd suggest implementing a fall-back to the most basic of http methods (standard get/post and polling for instance). You'll find all types of problems once you start encountering middleware that just absolutely thrashes your expectations of a well functioning internet.

lantastic 3y ago
Interesting. Can you expand a bit on the kinds of problems that websockets would encounter with such middleware? I've seen a fair share of surprises on just TCP from DPI/MitM and high bandwidth-delay products, so I'm curious about ws analogues.
- voidwtf 3y ago
  This article was not about using web sockets, it was about using a standard HTTP stream in a somewhat unorthodox way by never completing the request. Instead just beginning the response and occasionally "flush"ing data in the buffer to the client. The problem is that you'll often run into middleware that will do one of the following. I've personally encountered WAF proxies that will wait for the request to complete before sending any data to the client, that means that the client would appear to receive no data until the request timed out or the server completed the response. I've also encountered MITM/DPI and load-balancing proxies that will do the same, waiting until a buffer is filled and/or the response is completed.
Matthias247 3y ago
HTTP Streams (and Server Sent Events - which is a predefined body format that is understood among web browsers) are supported just fine by most infrastructure - probably even better than websockets. The advantage of them is that "they are just HTTP requests", so as long as your proxy actually can forward request and response bodies in a streaming fashion it will work. There's no need to understand the contents. It will not work if the proxy is implemented by waiting for the whole HTTP response to complete before forwarding the body. But that wouldn't work very well for a whole lot of other use-cases like file transfers, where buffering the whole body is not practical. A caveat is that most proxies won't allow for indefinite body body streaming, since they want to avoid all the TCP connections being tied up, so it's likely that the streaming would be interrupted at some time. In that case a reconnect logic would be necessary. But same applies to websockets.

riverdweller 3y ago

This approach was what we had to use back in the late 90s. We called it "iframe streaming", or "forever iframe" (and years later, as an industry term emerged, "comet"). It worked surprisingly well, except in cases where a client sat behind a greedily buffering proxy. We would send JavaScript statements that invoked callbacks on the client, rather than just JSON, as this avoided the need to parse data to determine which business logic to use on the client. This has the limitation of being "non-cross domain" (i.e. the web page containing the callback functions have to be served from the same domain as the infinite document). To get around buffering proxies, we optionally allowed our clients to use a JSONP long polling approach instead, whereby the client would dynamically generate a form inside a hidden iframe, and POST a request for JSONP data (JSON delivered wrapped in a callback), and the server would return data as soon as any was available. The client would immediately repeat the process to request more data, an infinitum. Eventually, the emergence of the XMLHTTPRequest object in IE (and subsequently in other browsers) allowed us to implement cleaner long-poll-style methods, holding the connection open until data was available (and automatically reconnecting on error). This was later enhanced with CORS for delivery of data from arbitrary domains. As support for detecting updates to an in-progress response became available (via XMLHTTPRequest's "progress" event, which for a long time was horribly buggy in IE ) our payloads became infinite streams too (of JavaScript callback invocations). Early versions of approach also required us to reload the entire page from time to time, as IE's underlying implementations of these browser objects appeared to have memory leaks (that we did not see in Firefox, for example). When IE8 was released, we allowed clients to optionally use its XDomainRequest object to stream a response instead. Years later, the much cleaner Server Sent Events (SSE) and WebSocket options became possible. Intermediate proxy support was initially troublesome however, and while both of these were our preferred choices from a performance and API perspective, it took several years before we could consider removing support for our earlier approaches. Even today, there are network environments where an infinite sequence of long polls is the only reliable option... My preference today? The JavaScript fetch API for sending commands, with a simple ack as a response, and an async flow of events over a persistent SSE connection, that feed into a simple JS message bus (implemented using the browser's native event API) for delivery to vanilla JavaScript web components. Simple, clean, and consistent.