5 ms·

Clojure: Realtime collaborative web apps without ClojureScript

172 points by bko 2y ago

sooheon 2y ago
To clarify, dealing with spotty connection / offline mode is not possible with this kind of approach?
- andriusbartulis 2y ago
  Offline mode - not possible. Spotty connection - no problem, SSE reconnects.
  - realharo 2y ago
    Reconnects eventually when things time out, but in the meantime, there will be a lot of frustrated waiting.
    - andersmurphy 2y ago
      In my experience it reconnects faster and more reliably than web sockets. Also nothing is stopping you telling the user the connection is down/retrying etc.
- andersmurphy 2y ago
  Also because of the compression being used the game runs fine on 3G connections.
nkh 2y ago
One thing I've always liked about the Clojure community, is that they are obessesed with state. I think they have correctly identified, that when things go sideways, it is because state management was involved in some way. This is a cool front end approach using datastar to put all of the state on the backend and side step front end state management entirely. There are some other really interesting things happening with electric clojure and a new "framework" called Replicant. Of all of them, Replicant seems the most intriguing to me personally. If it didn't exist, I think I would be trying to use datastar as this article outlines.
- fancyswimtime 2y ago
  single minded react user signing in. When dealing with state in react theres generally a good incentive not to model state exactly like the database model. Does Clojure removing this abstraction improve the application? I can see many pros but not knowledgable enough to see the potential shotgun
  - nkh 2y ago
    The Clojure ecosystem embraced react early and built on top of it (with OM, Reagent and Re-Frame (for SPAs). The UI = f(applicationState) is definitely viewed as the correct approach. In other words, gather all your application state in one place. Whenever it changes, pass all of it to a single function to produce the next version of your UI. Replicant takes this idea and runs with it, narrowing it down even further: The function is a pure function that returns the new UI as data. Replicant uses this representation to update the DOM accordingly. That’s it. No partial updates, no mutable objects, no network activity from UI components. It’s basically a templating system. You give it data, and it returns a view. Whether that data comes from a DB or any other place, it's just a clojure data structure. Here is the article that most of this comment is lifted from: https://replicant.fun/top-down/ https://replicant.fun/top-down/
    - deleted 2y ago
      [deleted]
    - enugu 2y ago
      Doesn't re-frame also work with ui as a pure function of state? (https://day8.github.io/re-frame/a-loop/ https://day8.github.io/re-frame/a-loop/) So, is the main difference that replicant is not built on top of react?
      - diggan 2y ago
        reagent (which you'd put re-frame on top of) is what handles rendering (via React) when we're using re-frame, and is the library you can say does "UI as a pure function of state". re-frame is basically like redux (over-simplification) but for the CLJS ecosystem, focuses solely on state and how to mutate it.
pm2222 2y ago
My summary: one single copy of state is maintained on server side and clients are updated through polling or SSE.
- andersmurphy 2y ago
  Basically. It's generating the html for the game state every 200ms (regardless of the number of connected users) and pushing it down to any connected clients via SSE. The magic is Brtoli + SSE + a properly sized context window means it runs well even on 3G connections (you can play with this in chrome dev under the network tab throttle option).
mike_ivanov 2y ago
But.. SSE is limited to like 6 or so connections per browser, no?
- millipede 2y ago
  Not with HTTP/2
- andriusbartulis 2y ago
  With HTTP/1 the limit is 6 per domain per browser. With HTTP/2 the limit is negotiated between the server and client, 100 by default. It's not a problem in practice, just ensure to use HTTP/2.
- andersmurphy 2y ago
  I'd also add even if for whatever reason you are on http1.1 (you should really use http2 for server driven apps there's a lot of other advantages). Datastar prunes connection based on the browser visibility API so on http1.1 a user would need 5-6 visible tabs/windows visible at the same time for you to hit any issues. Assuming you are using Datastar the way I do with a single persistent SSE connection. This feature is also much better for the battery life of devices and the server as you are only sending stuff if the user is there.
mike_ivanov 2y ago
BTW the demo app is broken: white screen with some Content-Security-Policy errors in the console.
- andersmurphy 2y ago
  It's embedded in an iframe on the blog, you can always try example.andersmurphy.com to access the game directly.
  - geokon 2y ago
    Sorry, I'm not a webdev so I'm not sure how to check this - but how large is the final downloaded page? (HTML+JS) The inspector in Firefox says it's 20MB, which seems large. But maybe I'm looking at it wrong
    - andersmurphy 2y ago
      There's is not final download it's a streaming game. It's live multiplayer. The initial bundle is 12kb (datastar + initial html + css). What you are seeing is the constant SSE connection streaming compressed data. Are you behind a corporate proxy? Also what version of firefox? Because it's working fine on firefox for me?
j13n 2y ago
The linked post relies on the Datastar project, which requires use of `unsafe-eval` in one’s Content-Security-Policy [1]: > When using a Content Security Policy (CSP), unsafe-eval must be allowed for scripts, since Datastar evaluates expressions using an IIFE (Immediately Invoked Function Expression). The project itself links to Mozilla’s docs on CSP, which state: > The unsafe-eval keyword can be used to override this behavior, and as with unsafe-inline, and for the same reasons: developers should avoid unsafe-eval. Out of the box, htmx uses a similar approach, but one can disable this use of eval [2]: htmx.config.allowEval - can be set to false to disable all features of htmx that rely on eval: - event filters - hx-on: attributes - hx-vals with the js: prefix - hx-headers with the js: prefix [1]: https://github.com/starfederation/datastar/blob/develop/site/static/md/reference/security.md#content-security-policy https://github.com/starfederation/datastar/blob/develop/site... [2]: https://htmx.org/docs/#configuration-options https://htmx.org/docs/#configuration-options
oldpersonintx 2y ago
[dead]
andersmurphy 2y ago
Author of the blog post here. Happy to answer any question!
andriusbartulis 2y ago
There are two main high-level approaches for architecting web application client state: 1. Client-heavy state management (separate client application) 2. Client-light state management (thin client) The state has to be tracked and managed somewhere. In most web applications, that state is persistable - at some point, it will end up in a DB of some kind. As such, we already have to handle it on the server side, regardless of whether we also choose to separately manage it on the client side or not. So the question at the end of the day is: Do I want to separately manage the application state on the client side, when I am already managing it on the server side? In a lot of situations, the pragmatic answer would be "No". Why? Because managing the state in two places causes duplication, synchronisation issues, diverging logic issues, and so on. We are increasing the accidental complexity of our solution. Maintainability becomes slower, riskier and more expensive. For some applications, the extra complexity might be worth it. In most web applications I have worked with (eCommerce, internal business tools, booking systems, online calculators, business dashboards), it is not. And "let's keep all state and processing on the server" does not anymore mean no real-time or highly interactive features. With SSE and fast HTML fragment merging, we can enjoy highly interactive and performant web applications with almost no client-side application state management.
- sesm 2y ago
  In eCommerce, saving the cart on client side and doing optimistic updates in case client network is unstable is a standard feature.
- dominicrose 2y ago
  It still means less real-time and less interactive and likely more costly. Separating front and server is not a bad thing even if it brings some kind of duplication. With .cljc files you can avoid the duplication anyway.
  - lukev 2y ago
    I've had quite my fill of "real time" and "interactive" web pages. Slow, bloated, terrible UX as often as not. Even Datastar/HTMX might be too much. Server-side HTML can get you very, very far in most applications. It's faster, lighter, and looks and feels cleaner. Case in point: do you really prefer a heavy, real time interactive message board like the default interface of Reddit? Because we're talking on HN, which is super fast, light, usable, and completely server-side.
andersmurphy 2y ago
Oh and something I forgot to mention in the blog post. - The server is in Europe. - The process can be modified at the REPL while It's running (and the changes get pushed out to all users by the SSE). So the board, HTML, CSS etc can all be changed on the fly without a restart.
lerp-io 2y ago
That's cool, but as a developer building web apps, I'm not competing with php websites, I'm competing with native apps.

172 points by bko 2y ago

sooheon 2y ago

To clarify, dealing with spotty connection / offline mode is not possible with this kind of approach?

andriusbartulis 2y ago
Offline mode - not possible. Spotty connection - no problem, SSE reconnects.
- realharo 2y ago
  Reconnects eventually when things time out, but in the meantime, there will be a lot of frustrated waiting.
  - andersmurphy 2y ago
    In my experience it reconnects faster and more reliably than web sockets. Also nothing is stopping you telling the user the connection is down/retrying etc.
andersmurphy 2y ago
Also because of the compression being used the game runs fine on 3G connections.

nkh 2y ago

One thing I've always liked about the Clojure community, is that they are obessesed with state. I think they have correctly identified, that when things go sideways, it is because state management was involved in some way. This is a cool front end approach using datastar to put all of the state on the backend and side step front end state management entirely. There are some other really interesting things happening with electric clojure and a new "framework" called Replicant. Of all of them, Replicant seems the most intriguing to me personally. If it didn't exist, I think I would be trying to use datastar as this article outlines.

fancyswimtime 2y ago
single minded react user signing in. When dealing with state in react theres generally a good incentive not to model state exactly like the database model. Does Clojure removing this abstraction improve the application? I can see many pros but not knowledgable enough to see the potential shotgun
- nkh 2y ago
  The Clojure ecosystem embraced react early and built on top of it (with OM, Reagent and Re-Frame (for SPAs). The UI = f(applicationState) is definitely viewed as the correct approach. In other words, gather all your application state in one place. Whenever it changes, pass all of it to a single function to produce the next version of your UI. Replicant takes this idea and runs with it, narrowing it down even further: The function is a pure function that returns the new UI as data. Replicant uses this representation to update the DOM accordingly. That’s it. No partial updates, no mutable objects, no network activity from UI components. It’s basically a templating system. You give it data, and it returns a view. Whether that data comes from a DB or any other place, it's just a clojure data structure. Here is the article that most of this comment is lifted from: https://replicant.fun/top-down/ https://replicant.fun/top-down/
  - deleted 2y ago
    [deleted]
  - enugu 2y ago
    Doesn't re-frame also work with ui as a pure function of state? (https://day8.github.io/re-frame/a-loop/ https://day8.github.io/re-frame/a-loop/) So, is the main difference that replicant is not built on top of react?
    - diggan 2y ago
      reagent (which you'd put re-frame on top of) is what handles rendering (via React) when we're using re-frame, and is the library you can say does "UI as a pure function of state". re-frame is basically like redux (over-simplification) but for the CLJS ecosystem, focuses solely on state and how to mutate it.

pm2222 2y ago

My summary: one single copy of state is maintained on server side and clients are updated through polling or SSE.

andersmurphy 2y ago
Basically. It's generating the html for the game state every 200ms (regardless of the number of connected users) and pushing it down to any connected clients via SSE. The magic is Brtoli + SSE + a properly sized context window means it runs well even on 3G connections (you can play with this in chrome dev under the network tab throttle option).

mike_ivanov 2y ago

But.. SSE is limited to like 6 or so connections per browser, no?

millipede 2y ago
Not with HTTP/2
andriusbartulis 2y ago
With HTTP/1 the limit is 6 per domain per browser. With HTTP/2 the limit is negotiated between the server and client, 100 by default. It's not a problem in practice, just ensure to use HTTP/2.
andersmurphy 2y ago
I'd also add even if for whatever reason you are on http1.1 (you should really use http2 for server driven apps there's a lot of other advantages). Datastar prunes connection based on the browser visibility API so on http1.1 a user would need 5-6 visible tabs/windows visible at the same time for you to hit any issues. Assuming you are using Datastar the way I do with a single persistent SSE connection. This feature is also much better for the battery life of devices and the server as you are only sending stuff if the user is there.

mike_ivanov 2y ago

BTW the demo app is broken: white screen with some Content-Security-Policy errors in the console.

andersmurphy 2y ago
It's embedded in an iframe on the blog, you can always try example.andersmurphy.com to access the game directly.
- geokon 2y ago
  Sorry, I'm not a webdev so I'm not sure how to check this - but how large is the final downloaded page? (HTML+JS) The inspector in Firefox says it's 20MB, which seems large. But maybe I'm looking at it wrong
  - andersmurphy 2y ago
    There's is not final download it's a streaming game. It's live multiplayer. The initial bundle is 12kb (datastar + initial html + css). What you are seeing is the constant SSE connection streaming compressed data. Are you behind a corporate proxy? Also what version of firefox? Because it's working fine on firefox for me?

j13n 2y ago

The linked post relies on the Datastar project, which requires use of `unsafe-eval` in one’s Content-Security-Policy [1]: > When using a Content Security Policy (CSP), unsafe-eval must be allowed for scripts, since Datastar evaluates expressions using an IIFE (Immediately Invoked Function Expression). The project itself links to Mozilla’s docs on CSP, which state: > The unsafe-eval keyword can be used to override this behavior, and as with unsafe-inline, and for the same reasons: developers should avoid unsafe-eval. Out of the box, htmx uses a similar approach, but one can disable this use of eval [2]: htmx.config.allowEval - can be set to false to disable all features of htmx that rely on eval: - event filters - hx-on: attributes - hx-vals with the js: prefix - hx-headers with the js: prefix [1]: https://github.com/starfederation/datastar/blob/develop/site/static/md/reference/security.md#content-security-policy https://github.com/starfederation/datastar/blob/develop/site... [2]: https://htmx.org/docs/#configuration-options https://htmx.org/docs/#configuration-options

oldpersonintx 2y ago

[dead]

andersmurphy 2y ago

Author of the blog post here. Happy to answer any question!

andriusbartulis 2y ago

There are two main high-level approaches for architecting web application client state: 1. Client-heavy state management (separate client application) 2. Client-light state management (thin client) The state has to be tracked and managed somewhere. In most web applications, that state is persistable - at some point, it will end up in a DB of some kind. As such, we already have to handle it on the server side, regardless of whether we also choose to separately manage it on the client side or not. So the question at the end of the day is: Do I want to separately manage the application state on the client side, when I am already managing it on the server side? In a lot of situations, the pragmatic answer would be "No". Why? Because managing the state in two places causes duplication, synchronisation issues, diverging logic issues, and so on. We are increasing the accidental complexity of our solution. Maintainability becomes slower, riskier and more expensive. For some applications, the extra complexity might be worth it. In most web applications I have worked with (eCommerce, internal business tools, booking systems, online calculators, business dashboards), it is not. And "let's keep all state and processing on the server" does not anymore mean no real-time or highly interactive features. With SSE and fast HTML fragment merging, we can enjoy highly interactive and performant web applications with almost no client-side application state management.

sesm 2y ago
In eCommerce, saving the cart on client side and doing optimistic updates in case client network is unstable is a standard feature.
dominicrose 2y ago
It still means less real-time and less interactive and likely more costly. Separating front and server is not a bad thing even if it brings some kind of duplication. With .cljc files you can avoid the duplication anyway.
- lukev 2y ago
  I've had quite my fill of "real time" and "interactive" web pages. Slow, bloated, terrible UX as often as not. Even Datastar/HTMX might be too much. Server-side HTML can get you very, very far in most applications. It's faster, lighter, and looks and feels cleaner. Case in point: do you really prefer a heavy, real time interactive message board like the default interface of Reddit? Because we're talking on HN, which is super fast, light, usable, and completely server-side.

andersmurphy 2y ago

Oh and something I forgot to mention in the blog post. - The server is in Europe. - The process can be modified at the REPL while It's running (and the changes get pushed out to all users by the SSE). So the board, HTML, CSS etc can all be changed on the fly without a restart.

lerp-io 2y ago

That's cool, but as a developer building web apps, I'm not competing with php websites, I'm competing with native apps.