8 ms·

Electric Clojure – A signals DSL for full-stack web UI

330 points by Borkdude 4y ago

patates 4y ago
> There is no client/server dichotomy from the programmer's perspective But on the first example, I see a lot of e/client and e/server calls. Am I misunderstanding this? If so, how is this different from meteor.js?
- popinman322 4y ago
  Seems like e/client and e/server are for when you'd like to force sections of code to run on one side or the other. Otherwise the compiler decides what lives where?
  - kimi 4y ago
    Exactly. And when one side needs the other, it makes magic happen.
    - patates 4y ago
      Well then 2 humble suggestions: - Would be better to show an example where magic would happen - Selectable text instead of a screenshot would have been nicer
      - kimi 4y ago
        Magic does happen: you are rendering a table on the client, pause to fetch data on the server, and display it. And it looks like a PHP page written in Clojure.
kimi 4y ago
This is major - potentially killing off a thousand tons of boilerplate. Not sure if it is too late, because Elixir's LiveView seem to have gotten important because it works around the same issue. But it's time to push the wooden stake through the heart of the SPA (and the phone app too, but that's for another day).
- incrudible 4y ago
  You can already kill the boilerplate with code generation and metaprogramming. Once you have done that, you figure out why that is not always a good idea and how to best deal with the exceptions that your particular use case brings to the table.
  - kimi 4y ago
    I'm thinking more of all the cruft that goes to handle state on the client and keep it synced to the server in your React app. Redux and the gazillions of other packages which names sound like warthogs having breakfast.
    - incrudible 4y ago
      The same argument applies. I think Redux is a pointless indirection, but you can use metaprogramming to make it more palatable, at the cost of another indirection.
i_cannot_hack 4y ago
I don't know much about this kind of architecture, but wouldn't it be problematic from an infosec standpoint? It seems like it would be very easy to accidentally leak information from the server to the client in a subtle manner if the developer makes a mistake somewhere. And in ways which would not be obvious from reading the code unless you carefully consider the implications of each e/server and e/client with respect to the mentioned graph analysis? Are there any mitigations in place to prevent this, except the obvious "be careful and don't write bad code"? Or have I misunderstood things completely and it's a non-issue?
- prettyStandard 4y ago
  I tried to get hands-on experience with this hyper fiddle stuff a few months ago but I think it was still in closed beta. I would imagine you can inspect the network traffic. I would hope it would be readable so that you could have a good idea of what's going on and errors would be more obvious.
  - moondowner 4y ago
    I see a sample todo app in their repo, so yes, one can run it and check out how it works https://github.com/hyperfiddle/electric-starter-app https://github.com/hyperfiddle/electric-starter-app
- adlpz 4y ago
  Just as a data point, this is also true about very much noeadays "vanilla" stacks like Next.js. It seems like you're doing SSR but not exactly. You return the wrong prop in a getServerSideProps and boom, your stripe keys are dumped in a nice JSON for all to see.
  - reilly3000 4y ago
    Yikes!!! Is anyone aware of automated testing/scanning for secret leakage via js framework magic? If not, how would you implement one?
- 4y ago
yladiz 4y ago
Not to be a downer, but while I think in theory this sounds interesting, and treating frontend and backend differences as a DAG that is compiled for the different environments seems cool, I'm skeptical that it would actually be productive in a large application. One thing I'd be particularly concerned about is debugging. One of the things that's on the roadmap is "developer experience", and I don't see any particular debug docs or helpers, so it tells me that if there is a bug somewhere debugging it is going to be a pain, even more so because I won't easily be able to see whether the problem is my client or server code as they live together and are compiled into separate things. As a critique the fact that developer experience is on the roadmap rather than thought about from the beginning is a little off-putting, but maybe that's not what was meant here. Another thing that gives me hesitation is that I can't see how I would extend the setup to support offline capable applications. This is increasingly becoming a big thing for web+desktop apps, does this easily support that? Lastly, the docs seem to point to use a homegrown DOM library, rather than one of the major players. How easily would this work with a library like Reagent, Helix, or UIx?
- scary-size 4y ago
  Agreeing on the debugging concerns. Adding another layer of abstraction on top of everything else isn't going to help anyone. Creating an abstraction that fits so well that you don't have to worry about the layer underneath is extremely hard. That might not show during initial development, but will come up in the maintenance phase for sure. A bit unsure where the observation of more offline capabilities comes from. I have the impression that things are moving in the opposite direction: Everything needs to be online.
  - ldh0011 4y ago
    I think they mean apps that don't just stop working when you're offline, not apps that are always offline.
    - yladiz 4y ago
      Yes, that's what I meant. I didn't mean an app that's never online (or connected to a server), since that doesn't really make sense in the context of this kind of framework, I meant something that is resilient to being offline (aka "offline capable").
tr14 4y ago
I respect an engineers that can code Clojure. Nice job.
college_physics 4y ago
Practicalities aside (and there might be many issues and even be show-stoppers), the approach is a breath of fresh air in what appears like a stagnating swamp. How to orchestrate overall client/server computations is obviously a vital importance if not the most important design choice in the modern tech landscape. Yet while strong opinions abound, frameworks that help us to think more concretely and rationally about the tradeoffs and break the redundancy of infinite alternative possibilities are not common. If I understand the idea correctly effectively here you have a platform where alternative choices are simple rearrangements of code.
- dustingetz 4y ago
  that’s right!
nih0 4y ago
Is this similar too Java/Vaadin, that also allows calling of backend database repos and the like from the frontend ?
- den1k 4y ago
  Yes, literally anything on the backend.
  - la-ruby 4y ago
    lately I've been doing https://github.com/hotwired/turbo-rails/blob/main/app/models/concerns/turbo/broadcastable.rb https://github.com/hotwired/turbo-rails/blob/main/app/models...
samsquire 4y ago
This is similar to my idea "structured logging 2 system" The idea is that we write what should be done and it is scheduled onto different systems and communication is inserted. https://github.com/samsquire/structured-logging-2-system https://github.com/samsquire/structured-logging-2-system I really like the idea of being capable of switching where something runs on the server or the client by changing a boundary.
dustingetz 4y ago
Founder here - woke up to this. Here is some context for HN: Demo videos: https://hyperfiddle.notion.site/Electric-Clojure-progress-Dec-2022-5416dda526e24e5ab7ccb7eb48c797ed https://hyperfiddle.notion.site/Electric-Clojure-progress-De... Prior HN discussions addressing many of the FAQs: https://news.ycombinator.com/item?id=28630209 https://news.ycombinator.com/item?id=28630209 https://news.ycombinator.com/item?id=31217448 https://news.ycombinator.com/item?id=31217448 Some notes on the network layer: https://www.reddit.com/r/Clojure/comments/vizdcc/hyperfiddlephoton_progress_update/idk0dy5/ https://www.reddit.com/r/Clojure/comments/vizdcc/hyperfiddle... I'll try to address some FAQs in another post
- ordinaryradical 4y ago
  This seems similar in approach to Lamdera for Elm. Just curious, did you take any inspiration from that project or arrive at this solution independently?
  - dustingetz 4y ago
    (founder here) We got here from first principles by thinking deeply about pure functional programming. A lot of what is happening in pure FP ecosystems is "http servers but with monads" and we think that misses the point.
  - zem 4y ago
    also ocsigen for ocaml
den1k 4y ago
I've been building with electric for several months now and it's hands down incredible! Why? I think most importantly because it allows for functional composition across the network (client/server–distributed systems more generally) while also ensuring payloads are as small as possible (only what functions in each env access) which translates to FAST. IMO this is a paradigm shift similar to GC. No one wants to manage their garbage. No one wants to manage the network. It's grunt work. Let the compiler take care of it. I've naively rewritten a re-frame (react/redux in js) app that started getting prohibitively slow due to a bunch of deserialization on the client in electric and the performance is outstanding! TBH I'm taking a break from additional UI work on the app to do research because I realized that with electric I can build something much more amazing than I thought possible before. Caveat: the future will not be served on a silver platter. it's early days and there are some quirks
- yenda 4y ago
  Could you share which kind of quirks you've identified?
  - den1k 4y ago
    Small bugs overall, some of which the team already addressed. One Clojure specific that is outstanding is that dynamic variables are not rebound between electric and Clojure functions. Also, the compiler currently does not optimize the callstack so the whole app stream-renders in and out which has a cascading effect. None of these are deal breakers for me, they're being worked on and I'm glad the community can now help ship fixes and improvements.
    - dustingetz 4y ago
      (founder here) Thank you Dennis! All, broadly, to date we've focused on achieving correct program semantics over syntax/perf/ux/dx. We're rotating onto these issues now, since we've reached a point where, after 2 years of hardcore dogfooding and 8 months of private beta, we're comfortable that brave Clojurists like Dennis will be successful and have an acceptable development experience. (Especially when compared with the benefit of strong composition at the system level - functional programming over entire systems!)
beders 4y ago
Can't wait to try this. Congratulations on the release! A great showcase for the flexibility of Lisp. Others invent whole programming languages around a paradigm, while Lisp just marches on with s-expressions. Well done!
gfodor 4y ago
This looks cool. If I think one step ahead, this reduces enough incidental complexity its the kind of thing that, when combined with AI, could produce the holy grail of an AI writing most of the code for a full CRUD app.
- dustingetz 4y ago
  yup! For a sneak peak - https://hyperfiddle.notion.site/Hyperfiddle-progress-report-2023-Feb-8cc45f9da47c4719bb16851d129e3a3d https://hyperfiddle.notion.site/Hyperfiddle-progress-report-...
eduction 4y ago
“The Electric compiler performs deep graph analysis of your unified frontend/backend program to automatically determine the optimal network cut, and then compile it into separate client and server target programs that cooperate and anticipate each other's needs.” This make it sound like client and server segments are determined on the fly based on prevailing conditions, but then the sample code has forms wrapped with e/client and e/server. What do you mean when you say you determine “the optimal network cut” and how do you reconcile this with the expressly marked client and server code? By the way I’ve been following this project for a while and am very excited about it, kudos on getting to this point.
- dustingetz 4y ago
  Great question. A trivial example, expr (e/client (inc (e/server (e/client x)))) should simplify to just (p/client (inc x)) and eliminate the pointless (and perf-damaging) server round trip. It gets interesting when you introduce lambdas, revealing that this optimization is non-local – you don't know if the transfer is superfluous until you know where the function is called from, and consider the caller and callee together. The good news is that the DAG (Circuit really) contains everything there is to know about the data flow, so insert compilers research here and poof, insanely fast app.
emmanueloga_ 4y ago
Hey Dustin, could you share a few details of how you are funding your projects? I know that you have been working in hyperfiddle for a long while but not sure if you have any revenue from it yet. Your projects are very “futuristic” and the kind of thing I imagine may be hard to get financial backing for, because of the novelty and other considerations… so makes me curious how you balance innovation with monetization in your work. Thank you!
- zubairq 4y ago
  Personally I think it is ok to say that I am one of the people backing Dustin financially. The last project I backed in this space was Light Table/Eve who did some interesting work in the space. I think Dustin's work shows promise too.
iLemming 4y ago
Looks very awesome. Will definitely give it a try. Hopefully it doesn't end up like Om.next. Om.next felt like an amazing quasi-revolutionary thing when David announced it. But it was so poorly documented and hard to get it right, almost nobody used it in production. I don't know for a fact, but it felt like it complicated Circle CI's frontend to the point that they'd decided to abandon Clojurescript altogether. It's sad, because Om.Next really did have some cool ideas. Fulcro project fixed many problems, but it was too late, the momentum for Om.Next was already wavering.
waldrews 4y ago
Could someone who understands the similar blurring of lines between client and server in the F# world, through Fable or more broadly in Blazor/Bolero, comment on how this approach compares?
the-alchemist 4y ago
Noob question: Can HTMX be used together with hyperfiddle?
- butternoodle 4y ago
  I may be mistaken but I think Electric Clojure renders HTMX unnecessary. Where HTMX still works by a developer specifying the network calls (albeit in a declarative manner), Electric Clojure hands over the same responsibility to the compiler.

330 points by Borkdude 4y ago

> There is no client/server dichotomy from the programmer's perspective But on the first example, I see a lot of e/client and e/server calls. Am I misunderstanding this? If so, how is this different from meteor.js?

popinman322 4y ago
Seems like e/client and e/server are for when you'd like to force sections of code to run on one side or the other. Otherwise the compiler decides what lives where?
- kimi 4y ago
  Exactly. And when one side needs the other, it makes magic happen.
  - patates 4y ago
    Well then 2 humble suggestions: - Would be better to show an example where magic would happen - Selectable text instead of a screenshot would have been nicer
    - kimi 4y ago
      Magic does happen: you are rendering a table on the client, pause to fetch data on the server, and display it. And it looks like a PHP page written in Clojure.

kimi 4y ago

This is major - potentially killing off a thousand tons of boilerplate. Not sure if it is too late, because Elixir's LiveView seem to have gotten important because it works around the same issue. But it's time to push the wooden stake through the heart of the SPA (and the phone app too, but that's for another day).

incrudible 4y ago
You can already kill the boilerplate with code generation and metaprogramming. Once you have done that, you figure out why that is not always a good idea and how to best deal with the exceptions that your particular use case brings to the table.
- kimi 4y ago
  I'm thinking more of all the cruft that goes to handle state on the client and keep it synced to the server in your React app. Redux and the gazillions of other packages which names sound like warthogs having breakfast.
  - incrudible 4y ago
    The same argument applies. I think Redux is a pointless indirection, but you can use metaprogramming to make it more palatable, at the cost of another indirection.

i_cannot_hack 4y ago

I don't know much about this kind of architecture, but wouldn't it be problematic from an infosec standpoint? It seems like it would be very easy to accidentally leak information from the server to the client in a subtle manner if the developer makes a mistake somewhere. And in ways which would not be obvious from reading the code unless you carefully consider the implications of each e/server and e/client with respect to the mentioned graph analysis? Are there any mitigations in place to prevent this, except the obvious "be careful and don't write bad code"? Or have I misunderstood things completely and it's a non-issue?

prettyStandard 4y ago
I tried to get hands-on experience with this hyper fiddle stuff a few months ago but I think it was still in closed beta. I would imagine you can inspect the network traffic. I would hope it would be readable so that you could have a good idea of what's going on and errors would be more obvious.
- moondowner 4y ago
  I see a sample todo app in their repo, so yes, one can run it and check out how it works https://github.com/hyperfiddle/electric-starter-app https://github.com/hyperfiddle/electric-starter-app
adlpz 4y ago
Just as a data point, this is also true about very much noeadays "vanilla" stacks like Next.js. It seems like you're doing SSR but not exactly. You return the wrong prop in a getServerSideProps and boom, your stripe keys are dumped in a nice JSON for all to see.
- reilly3000 4y ago
  Yikes!!! Is anyone aware of automated testing/scanning for secret leakage via js framework magic? If not, how would you implement one?
4y ago

yladiz 4y ago

Not to be a downer, but while I think in theory this sounds interesting, and treating frontend and backend differences as a DAG that is compiled for the different environments seems cool, I'm skeptical that it would actually be productive in a large application. One thing I'd be particularly concerned about is debugging. One of the things that's on the roadmap is "developer experience", and I don't see any particular debug docs or helpers, so it tells me that if there is a bug somewhere debugging it is going to be a pain, even more so because I won't easily be able to see whether the problem is my client or server code as they live together and are compiled into separate things. As a critique the fact that developer experience is on the roadmap rather than thought about from the beginning is a little off-putting, but maybe that's not what was meant here. Another thing that gives me hesitation is that I can't see how I would extend the setup to support offline capable applications. This is increasingly becoming a big thing for web+desktop apps, does this easily support that? Lastly, the docs seem to point to use a homegrown DOM library, rather than one of the major players. How easily would this work with a library like Reagent, Helix, or UIx?

scary-size 4y ago
Agreeing on the debugging concerns. Adding another layer of abstraction on top of everything else isn't going to help anyone. Creating an abstraction that fits so well that you don't have to worry about the layer underneath is extremely hard. That might not show during initial development, but will come up in the maintenance phase for sure. A bit unsure where the observation of more offline capabilities comes from. I have the impression that things are moving in the opposite direction: Everything needs to be online.
- ldh0011 4y ago
  I think they mean apps that don't just stop working when you're offline, not apps that are always offline.
  - yladiz 4y ago
    Yes, that's what I meant. I didn't mean an app that's never online (or connected to a server), since that doesn't really make sense in the context of this kind of framework, I meant something that is resilient to being offline (aka "offline capable").

tr14 4y ago

I respect an engineers that can code Clojure. Nice job.

college_physics 4y ago

Practicalities aside (and there might be many issues and even be show-stoppers), the approach is a breath of fresh air in what appears like a stagnating swamp. How to orchestrate overall client/server computations is obviously a vital importance if not the most important design choice in the modern tech landscape. Yet while strong opinions abound, frameworks that help us to think more concretely and rationally about the tradeoffs and break the redundancy of infinite alternative possibilities are not common. If I understand the idea correctly effectively here you have a platform where alternative choices are simple rearrangements of code.

dustingetz 4y ago
that’s right!

nih0 4y ago

Is this similar too Java/Vaadin, that also allows calling of backend database repos and the like from the frontend ?

den1k 4y ago
Yes, literally anything on the backend.
- la-ruby 4y ago
  lately I've been doing https://github.com/hotwired/turbo-rails/blob/main/app/models/concerns/turbo/broadcastable.rb https://github.com/hotwired/turbo-rails/blob/main/app/models...

samsquire 4y ago

This is similar to my idea "structured logging 2 system" The idea is that we write what should be done and it is scheduled onto different systems and communication is inserted. https://github.com/samsquire/structured-logging-2-system https://github.com/samsquire/structured-logging-2-system I really like the idea of being capable of switching where something runs on the server or the client by changing a boundary.

dustingetz 4y ago

Founder here - woke up to this. Here is some context for HN: Demo videos: https://hyperfiddle.notion.site/Electric-Clojure-progress-Dec-2022-5416dda526e24e5ab7ccb7eb48c797ed https://hyperfiddle.notion.site/Electric-Clojure-progress-De... Prior HN discussions addressing many of the FAQs: https://news.ycombinator.com/item?id=28630209 https://news.ycombinator.com/item?id=28630209 https://news.ycombinator.com/item?id=31217448 https://news.ycombinator.com/item?id=31217448 Some notes on the network layer: https://www.reddit.com/r/Clojure/comments/vizdcc/hyperfiddlephoton_progress_update/idk0dy5/ https://www.reddit.com/r/Clojure/comments/vizdcc/hyperfiddle... I'll try to address some FAQs in another post

ordinaryradical 4y ago
This seems similar in approach to Lamdera for Elm. Just curious, did you take any inspiration from that project or arrive at this solution independently?
- dustingetz 4y ago
  (founder here) We got here from first principles by thinking deeply about pure functional programming. A lot of what is happening in pure FP ecosystems is "http servers but with monads" and we think that misses the point.
- zem 4y ago
  also ocsigen for ocaml

den1k 4y ago

I've been building with electric for several months now and it's hands down incredible! Why? I think most importantly because it allows for functional composition across the network (client/server–distributed systems more generally) while also ensuring payloads are as small as possible (only what functions in each env access) which translates to FAST. IMO this is a paradigm shift similar to GC. No one wants to manage their garbage. No one wants to manage the network. It's grunt work. Let the compiler take care of it. I've naively rewritten a re-frame (react/redux in js) app that started getting prohibitively slow due to a bunch of deserialization on the client in electric and the performance is outstanding! TBH I'm taking a break from additional UI work on the app to do research because I realized that with electric I can build something much more amazing than I thought possible before. Caveat: the future will not be served on a silver platter. it's early days and there are some quirks

yenda 4y ago
Could you share which kind of quirks you've identified?
- den1k 4y ago
  Small bugs overall, some of which the team already addressed. One Clojure specific that is outstanding is that dynamic variables are not rebound between electric and Clojure functions. Also, the compiler currently does not optimize the callstack so the whole app stream-renders in and out which has a cascading effect. None of these are deal breakers for me, they're being worked on and I'm glad the community can now help ship fixes and improvements.
  - dustingetz 4y ago
    (founder here) Thank you Dennis! All, broadly, to date we've focused on achieving correct program semantics over syntax/perf/ux/dx. We're rotating onto these issues now, since we've reached a point where, after 2 years of hardcore dogfooding and 8 months of private beta, we're comfortable that brave Clojurists like Dennis will be successful and have an acceptable development experience. (Especially when compared with the benefit of strong composition at the system level - functional programming over entire systems!)

beders 4y ago

Can't wait to try this. Congratulations on the release! A great showcase for the flexibility of Lisp. Others invent whole programming languages around a paradigm, while Lisp just marches on with s-expressions. Well done!

gfodor 4y ago

This looks cool. If I think one step ahead, this reduces enough incidental complexity its the kind of thing that, when combined with AI, could produce the holy grail of an AI writing most of the code for a full CRUD app.

dustingetz 4y ago
yup! For a sneak peak - https://hyperfiddle.notion.site/Hyperfiddle-progress-report-2023-Feb-8cc45f9da47c4719bb16851d129e3a3d https://hyperfiddle.notion.site/Hyperfiddle-progress-report-...

eduction 4y ago

“The Electric compiler performs deep graph analysis of your unified frontend/backend program to automatically determine the optimal network cut, and then compile it into separate client and server target programs that cooperate and anticipate each other's needs.” This make it sound like client and server segments are determined on the fly based on prevailing conditions, but then the sample code has forms wrapped with e/client and e/server. What do you mean when you say you determine “the optimal network cut” and how do you reconcile this with the expressly marked client and server code? By the way I’ve been following this project for a while and am very excited about it, kudos on getting to this point.

dustingetz 4y ago
Great question. A trivial example, expr (e/client (inc (e/server (e/client x)))) should simplify to just (p/client (inc x)) and eliminate the pointless (and perf-damaging) server round trip. It gets interesting when you introduce lambdas, revealing that this optimization is non-local – you don't know if the transfer is superfluous until you know where the function is called from, and consider the caller and callee together. The good news is that the DAG (Circuit really) contains everything there is to know about the data flow, so insert compilers research here and poof, insanely fast app.

emmanueloga_ 4y ago

Hey Dustin, could you share a few details of how you are funding your projects? I know that you have been working in hyperfiddle for a long while but not sure if you have any revenue from it yet. Your projects are very “futuristic” and the kind of thing I imagine may be hard to get financial backing for, because of the novelty and other considerations… so makes me curious how you balance innovation with monetization in your work. Thank you!

zubairq 4y ago
Personally I think it is ok to say that I am one of the people backing Dustin financially. The last project I backed in this space was Light Table/Eve who did some interesting work in the space. I think Dustin's work shows promise too.

iLemming 4y ago

Looks very awesome. Will definitely give it a try. Hopefully it doesn't end up like Om.next. Om.next felt like an amazing quasi-revolutionary thing when David announced it. But it was so poorly documented and hard to get it right, almost nobody used it in production. I don't know for a fact, but it felt like it complicated Circle CI's frontend to the point that they'd decided to abandon Clojurescript altogether. It's sad, because Om.Next really did have some cool ideas. Fulcro project fixed many problems, but it was too late, the momentum for Om.Next was already wavering.

waldrews 4y ago

Could someone who understands the similar blurring of lines between client and server in the F# world, through Fable or more broadly in Blazor/Bolero, comment on how this approach compares?

the-alchemist 4y ago

Noob question: Can HTMX be used together with hyperfiddle?

butternoodle 4y ago
I may be mistaken but I think Electric Clojure renders HTMX unnecessary. Where HTMX still works by a developer specifying the network calls (albeit in a declarative manner), Electric Clojure hands over the same responsibility to the compiler.