10 ms·

Towards Principled Reactive UI

234 points by raphlinus 6y ago

anoncept 6y ago
Lovely post, and an interesting problem area. Three additional perspectives that have been on my mind in this area: 1. Differential dataflow. Like Adapton and Incremental (which Raph cites), I have found DD to be hugely helpful for understanding which computations can be efficiently incrementalized, and how (including in Rust). 2. Operational transforms. Often times, I am working on UI as part of a larger distributed/networked system. In these cases (especially when I already want the UI to display other people and their actions), the division of responsibility between the UI, the network code, and the various internal models has always felt like a missed opportunity for better and more principled integration / compositionality. 3. Build systems. The problem of figuring out how to bring a UI efficiently up to date in response to some parts of the underlying state changing can also be seen as a build system, or cache coherency problem. From that standpoint, where do the paradigms of the Build Systems a la Carte paper fit into Raph’s pictured theory of reactive UI? https://github.com/TimelyDataflow/differential-dataflow https://github.com/TimelyDataflow/differential-dataflow https://github.com/mstone/focus/blob/master/docs/ot-theory.adoc https://github.com/mstone/focus/blob/master/docs/ot-theory.a... https://doi.org/10.1017/S0956796820000088 https://doi.org/10.1017/S0956796820000088
- raphlinus 6y ago
  Thank you for your kind words. I agree that all these resources are relevant. I'm familiar with some but will study others more deeply. Certainly I've spent a lot of time thinking about collaborative and distributed systems, and there is no doubt scope for cross-fertilization of ideas. A huge inspiration (cited in the post) is Martin Kleppmann's automerge, which uses a proxy object to capture an explicit diff from a computation which is expressed using ordinary mutable logic. Regarding the build system work, I will say this: the build task is inherently graph-like, and a lot of the interesting problems involve choices of sort order to utilize parallelism best; I'm fortunate to have a 16 core machine, and it makes me sad to wait on a single-threaded link step. I think by comparison the reactive UI task is mostly dealing with tree structure, and that's a simplification that could be useful to exploit. It's easier to describe mutation of a tree than general mutation of a graph. But certainly there are common themes in all these incremental systems, and I continue to be drawn to figuring out the fundamental principles rather than trying to build systems ad-hoc.
  - anoncept 6y ago
    Martin’s writing and thinking has been hugely inspirational for me too — thanks for highlighting the automerge work here! Next, regarding your other thoughts, two notes, one on performance and one on “common themes”. Re performance: while in my initial comment I linked directly to McSherry & friends’ work on differential dataflow mostly because I value it as a principled and effective framework for incrementalizing many kinds of computation I care about, I would be remiss if I failed to also highlight here their complementary “COST” project of using this framework to demonstrate how carefully engineered (single-threaded!) implementations of graph computations in Rust can often considerably outperform more complex but more wasteful pre-existing approaches (at least in the database world). (Whether the same flavor of result will also appear over time in the build systems or UI worlds seems like an interesting question for future work!) Finally, re “common themes in incremental systems”: yes! - this and your point about “fundamental principles” are exactly what I was hoping to suggest by linking to the build systems + OT materials! Thanks! http://www.frankmcsherry.org/assets/COST.pdf http://www.frankmcsherry.org/assets/COST.pdf https://github.com/frankmcsherry/COST https://github.com/frankmcsherry/COST
chriskrycho 6y ago
I’m curious, given salsa’s[1] so-far quite-effective use of the incremental computation (Adapton/Incremental/Bonsai), if you could elaborate on some of the pitfalls there for UI specifically. The primary salsa use case so far (in rust-analyzer[2]) is obviously fairly different from the pure UI use case, as it’s allowing other things to do the work on the UI and is using it purely for the incremental computation of compilation results, which are a very different set of constraints from UI painting. But at that point, my own knowledge runs out: thus the question. [1]: https://salsa-rs.github.io/salsa/ https://salsa-rs.github.io/salsa/ [2]: https://rust-analyzer.github.io/ https://rust-analyzer.github.io/ This is particularly of interest to me because I work day in and day out with what is to my knowledge the only application of the incremental computation approach to browser UI,[3] and colleagues and I have talked about what it would take to reimplement in Rust—what the tradeoffs would be, where the problems would emerge, etc. I suspect part of the issue is that the constraints of using the DOM informs a lot of the design considerations in the space, whereas implementing from scratch gives very different tradeoffs? [3]: https://v5.chriskrycho.com/journal/autotracking-elegant-dx-via-cutting-edge-cs/ https://v5.chriskrycho.com/journal/autotracking-elegant-dx-v...
- raphlinus 6y ago
  It's a very good question. I haven't really tried using salsa directly, but I know Adam Perry spent some time with it. I believe an earlier prototype of Moxie actually used salsa, but he ended up moving away from that for reasons I don't fully understand myself. In any case, Crochet is very deeply inspired by Moxie, so there's certainly an influence there, even if we don't end up using the code directly. Regarding DOM, yes. I think it's held up pretty well considering how long the ideas have been around. It's slow, it's clunky, but it gets the job done. But I also think now is a good time to consider better ways to implement its core functionality, which is tree mutation.
  - mst 6y ago
    I'm unsure how directly relevant this is but when I dug into the implementation of Elixir's LiveView stuff I was frankly amazed as to how little javascript was needed client side to apply the diffs being sent down. I'm aware this is significantly because they "cheat", but I feel like you're already looking at borrowing from other peoples' trickery so maybe it'll still be of interest.
stephc_int13 6y ago
In practice, an immediate mode API is clearly the way to go. We've all seen "smarter" approaches failing more than a few times, obviously on the simplicity side but also, sadly, on the performance side, even with all the caching and retained structures. This is the strange beauty of brute force algorithms, they may look dumb and ugly for our refined human minds, but they tend to map very well to the capacities of computers. It is often faster to not bother and blindly render something instead of running complex logic to decide if you really need to render or not. And if you're not satisfied with the performances of a "dumb" imgui, you can always add some smarter caching without touching the API. Altough I bet this approach would only get you marginal benefits.
- raphlinus 6y ago
  How do you plan to address accessibility using imgui? As I argue in the post, the model of an imgui-like API for clear expression of app logic, backed by a more traditional retained render object tree, has the potential to give you the best of both worlds.
  - stephc_int13 6y ago
    >How do you plan to address accessibility using imgui? I am not sure I understand this question.
    - raphlinus 6y ago
      > I am not sure I understand this question. I rest my case.
      - boulos 6y ago
        I think it’s fair to say that accessibility is an ambiguous term. Which one did you mean? (My charitable guess was a11y in the screen reader, color blindness, font size, etc. sense)
        6y ago
adamnemecek 6y ago
I'm pumped to check this out, Raph has been doing really good work on the Rust UI front. You should join the xi channel on zulip https://xi.zulipchat.com/ https://xi.zulipchat.com/, there's a dedicated Crotchet subchannel.
chrisweekly 6y ago
Awesome post, thanks for writing and sharing it. "The idea of the app logic producing an explicit tree mutation while holding an immutable reference to the old state of the tree feels very functional, though the app state itself is mutable. Even so, I haven’t seen this particular pattern in the functional programming UI literature (hopefully readers can fill me in). " IIUC, this sounds a lot like what MWeststrate does with Mobx-State-Tree^1 1. https://github.com/mobxjs/mobx-state-tree https://github.com/mobxjs/mobx-state-tree
nickmqb 6y ago
Do UIs need to be trees? How do things change if we were to use a list instead? What becomes easier/harder? Layout calculations might become slightly more expensive, though it seems that that may be acceptable since most time is spent rendering; and perhaps there are other ways to accelerate these calculations if needed. The fact that most UIs are happy to provide a single linear tab order (and not something more involved that allows users to navigate the tree structure) makes me think that there might be something there.
- adamnemecek 6y ago
  How would you represent a ui as a list?
  - nickmqb 6y ago
    A list of widgets, with each widget having a x, y, width and height.
    - alvarlagerlof 6y ago
      I dare you to make a website using only css position: absolute; Make it work on different screen sizes. Do independent scrolling. It'll be hell very quickly. Nesting is used in all somewhat complicated layouts.
      - dsego 6y ago
        How about with css grid?
      - nyanpasu64 6y ago
        Win32/MFC dialog layouts are based off absolute positioning of all widgets, with a few concessions to resizability (like the ability to select whether a widget moves down or stay still when a window gets taller). This is the reason why many dialogs, like Run, or Windows Explorer file properties, cannot be resized. Any hierarchy (group boxes holding widgets) is purely visual, and the rectangle doesn't "own" its child widgets in a programmatic sense. Text widgets have a fixed size. If your font gets wider (from DPI scaling or translations), text can become cut off or overflow onto the next line and disappear.
Kinrany 6y ago
Superficially Terraform seems to be solving a similar problem: a declarative interface for updating a stateful process.
- Kinrany 6y ago
  So: - functional core is responsible for computing the new desired state in response to events - driver/plugin is responsible for computing the diff between the current and the desired state - imperative shell is responsible for applying the diff to the process... which may be partially or wholly implemented as another component
liuliu 6y ago
It is a long post with a lot of good points. One minor thing I would argue: the "setter support" in languages is not really a plus for observer patterns. In many such languages support "setter" and use such to naively implement observer patterns, you can see a lot of repeated updates for the same object through the call chains. Because often than not, you are not update one property for one state object, you need to update a set of properties across some number of state objects, and simplistic "setter" based observation cannot differentiate these and coalescing properly. People can workaround this and implement "backpressure" etc, but it just to add more complexity on a broken promises. What you really need is an observer implementation with transactional guarantees. Thus, the state change cannot be observed until an update transaction is done. This will naturally reduce number of unnecessary updates, and doesn't really need "setter support" from the language.
mwcampbell 6y ago
> Down the line, those needs include accessibility, a serious Achilles heel for imgui-flavored designs in particular. Thank you for drawing attention to this. I spent a good chunk of last weekend pondering how accessibility might be added to IMGUI toolkits, and I do think it would be difficult. In particular, as you said: > I believe a proper approach to this problem involves stable identity of widgets, about which much more below. Agreed. In platform accessibility APIs such as UI Automation, each node has an identity ("runtime ID" in UIA) which is expected to be stable. One of the IMGUI toolkits I looked at was Nuklear, and I didn't come up with a way to derive stable identities for its widgets. On the other hand, Gio [1] (an immediate-mode toolkit for Go) looks more tractable, because the application holds a struct for each widget's state. Still, in an accessibility API like UIA, even simple static text nodes are supposed to have stable identity; I don't know how that would be solved with something like Gio. [1]: https://gioui.org/ https://gioui.org/
- mattgreenrocks 6y ago
  I'm probably missing something here, but is there a reason these toolkits don't require the dev to specify a stable, unique ID for each node? Feels like there's strong precedent with HTML's id attribute and CSS.
  - deleted 6y ago
    [deleted]
  - michannne 6y ago
    Because IMGUIs are designed to be procedural, making it easy to couple changes in UI with changes in control flow. Requiring an ID for every widget would get really annoying really fast, considering IDs would become voided pretty often. The default is no IDs, leave the ID assignment up to the developer who knows if and when an ID would be stable You can wrap tools like Dear ImGui and Nuklear into a reactive framework that handles state management and IDs and such and presents an API with a declarative interface a la React, but at that point you're pretty much building your own UI engine and just passing rendering on to those tools, which is not simple to architect.
the__alchemist 6y ago
There's an interesting link in these UI patterns: Code clarity and performance. For example, comparing a virtual DOM model with manual UI changes: Deliberate UI changes, if done well (The article mentions they're often not) are faster... at the expense of losing the elegant model of declarative UI. VDOMs conduct many more operations than are required, but allow you to describe how the UI should behave elegantly. As the article mentions, Svelte is a cool approach that attempts to reconcile this. Personally, I wrote a React/Elm-like WASM framework in Rust. After stepping back, I'd only use something like that for complex, non-performance-critical UIs. In embedded or simple websites, that sort of overhead isn't appropriate, even though it leads to nice code. I think we'll be able to dodge this compromise soon. Maybe with something like Svelte, or maybe with a different approach.
jules 6y ago
Excellent article! While reading it I got the impression that some of the decisions made by the author were influenced by the limitations of Rust, so I wondered how you'd design a language to be ideally suited to express reactive UI? What would such a language look like, and what would the UI library look like.
nikitaga 6y ago
> The problems with observable boil down to the fact that the callback requires a lot of context to know what change should happen in response to the event. In this way, I think it is fundamentally in tension with the goals of reactive UI, though some systems have managed to reconcile the two, often with compiler help to translate fairly straightforward declarative logic into an observable-based implementation: SwiftUI and Svelte come to mind. I don't understand the main issue brought up with observables. What exactly is that "context" you're talking about, and how is it bad? Observables, at least the way used in my https://github.com/raquo/Laminar/ https://github.com/raquo/Laminar/ UI library, define the dataflow graph of your application, letting you channel data into other data and/or into incremental DOM updates without using virtual DOM. You don't need any additional "context" to know what change happens in response to what, defining the flow of data with observables and then binding some of them to DOM parts that you want them to affect or to source data from is literally how you write your application logic. You will need to express that application logic somehow, and doing that using observables is quite straightforward. I'm not familiar with SwiftUI, but Svelte is not really any more about observables than React is. Disregarding how it's implemented under the hood, Svelte's API is made to be superficially similar to React but without virtual DOM, and that comes at a very high cost of designing their own language and requiring their own compiler. So you end up with a callback-driven API that to the user feels a lot like React's state + props + render method, and it's nothing like working with plain observables.
- jbritton 6y ago
  I think there might be an issue with definitions. I thought maybe Raph was talking about the Observer pattern from Gang of Four, and that you are talking about Observables as incrementally computed expressions. I can remember a long time ago reading about an Observer pattern for a UI. Views would subscribe to a model. When the model changed, the views were notified. The Views then queried the model and determined how to update. It was suggested that for more efficient updates the change notification could carry some information(context) to indicate what had changed. The article didn’t discuss this information optimization further.
  - nikitaga 6y ago
amelius 6y ago
Rust is a systems language. For UIs, you want to trade a little performance for ergonomics, thus you'd want a different language. Just as one example, closures are very useful when writing UIs and you need a garbage collector to use them in their fullest potential. Of course, you can still write your low level bitblt functions in Rust if you need to (though these functions are probably best left to a GPU). But an entire application and its UI, then you're probably pushing it too far.
- wtetzner 6y ago
  C++ is a systems language, yet Qt seems to be quite a successful GUI toolkit.
  - amelius 6y ago
    Qt has issues with lifetimes of objects. It's easy to shoot yourself in the foot by accessing objects that have already been destroyed. The borrow checker can solve that, but still shows that C++ was not really the best choice as a language for building UIs. I also think that the success of the combination of Qt/C++ is mostly a result of the evolution of programming environments. C++ was massively popular, so that could explain it.
- hitekker 6y ago
  You’re being downvoted but I agree with your opinion. This blogpost doesn’t show any UIs built with this rust paradigm, and the github homepage only displays a few screenshots of a calculator app. Technically, it appears interesting. Practically, it seems very far off from the tools I use to build usable (or hopefully aesthetically pleasing) UIs.
  - chrismorgan 6y ago
    That’s… kinda the point of research work. No one claimed it was ready to replace the tools you use in production.
- ComputerGuru 6y ago
c-smile 6y ago
"Because DOM is slow" DOM is not slow, in principle. What makes updates slow is the fact that each mutating DOM function must left the DOM in consistent state - rendering trees invalidated, etc. This creates significant overhead on massive updates. But: element.innerHTML = "new content"; is significantly faster than sequence of appendNode & Co. because element.innerHTML mutation is transactional. Browsers do not need to notify observers on each element parsed - only whole subtree at the end. I have tests of different methods of DOM updates in Sciter (https://sciter.com https://sciter.com) see: https://github.com/c-smile/sciter-sdk/blob/master/samples/tests/test-scrollable-table.htm#L114 https://github.com/c-smile/sciter-sdk/blob/master/samples/te... The test shows that these two methods: element.html = "new content"; and element.content( vector-of-vDOM-nodes ); are significantly faster than anything else as both methods are transactional.
- raphlinus 6y ago
  What you're saying seems to be consistent with what I'm saying. Web-browser standard DOM is slow, for exactly the reasons you state. Adding an additional tree mutation API, not supported by any web browser, can obviously be a lot faster, and as you say, a good way to make that faster is to group the mutation into a transaction, rather than effectively have one transaction for each appendChild and setAttribute call. What you've done is a pretty small change to the DOM and generally compatible with it. What I've done in Crochet is a more radical rework, but certainly with some of the same general principles (my `Mutation` datatype is also a transaction). I should also point out that my tree mutation is a bit more limited, as it can't do general movement of subtrees, only sibling movement within the same parent (and, in the current prototype, no swapping, only order-preserving, which includes insert, delete, and update). If some compatibility with browser DOM is a goal, then your approach is appealing. I'm not pursuing it in my prototype because I'm also holding overall system simplicity as a whole. But in any case, I agree that people should be aware of what Sciter does; it's an impressive system.
  - c-smile 6y ago
    We just need to add one method to Web API: document.mutate(function(mutator) { mutator.changeAttributes(el,{...}); mutator.appendNodes(el,...); ... }); The mutate will do transactional update as fast as possible.
cryptMackay 6y ago
Makes sense!
weeboid 6y ago
Man, reading this just reinforces that DOMs and HTML are not "right" for webapps. These frameworks, contrivances, and the resulting principles will spiral on, never solving the real issue: DOCUMENT Object Models and HYPER TEXT MARKUP Language are for representing linked documents, not stateful object based applications. It's right there, in the names.
- fallennode 6y ago
  Is that true though? In the article he mentions how a lot of the web frameworks don't have to worry about handling tabindex because the browser does it for them. I have also yet to find another non-web UI framework that handles styling nearly as well as CSS. Between flexbox, grid, and absolute positioning, you have so much power for creating layouts. Throw in some drop shadows and borders to help highlight the important UI pieces and bam. I think the common critique of React, Vue, etc on HN is fair when talking about people building simple websites with these highly sophisticated and massively overkill tools. But when it comes to applications I have yet to find something nearly as flexible and good looking with so little work. And they do it by using HTML and CSS which as it turns out are pretty good abstractions to build on. Or at least they are from my perspective as someone who's been using a variety of these frameworks for the past 5 years or so, on quite a few medium sized projects.
svilen_dobrev 6y ago
looong time ago, we've discussed with a colleague who was building some UI for a game as side project, about what "hierarchies" (essentialy, aspects of structure) are there in an UI. So we ended with about 4: * encapsulation - what includes what / belongs to what * visual - what overlays/obscures what * event-passing - if X gets clicked, or otherwise interacted with, who else would get the event passed to/notified * ??? can't remember this one.. but i remember it took many conversations to find out we need it. Probably was dependency-linkage (in aspects other than above 3, e.g. some model-logic dependencies) This aspect-distinguishing may or may not be usable for you. For example, a usual "scrollbar" is composed of multiple smaller elements, which are tied together in all above aspects. But it does not have to be like that, e.g. the position-mover ("thumb") and the lower/higher buttons need not be on same visual place, e.g. a circle volume knob and separate quieter + louder knobs elsewhere (and that may be very far). One may argue it's not a scrollbar anymore.. expect it does the same thing, just visually differently. It would be interesting if one can really separate these aspects in the code, somehow. That said, how would your toolkit handle this: a small "innocent" interaction that causes lots of small changes all over the tree. Think taxforms: a form, mixture of subforms and fields, many levels deep, and i need to highlight the user-changed fields (or subforms containing changed fields), all the way up. What i ended up recently, in React (i.e. in DOM-recipe-tree), is, anytime the user presses some keystroke in some text field down in the depths, the whole thing needs be repainted. A few hundred fields probably. Which.. isn't that fast anymore, when depth goes beyond 3-4. (From user perspective there are 2-3 times less levels than programatical). Probably i've missed some "i have not changed, don't repaint me" step somewhere.
rangoon626 6y ago
The web does it better, and the web does it first. But I am glad to see native developers finally get a half-baked, sparsely documented reactive UI framework (still a year later!), and like a decade late. Apple is over, and it’s hilarious!
archagon 6y ago
> I think the debate is now essentially over, the reactive approach is winning. Is this really true? My intuition is that reactive UIs are fantastic for widgets and "wraps a database" type apps, but fall flat when it comes to deeply interactive software where performance and responsiveness is paramount — i.e. where there's a tight feedback loop between user input and rendering, and where the model state tends to be quite large and granular. (DAWs, drawing/illustration software, movie editors, mapping applications, text editors, etc.) I also get the sense that most revolutionary or industry-changing software falls into this category. So how can the reactive approach be anything other than a small tool in the app developer's toolbox? Could somebody dissuade me from this notion? In my admittedly limited experience with the reactive paradigm, I feel like I run into a wall as soon as I've implemented all the basic widgets and need to actually work on the main content view of my app. It doesn't feel like it could ever be a good fit for software that pushes boundaries. (Perhaps this is an entirely separate matter from "object-oriented UI" vs. "reactive UI", though. Or maybe I'm "using it wrong", in the sense that reactive UIs are meant to be mostly about widgets and not complex interactive views. But then I often hear people talk about reactive UI as the future of app development, so I'm not sure.)

234 points by raphlinus 6y ago

Lovely post, and an interesting problem area. Three additional perspectives that have been on my mind in this area: 1. Differential dataflow. Like Adapton and Incremental (which Raph cites), I have found DD to be hugely helpful for understanding which computations can be efficiently incrementalized, and how (including in Rust). 2. Operational transforms. Often times, I am working on UI as part of a larger distributed/networked system. In these cases (especially when I already want the UI to display other people and their actions), the division of responsibility between the UI, the network code, and the various internal models has always felt like a missed opportunity for better and more principled integration / compositionality. 3. Build systems. The problem of figuring out how to bring a UI efficiently up to date in response to some parts of the underlying state changing can also be seen as a build system, or cache coherency problem. From that standpoint, where do the paradigms of the Build Systems a la Carte paper fit into Raph’s pictured theory of reactive UI? https://github.com/TimelyDataflow/differential-dataflow https://github.com/TimelyDataflow/differential-dataflow https://github.com/mstone/focus/blob/master/docs/ot-theory.adoc https://github.com/mstone/focus/blob/master/docs/ot-theory.a... https://doi.org/10.1017/S0956796820000088 https://doi.org/10.1017/S0956796820000088

raphlinus 6y ago
Thank you for your kind words. I agree that all these resources are relevant. I'm familiar with some but will study others more deeply. Certainly I've spent a lot of time thinking about collaborative and distributed systems, and there is no doubt scope for cross-fertilization of ideas. A huge inspiration (cited in the post) is Martin Kleppmann's automerge, which uses a proxy object to capture an explicit diff from a computation which is expressed using ordinary mutable logic. Regarding the build system work, I will say this: the build task is inherently graph-like, and a lot of the interesting problems involve choices of sort order to utilize parallelism best; I'm fortunate to have a 16 core machine, and it makes me sad to wait on a single-threaded link step. I think by comparison the reactive UI task is mostly dealing with tree structure, and that's a simplification that could be useful to exploit. It's easier to describe mutation of a tree than general mutation of a graph. But certainly there are common themes in all these incremental systems, and I continue to be drawn to figuring out the fundamental principles rather than trying to build systems ad-hoc.
- anoncept 6y ago
  Martin’s writing and thinking has been hugely inspirational for me too — thanks for highlighting the automerge work here! Next, regarding your other thoughts, two notes, one on performance and one on “common themes”. Re performance: while in my initial comment I linked directly to McSherry & friends’ work on differential dataflow mostly because I value it as a principled and effective framework for incrementalizing many kinds of computation I care about, I would be remiss if I failed to also highlight here their complementary “COST” project of using this framework to demonstrate how carefully engineered (single-threaded!) implementations of graph computations in Rust can often considerably outperform more complex but more wasteful pre-existing approaches (at least in the database world). (Whether the same flavor of result will also appear over time in the build systems or UI worlds seems like an interesting question for future work!) Finally, re “common themes in incremental systems”: yes! - this and your point about “fundamental principles” are exactly what I was hoping to suggest by linking to the build systems + OT materials! Thanks! http://www.frankmcsherry.org/assets/COST.pdf http://www.frankmcsherry.org/assets/COST.pdf https://github.com/frankmcsherry/COST https://github.com/frankmcsherry/COST

chriskrycho 6y ago

I’m curious, given salsa’s[1] so-far quite-effective use of the incremental computation (Adapton/Incremental/Bonsai), if you could elaborate on some of the pitfalls there for UI specifically. The primary salsa use case so far (in rust-analyzer[2]) is obviously fairly different from the pure UI use case, as it’s allowing other things to do the work on the UI and is using it purely for the incremental computation of compilation results, which are a very different set of constraints from UI painting. But at that point, my own knowledge runs out: thus the question. [1]: https://salsa-rs.github.io/salsa/ https://salsa-rs.github.io/salsa/ [2]: https://rust-analyzer.github.io/ https://rust-analyzer.github.io/ This is particularly of interest to me because I work day in and day out with what is to my knowledge the only application of the incremental computation approach to browser UI,[3] and colleagues and I have talked about what it would take to reimplement in Rust—what the tradeoffs would be, where the problems would emerge, etc. I suspect part of the issue is that the constraints of using the DOM informs a lot of the design considerations in the space, whereas implementing from scratch gives very different tradeoffs? [3]: https://v5.chriskrycho.com/journal/autotracking-elegant-dx-via-cutting-edge-cs/ https://v5.chriskrycho.com/journal/autotracking-elegant-dx-v...

raphlinus 6y ago
It's a very good question. I haven't really tried using salsa directly, but I know Adam Perry spent some time with it. I believe an earlier prototype of Moxie actually used salsa, but he ended up moving away from that for reasons I don't fully understand myself. In any case, Crochet is very deeply inspired by Moxie, so there's certainly an influence there, even if we don't end up using the code directly. Regarding DOM, yes. I think it's held up pretty well considering how long the ideas have been around. It's slow, it's clunky, but it gets the job done. But I also think now is a good time to consider better ways to implement its core functionality, which is tree mutation.
- mst 6y ago
  I'm unsure how directly relevant this is but when I dug into the implementation of Elixir's LiveView stuff I was frankly amazed as to how little javascript was needed client side to apply the diffs being sent down. I'm aware this is significantly because they "cheat", but I feel like you're already looking at borrowing from other peoples' trickery so maybe it'll still be of interest.

stephc_int13 6y ago

In practice, an immediate mode API is clearly the way to go. We've all seen "smarter" approaches failing more than a few times, obviously on the simplicity side but also, sadly, on the performance side, even with all the caching and retained structures. This is the strange beauty of brute force algorithms, they may look dumb and ugly for our refined human minds, but they tend to map very well to the capacities of computers. It is often faster to not bother and blindly render something instead of running complex logic to decide if you really need to render or not. And if you're not satisfied with the performances of a "dumb" imgui, you can always add some smarter caching without touching the API. Altough I bet this approach would only get you marginal benefits.

raphlinus 6y ago
How do you plan to address accessibility using imgui? As I argue in the post, the model of an imgui-like API for clear expression of app logic, backed by a more traditional retained render object tree, has the potential to give you the best of both worlds.
- stephc_int13 6y ago
  >How do you plan to address accessibility using imgui? I am not sure I understand this question.
  - raphlinus 6y ago
    > I am not sure I understand this question. I rest my case.
    - boulos 6y ago
      I think it’s fair to say that accessibility is an ambiguous term. Which one did you mean? (My charitable guess was a11y in the screen reader, color blindness, font size, etc. sense)
      - 6y ago

adamnemecek 6y ago

I'm pumped to check this out, Raph has been doing really good work on the Rust UI front. You should join the xi channel on zulip https://xi.zulipchat.com/ https://xi.zulipchat.com/, there's a dedicated Crotchet subchannel.

chrisweekly 6y ago

Awesome post, thanks for writing and sharing it. "The idea of the app logic producing an explicit tree mutation while holding an immutable reference to the old state of the tree feels very functional, though the app state itself is mutable. Even so, I haven’t seen this particular pattern in the functional programming UI literature (hopefully readers can fill me in). " IIUC, this sounds a lot like what MWeststrate does with Mobx-State-Tree^1 1. https://github.com/mobxjs/mobx-state-tree https://github.com/mobxjs/mobx-state-tree

nickmqb 6y ago

Do UIs need to be trees? How do things change if we were to use a list instead? What becomes easier/harder? Layout calculations might become slightly more expensive, though it seems that that may be acceptable since most time is spent rendering; and perhaps there are other ways to accelerate these calculations if needed. The fact that most UIs are happy to provide a single linear tab order (and not something more involved that allows users to navigate the tree structure) makes me think that there might be something there.

adamnemecek 6y ago
How would you represent a ui as a list?
- nickmqb 6y ago
  A list of widgets, with each widget having a x, y, width and height.
  - alvarlagerlof 6y ago
    I dare you to make a website using only css position: absolute; Make it work on different screen sizes. Do independent scrolling. It'll be hell very quickly. Nesting is used in all somewhat complicated layouts.
    - dsego 6y ago
      How about with css grid?
    - nyanpasu64 6y ago
      Win32/MFC dialog layouts are based off absolute positioning of all widgets, with a few concessions to resizability (like the ability to select whether a widget moves down or stay still when a window gets taller). This is the reason why many dialogs, like Run, or Windows Explorer file properties, cannot be resized. Any hierarchy (group boxes holding widgets) is purely visual, and the rectangle doesn't "own" its child widgets in a programmatic sense. Text widgets have a fixed size. If your font gets wider (from DPI scaling or translations), text can become cut off or overflow onto the next line and disappear.

Kinrany 6y ago

Superficially Terraform seems to be solving a similar problem: a declarative interface for updating a stateful process.

Kinrany 6y ago
So: - functional core is responsible for computing the new desired state in response to events - driver/plugin is responsible for computing the diff between the current and the desired state - imperative shell is responsible for applying the diff to the process... which may be partially or wholly implemented as another component

liuliu 6y ago

It is a long post with a lot of good points. One minor thing I would argue: the "setter support" in languages is not really a plus for observer patterns. In many such languages support "setter" and use such to naively implement observer patterns, you can see a lot of repeated updates for the same object through the call chains. Because often than not, you are not update one property for one state object, you need to update a set of properties across some number of state objects, and simplistic "setter" based observation cannot differentiate these and coalescing properly. People can workaround this and implement "backpressure" etc, but it just to add more complexity on a broken promises. What you really need is an observer implementation with transactional guarantees. Thus, the state change cannot be observed until an update transaction is done. This will naturally reduce number of unnecessary updates, and doesn't really need "setter support" from the language.

mwcampbell 6y ago

> Down the line, those needs include accessibility, a serious Achilles heel for imgui-flavored designs in particular. Thank you for drawing attention to this. I spent a good chunk of last weekend pondering how accessibility might be added to IMGUI toolkits, and I do think it would be difficult. In particular, as you said: > I believe a proper approach to this problem involves stable identity of widgets, about which much more below. Agreed. In platform accessibility APIs such as UI Automation, each node has an identity ("runtime ID" in UIA) which is expected to be stable. One of the IMGUI toolkits I looked at was Nuklear, and I didn't come up with a way to derive stable identities for its widgets. On the other hand, Gio [1] (an immediate-mode toolkit for Go) looks more tractable, because the application holds a struct for each widget's state. Still, in an accessibility API like UIA, even simple static text nodes are supposed to have stable identity; I don't know how that would be solved with something like Gio. [1]: https://gioui.org/ https://gioui.org/

mattgreenrocks 6y ago
I'm probably missing something here, but is there a reason these toolkits don't require the dev to specify a stable, unique ID for each node? Feels like there's strong precedent with HTML's id attribute and CSS.
- deleted 6y ago
  [deleted]
- michannne 6y ago
  Because IMGUIs are designed to be procedural, making it easy to couple changes in UI with changes in control flow. Requiring an ID for every widget would get really annoying really fast, considering IDs would become voided pretty often. The default is no IDs, leave the ID assignment up to the developer who knows if and when an ID would be stable You can wrap tools like Dear ImGui and Nuklear into a reactive framework that handles state management and IDs and such and presents an API with a declarative interface a la React, but at that point you're pretty much building your own UI engine and just passing rendering on to those tools, which is not simple to architect.

the__alchemist 6y ago

There's an interesting link in these UI patterns: Code clarity and performance. For example, comparing a virtual DOM model with manual UI changes: Deliberate UI changes, if done well (The article mentions they're often not) are faster... at the expense of losing the elegant model of declarative UI. VDOMs conduct many more operations than are required, but allow you to describe how the UI should behave elegantly. As the article mentions, Svelte is a cool approach that attempts to reconcile this. Personally, I wrote a React/Elm-like WASM framework in Rust. After stepping back, I'd only use something like that for complex, non-performance-critical UIs. In embedded or simple websites, that sort of overhead isn't appropriate, even though it leads to nice code. I think we'll be able to dodge this compromise soon. Maybe with something like Svelte, or maybe with a different approach.

jules 6y ago

Excellent article! While reading it I got the impression that some of the decisions made by the author were influenced by the limitations of Rust, so I wondered how you'd design a language to be ideally suited to express reactive UI? What would such a language look like, and what would the UI library look like.

nikitaga 6y ago

> The problems with observable boil down to the fact that the callback requires a lot of context to know what change should happen in response to the event. In this way, I think it is fundamentally in tension with the goals of reactive UI, though some systems have managed to reconcile the two, often with compiler help to translate fairly straightforward declarative logic into an observable-based implementation: SwiftUI and Svelte come to mind. I don't understand the main issue brought up with observables. What exactly is that "context" you're talking about, and how is it bad? Observables, at least the way used in my https://github.com/raquo/Laminar/ https://github.com/raquo/Laminar/ UI library, define the dataflow graph of your application, letting you channel data into other data and/or into incremental DOM updates without using virtual DOM. You don't need any additional "context" to know what change happens in response to what, defining the flow of data with observables and then binding some of them to DOM parts that you want them to affect or to source data from is literally how you write your application logic. You will need to express that application logic somehow, and doing that using observables is quite straightforward. I'm not familiar with SwiftUI, but Svelte is not really any more about observables than React is. Disregarding how it's implemented under the hood, Svelte's API is made to be superficially similar to React but without virtual DOM, and that comes at a very high cost of designing their own language and requiring their own compiler. So you end up with a callback-driven API that to the user feels a lot like React's state + props + render method, and it's nothing like working with plain observables.

jbritton 6y ago
I think there might be an issue with definitions. I thought maybe Raph was talking about the Observer pattern from Gang of Four, and that you are talking about Observables as incrementally computed expressions. I can remember a long time ago reading about an Observer pattern for a UI. Views would subscribe to a model. When the model changed, the views were notified. The Views then queried the model and determined how to update. It was suggested that for more efficient updates the change notification could carry some information(context) to indicate what had changed. The article didn’t discuss this information optimization further.
- nikitaga 6y ago

amelius 6y ago

Rust is a systems language. For UIs, you want to trade a little performance for ergonomics, thus you'd want a different language. Just as one example, closures are very useful when writing UIs and you need a garbage collector to use them in their fullest potential. Of course, you can still write your low level bitblt functions in Rust if you need to (though these functions are probably best left to a GPU). But an entire application and its UI, then you're probably pushing it too far.

wtetzner 6y ago
C++ is a systems language, yet Qt seems to be quite a successful GUI toolkit.
- amelius 6y ago
  Qt has issues with lifetimes of objects. It's easy to shoot yourself in the foot by accessing objects that have already been destroyed. The borrow checker can solve that, but still shows that C++ was not really the best choice as a language for building UIs. I also think that the success of the combination of Qt/C++ is mostly a result of the evolution of programming environments. C++ was massively popular, so that could explain it.
hitekker 6y ago
You’re being downvoted but I agree with your opinion. This blogpost doesn’t show any UIs built with this rust paradigm, and the github homepage only displays a few screenshots of a calculator app. Technically, it appears interesting. Practically, it seems very far off from the tools I use to build usable (or hopefully aesthetically pleasing) UIs.
- chrismorgan 6y ago
  That’s… kinda the point of research work. No one claimed it was ready to replace the tools you use in production.
ComputerGuru 6y ago

c-smile 6y ago

"Because DOM is slow" DOM is not slow, in principle. What makes updates slow is the fact that each mutating DOM function must left the DOM in consistent state - rendering trees invalidated, etc. This creates significant overhead on massive updates. But: element.innerHTML = "new content"; is significantly faster than sequence of appendNode & Co. because element.innerHTML mutation is transactional. Browsers do not need to notify observers on each element parsed - only whole subtree at the end. I have tests of different methods of DOM updates in Sciter (https://sciter.com https://sciter.com) see: https://github.com/c-smile/sciter-sdk/blob/master/samples/tests/test-scrollable-table.htm#L114 https://github.com/c-smile/sciter-sdk/blob/master/samples/te... The test shows that these two methods: element.html = "new content"; and element.content( vector-of-vDOM-nodes ); are significantly faster than anything else as both methods are transactional.

raphlinus 6y ago
What you're saying seems to be consistent with what I'm saying. Web-browser standard DOM is slow, for exactly the reasons you state. Adding an additional tree mutation API, not supported by any web browser, can obviously be a lot faster, and as you say, a good way to make that faster is to group the mutation into a transaction, rather than effectively have one transaction for each appendChild and setAttribute call. What you've done is a pretty small change to the DOM and generally compatible with it. What I've done in Crochet is a more radical rework, but certainly with some of the same general principles (my `Mutation` datatype is also a transaction). I should also point out that my tree mutation is a bit more limited, as it can't do general movement of subtrees, only sibling movement within the same parent (and, in the current prototype, no swapping, only order-preserving, which includes insert, delete, and update). If some compatibility with browser DOM is a goal, then your approach is appealing. I'm not pursuing it in my prototype because I'm also holding overall system simplicity as a whole. But in any case, I agree that people should be aware of what Sciter does; it's an impressive system.
- c-smile 6y ago
  We just need to add one method to Web API: document.mutate(function(mutator) { mutator.changeAttributes(el,{...}); mutator.appendNodes(el,...); ... }); The mutate will do transactional update as fast as possible.

cryptMackay 6y ago

Makes sense!

weeboid 6y ago

Man, reading this just reinforces that DOMs and HTML are not "right" for webapps. These frameworks, contrivances, and the resulting principles will spiral on, never solving the real issue: DOCUMENT Object Models and HYPER TEXT MARKUP Language are for representing linked documents, not stateful object based applications. It's right there, in the names.

fallennode 6y ago
Is that true though? In the article he mentions how a lot of the web frameworks don't have to worry about handling tabindex because the browser does it for them. I have also yet to find another non-web UI framework that handles styling nearly as well as CSS. Between flexbox, grid, and absolute positioning, you have so much power for creating layouts. Throw in some drop shadows and borders to help highlight the important UI pieces and bam. I think the common critique of React, Vue, etc on HN is fair when talking about people building simple websites with these highly sophisticated and massively overkill tools. But when it comes to applications I have yet to find something nearly as flexible and good looking with so little work. And they do it by using HTML and CSS which as it turns out are pretty good abstractions to build on. Or at least they are from my perspective as someone who's been using a variety of these frameworks for the past 5 years or so, on quite a few medium sized projects.

svilen_dobrev 6y ago

looong time ago, we've discussed with a colleague who was building some UI for a game as side project, about what "hierarchies" (essentialy, aspects of structure) are there in an UI. So we ended with about 4: * encapsulation - what includes what / belongs to what * visual - what overlays/obscures what * event-passing - if X gets clicked, or otherwise interacted with, who else would get the event passed to/notified * ??? can't remember this one.. but i remember it took many conversations to find out we need it. Probably was dependency-linkage (in aspects other than above 3, e.g. some model-logic dependencies) This aspect-distinguishing may or may not be usable for you. For example, a usual "scrollbar" is composed of multiple smaller elements, which are tied together in all above aspects. But it does not have to be like that, e.g. the position-mover ("thumb") and the lower/higher buttons need not be on same visual place, e.g. a circle volume knob and separate quieter + louder knobs elsewhere (and that may be very far). One may argue it's not a scrollbar anymore.. expect it does the same thing, just visually differently. It would be interesting if one can really separate these aspects in the code, somehow. That said, how would your toolkit handle this: a small "innocent" interaction that causes lots of small changes all over the tree. Think taxforms: a form, mixture of subforms and fields, many levels deep, and i need to highlight the user-changed fields (or subforms containing changed fields), all the way up. What i ended up recently, in React (i.e. in DOM-recipe-tree), is, anytime the user presses some keystroke in some text field down in the depths, the whole thing needs be repainted. A few hundred fields probably. Which.. isn't that fast anymore, when depth goes beyond 3-4. (From user perspective there are 2-3 times less levels than programatical). Probably i've missed some "i have not changed, don't repaint me" step somewhere.

rangoon626 6y ago

The web does it better, and the web does it first. But I am glad to see native developers finally get a half-baked, sparsely documented reactive UI framework (still a year later!), and like a decade late. Apple is over, and it’s hilarious!

archagon 6y ago

> I think the debate is now essentially over, the reactive approach is winning. Is this really true? My intuition is that reactive UIs are fantastic for widgets and "wraps a database" type apps, but fall flat when it comes to deeply interactive software where performance and responsiveness is paramount — i.e. where there's a tight feedback loop between user input and rendering, and where the model state tends to be quite large and granular. (DAWs, drawing/illustration software, movie editors, mapping applications, text editors, etc.) I also get the sense that most revolutionary or industry-changing software falls into this category. So how can the reactive approach be anything other than a small tool in the app developer's toolbox? Could somebody dissuade me from this notion? In my admittedly limited experience with the reactive paradigm, I feel like I run into a wall as soon as I've implemented all the basic widgets and need to actually work on the main content view of my app. It doesn't feel like it could ever be a good fit for software that pushes boundaries. (Perhaps this is an entirely separate matter from "object-oriented UI" vs. "reactive UI", though. Or maybe I'm "using it wrong", in the sense that reactive UIs are meant to be mostly about widgets and not complex interactive views. But then I often hear people talk about reactive UI as the future of app development, so I'm not sure.)