7 ms·

In my testing, I went through a bunch of Rust frameworks that sit on top of the core OS UI frameworks. My experience with all of those was that they all capped

In my testing, I went through a bunch of Rust frameworks that sit on top of the core OS UI frameworks. My experience with all of those was that they all capped out at about 3k rectangles on a screen at a time at 30 fps (vs the about 80k rectangles that fit at usually 30-60 fps in my demo). I wonder if your experience is different? As best I've been able to tell, right now you have to make a choice between (a) fast and (b) native, and you only get to choose one. And in my application domain, fast is nonnegotiable.

mtsr 3y ago
Mixed UIs could probably work well here. Using a canvas and specific gpu rendering just for the part requiring high throughput makes sense.
josephg 3y ago
What domain are you working in? Smoothly rendering 3k rectangles isn't a common use case for UI libraries, so I'm not surprised they perform poorly. They generally aren't optimized for that. The normal answer for weird, high performance rendering like that is to just use opengl or something and program against the GPU directly. 3d rendering contexts can render millions of rectangles on modern hardware without breaking a sweat. This advice is the same on desktop, mobile or the web. I'm not surprised rust UI frameworks have the same limitation. Most UIs don't need thousands of rectangles. But they do need lots of custom widgets, and implementing all of those well is a higher priority job for anyone building a custom UI library.
- eslaught 3y ago
  Like I said in my top comment, it's a profiler. I need to consume logs from distributed applications (on possibly tens of thousands of nodes) and render them in as coherent a fashion as possible. Going to 3d rendering via OpenGL / WebGL was one of the options I considered. But honestly, this is a boatload of complexity. Why do I need to pick up a full 3D rendering library just to draw rectangles on the screen? Plus, consider: the slot viewer (where the rectangles are shown) still needs to draw some UI components (e.g., on hoverover). I still need that UI integration there, even if it's not as performance-sensitive. So I end up needing something along the lines of egui anyway. I definitely understand that not all UI libraries are going to be optimized for this. But I was surprised that it wasn't easy to e.g. get a high-performance canvas dropped into libraries that specifically advertised themselves as high-performance.
  - IX-103 3y ago
    The reason to pick up a "a full 3D rendering library" is that it is actually a lower level interface to the hardware. Almost all 2D operations are now implemented on top of 3D hardware primitives. By limiting yourself to a 2D engine you're ensuring there are more layers of abstraction between you and the hardware. While # layers of abstraction doesn't always correlate with overhead, it does place limits on performance.
    - josephg 3y ago
      Yeah. And it’s not one or the other. On the web you can make a “3d” rendering surface for the profile traces and use absolutely positioned DOM elements rendered on top for you hover-over widgets and things. It’s still a bit of a mess though. I understand wanting to keep everything in a single UI library if you can.
  - josephg 3y ago
    I was curious so I took a look at firefox's profiler. It looks like it does a hybrid render, using a couple of canvas elements for the flame graph and the stack trace (up the top), but the rest of the UI is rendered using plain old DOM elements. It performs great, despite the whole thing being pure javascript. Example: https://share.firefox.dev/40QUthv https://share.firefox.dev/40QUthv . (Though this trace is probably smaller than the profiling traces you're demoing.)
danShumway 3y ago
Should you be changing 3K rectangles of your interface at 30fps? This seems like a very narrow use case that should be restricted to using Canvas with very specific parts of the application. Maybe you have graphs/charts that need that kind of rendering effect. Great, use Canvas for them, they're not accessible anyway. But if something like your main user controls are running into that problem, I feel like something has probably gone wrong with your user interface. In the demo you linked, it makes a lot of sense to render to Canvas for the individual charts, flame charts, breakdowns. It does not make sense to me to render to Canvas for the sidebar. It doesn't make sense to me that the list element itself holding the charts is going through Canvas. That's something in your UI that honestly shouldn't be updating at the speed you're talking about.
- eslaught 3y ago
  I don't know if you noticed, but my test has on the order of 10 billion rectangles. Only about 80k are drawn on the screen at once because I do very aggressive culling. To be honest, I'm not sure Canvas can handle either part of this: either drawing 80k rectangles at 60 fps or culling so that the other 10 billion don't need to waste CPU cycles. It requires a very tight integration with the UI library to make sure we do this fast. Yes, it's a very narrow use case. That's why I went with a specialized UI library.
  - danShumway 3y ago
    > I'm not sure Canvas can handle either part of this To be clear, WebGL/WebGPU is still using Canvas. You're not using the 2D api stuff, but that's just a high-level API on top of Canvas. You're still rendering to Canvas. > I don't know if you noticed, but my test has on the order of 10 billion rectangles Why? I've built apps like this before. Your choice is not "abandon the DOM" or "have zero performance." You can have most of your interface in the DOM and use Canvas (again WebGL/GPU is still Canvas) for specifically the parts of your app that wouldn't be accessible anyway (mainly charts). It does not require tight integration with the GPU to render the sliders on the left hand side of the screen in this demo. No one is saying that you need to render your charts in DOM. We're wondering why you got rid of `ul` elements.

mtsr 3y ago

Mixed UIs could probably work well here. Using a canvas and specific gpu rendering just for the part requiring high throughput makes sense.

josephg 3y ago

What domain are you working in? Smoothly rendering 3k rectangles isn't a common use case for UI libraries, so I'm not surprised they perform poorly. They generally aren't optimized for that. The normal answer for weird, high performance rendering like that is to just use opengl or something and program against the GPU directly. 3d rendering contexts can render millions of rectangles on modern hardware without breaking a sweat. This advice is the same on desktop, mobile or the web. I'm not surprised rust UI frameworks have the same limitation. Most UIs don't need thousands of rectangles. But they do need lots of custom widgets, and implementing all of those well is a higher priority job for anyone building a custom UI library.

eslaught 3y ago
Like I said in my top comment, it's a profiler. I need to consume logs from distributed applications (on possibly tens of thousands of nodes) and render them in as coherent a fashion as possible. Going to 3d rendering via OpenGL / WebGL was one of the options I considered. But honestly, this is a boatload of complexity. Why do I need to pick up a full 3D rendering library just to draw rectangles on the screen? Plus, consider: the slot viewer (where the rectangles are shown) still needs to draw some UI components (e.g., on hoverover). I still need that UI integration there, even if it's not as performance-sensitive. So I end up needing something along the lines of egui anyway. I definitely understand that not all UI libraries are going to be optimized for this. But I was surprised that it wasn't easy to e.g. get a high-performance canvas dropped into libraries that specifically advertised themselves as high-performance.
- IX-103 3y ago
  The reason to pick up a "a full 3D rendering library" is that it is actually a lower level interface to the hardware. Almost all 2D operations are now implemented on top of 3D hardware primitives. By limiting yourself to a 2D engine you're ensuring there are more layers of abstraction between you and the hardware. While # layers of abstraction doesn't always correlate with overhead, it does place limits on performance.
  - josephg 3y ago
    Yeah. And it’s not one or the other. On the web you can make a “3d” rendering surface for the profile traces and use absolutely positioned DOM elements rendered on top for you hover-over widgets and things. It’s still a bit of a mess though. I understand wanting to keep everything in a single UI library if you can.
- josephg 3y ago
  I was curious so I took a look at firefox's profiler. It looks like it does a hybrid render, using a couple of canvas elements for the flame graph and the stack trace (up the top), but the rest of the UI is rendered using plain old DOM elements. It performs great, despite the whole thing being pure javascript. Example: https://share.firefox.dev/40QUthv https://share.firefox.dev/40QUthv . (Though this trace is probably smaller than the profiling traces you're demoing.)

danShumway 3y ago

Should you be changing 3K rectangles of your interface at 30fps? This seems like a very narrow use case that should be restricted to using Canvas with very specific parts of the application. Maybe you have graphs/charts that need that kind of rendering effect. Great, use Canvas for them, they're not accessible anyway. But if something like your main user controls are running into that problem, I feel like something has probably gone wrong with your user interface. In the demo you linked, it makes a lot of sense to render to Canvas for the individual charts, flame charts, breakdowns. It does not make sense to me to render to Canvas for the sidebar. It doesn't make sense to me that the list element itself holding the charts is going through Canvas. That's something in your UI that honestly shouldn't be updating at the speed you're talking about.

eslaught 3y ago
I don't know if you noticed, but my test has on the order of 10 billion rectangles. Only about 80k are drawn on the screen at once because I do very aggressive culling. To be honest, I'm not sure Canvas can handle either part of this: either drawing 80k rectangles at 60 fps or culling so that the other 10 billion don't need to waste CPU cycles. It requires a very tight integration with the UI library to make sure we do this fast. Yes, it's a very narrow use case. That's why I went with a specialized UI library.
- danShumway 3y ago
  > I'm not sure Canvas can handle either part of this To be clear, WebGL/WebGPU is still using Canvas. You're not using the 2D api stuff, but that's just a high-level API on top of Canvas. You're still rendering to Canvas. > I don't know if you noticed, but my test has on the order of 10 billion rectangles Why? I've built apps like this before. Your choice is not "abandon the DOM" or "have zero performance." You can have most of your interface in the DOM and use Canvas (again WebGL/GPU is still Canvas) for specifically the parts of your app that wouldn't be accessible anyway (mainly charts). It does not require tight integration with the GPU to render the sliders on the left hand side of the screen in this demo. No one is saying that you need to render your charts in DOM. We're wondering why you got rid of `ul` elements.