3 ms·

Should you be changing 3K rectangles of your interface at 30fps? This seems like a very narrow use case that should be restricted to using Canvas with very spec

Should you be changing 3K rectangles of your interface at 30fps? This seems like a very narrow use case that should be restricted to using Canvas with very specific parts of the application. Maybe you have graphs/charts that need that kind of rendering effect. Great, use Canvas for them, they're not accessible anyway. But if something like your main user controls are running into that problem, I feel like something has probably gone wrong with your user interface. In the demo you linked, it makes a lot of sense to render to Canvas for the individual charts, flame charts, breakdowns. It does not make sense to me to render to Canvas for the sidebar. It doesn't make sense to me that the list element itself holding the charts is going through Canvas. That's something in your UI that honestly shouldn't be updating at the speed you're talking about.

eslaught 4y ago
I don't know if you noticed, but my test has on the order of 10 billion rectangles. Only about 80k are drawn on the screen at once because I do very aggressive culling. To be honest, I'm not sure Canvas can handle either part of this: either drawing 80k rectangles at 60 fps or culling so that the other 10 billion don't need to waste CPU cycles. It requires a very tight integration with the UI library to make sure we do this fast. Yes, it's a very narrow use case. That's why I went with a specialized UI library.
- danShumway 4y ago
  > I'm not sure Canvas can handle either part of this To be clear, WebGL/WebGPU is still using Canvas. You're not using the 2D api stuff, but that's just a high-level API on top of Canvas. You're still rendering to Canvas. > I don't know if you noticed, but my test has on the order of 10 billion rectangles Why? I've built apps like this before. Your choice is not "abandon the DOM" or "have zero performance." You can have most of your interface in the DOM and use Canvas (again WebGL/GPU is still Canvas) for specifically the parts of your app that wouldn't be accessible anyway (mainly charts). It does not require tight integration with the GPU to render the sliders on the left hand side of the screen in this demo. No one is saying that you need to render your charts in DOM. We're wondering why you got rid of `ul` elements.
  - eslaught 4y ago
    If you build an equivalent tech demo in HTML Canvas, I'd be curious to see how it performs. I'm sure you're aware that, just because Canvas layers on top of WebGL/GPU draw calls, doesn't necessarily mean it's performant in the way I need it to be. There are many ways for performance to degrade across abstraction layers, even if in principle it shouldn't. I didn't test Canvas specifically in my explorations, but I did test a few JavaScript based frameworks, and the ones I found that were efficient sat on top of WebGL, not Canvas. Ultimately I didn't pick those because they had a bunch of other stuff baked in that I didn't need or want, and were already equally non-native (sitting on top of WebGL). And I frankly didn't want to be writing JavaScript.
    - danShumway 4y ago
      > just because Canvas layers on top of WebGL/GPU draw calls, doesn't necessarily mean it's performant in the way I need it to be You are using Canvas right now. WebGL uses Canvas to render to the screen. If you open up your demo's DOM inspector right now, you will see a canvas element. It's: <canvas id="the_canvas_id" width="1914" height="580" style="width: 1914px; height: 580px;"></canvas> It's a little bit hard for me to trust this kind of dismissive talk about the browser adding too many abstraction layers when it's not clear to me that you're fully aware of what browser features your demo is using right now. Targetting the DOM and using embedded canvases really doesn't have anything to do with whether or not you're writing Javascript, and it doesn't mean you can't use WebGL to render charts.
      - josephg 4y ago
        > You are using Canvas right now. WebGL uses Canvas to render to the screen. Ehhh sort of. Canvas is much older than webgl. When we talked about canvas a few years ago, that always referred to the canvas DOM element and its associated 2D shape based API. (fillRect / moveTo / lineTo / etc). When Webgl appeared, the nomenclature was "using canvas" vs "using webgl", since webgl's API is completely different from the 2d canvas API. But browser manufacturers didn't give webgl its own DOM element. Webgl also uses a canvas DOM element, then configures it differently. (getContext("webgl") instead of getContext("2d")). So if you're using webgl (which I think this app is doing), then you're "using canvas" in the sense that your HTML contains a <canvas> element. But you aren't using the canvas 2d API - which, as I said, is totally different from webgl and is also called "canvas" because it was here first. And it doesn't have another name. In defence of the GP comment, I think its pretty clear from context that "Canvas layers on top of WebGL/GPU draw calls" refers to the canvas API, not the canvas DOM element.
anderspitman 4y ago
Maybe I'm misunderstanding you, but shouldn't this be the GUI toolkit's job, ie choosing what to render as native HTML vs what needs to rendered to canvas? Can you recommend an alternative to egui that let's you write entirely in Rust and get the performance of egui but still maintain some level of accessibility?
- danShumway 4y ago
  > shouldn't this be the GUI toolkit's job Ideally yes. This is a failing of the egui toolkit, not necessarily a failing of the people using it. The whole point of a high-level toolkit like this is that you shouldn't have to worry about how things are being rendered. However, it is still important to counter claims that egui is making if the people using the toolkit are parroting those claims. > Can you recommend an alternative to egui that let's you write entirely in Rust and get the performance of egui but still maintain some level of accessibility? I don't think I've ever built a web GUI in Rust so I have no idea what the ecosystem looks like. To my detriment, I tend to build most of my UIs from scratch. I can promise that it is possible to have performant UIs with a lot of embedded charts (like this demo has) using the DOM, but I have no idea what Rust framework devs have been getting up to or publishing. There's also a little bit of a catch-22 here because yes, any time you introduce the DOM, you are going to lose some performance. That doesn't mean that you're going to lose so much performance that the demo above would be stuttery or wouldn't work. But if you're just measuring "how many quads can I refresh on the screen", that's kind of missing the point that you really shouldn't be refreshing thousands and thousands of quads of information in a UI at 60fps. That's not a thing that most interfaces should be doing in the first place. And if you are genuinely changing thousands of boxes of information on a screen at the same time, kind of by definition that isn't accessible period. There is no way to present that volume of informational change to a screen reader in a way that will be understandable. But a lot of people when building UIs tend to think they're in the rare cases where they need that level of performance and so they need to sacrifice that accessibility, when in reality what they actually have is a giant list of charts and aside from the actual chart rendering, there's nothing particularly expensive going on.

eslaught 4y ago

I don't know if you noticed, but my test has on the order of 10 billion rectangles. Only about 80k are drawn on the screen at once because I do very aggressive culling. To be honest, I'm not sure Canvas can handle either part of this: either drawing 80k rectangles at 60 fps or culling so that the other 10 billion don't need to waste CPU cycles. It requires a very tight integration with the UI library to make sure we do this fast. Yes, it's a very narrow use case. That's why I went with a specialized UI library.

danShumway 4y ago
> I'm not sure Canvas can handle either part of this To be clear, WebGL/WebGPU is still using Canvas. You're not using the 2D api stuff, but that's just a high-level API on top of Canvas. You're still rendering to Canvas. > I don't know if you noticed, but my test has on the order of 10 billion rectangles Why? I've built apps like this before. Your choice is not "abandon the DOM" or "have zero performance." You can have most of your interface in the DOM and use Canvas (again WebGL/GPU is still Canvas) for specifically the parts of your app that wouldn't be accessible anyway (mainly charts). It does not require tight integration with the GPU to render the sliders on the left hand side of the screen in this demo. No one is saying that you need to render your charts in DOM. We're wondering why you got rid of `ul` elements.
- eslaught 4y ago
  If you build an equivalent tech demo in HTML Canvas, I'd be curious to see how it performs. I'm sure you're aware that, just because Canvas layers on top of WebGL/GPU draw calls, doesn't necessarily mean it's performant in the way I need it to be. There are many ways for performance to degrade across abstraction layers, even if in principle it shouldn't. I didn't test Canvas specifically in my explorations, but I did test a few JavaScript based frameworks, and the ones I found that were efficient sat on top of WebGL, not Canvas. Ultimately I didn't pick those because they had a bunch of other stuff baked in that I didn't need or want, and were already equally non-native (sitting on top of WebGL). And I frankly didn't want to be writing JavaScript.
  - danShumway 4y ago
    > just because Canvas layers on top of WebGL/GPU draw calls, doesn't necessarily mean it's performant in the way I need it to be You are using Canvas right now. WebGL uses Canvas to render to the screen. If you open up your demo's DOM inspector right now, you will see a canvas element. It's: <canvas id="the_canvas_id" width="1914" height="580" style="width: 1914px; height: 580px;"></canvas> It's a little bit hard for me to trust this kind of dismissive talk about the browser adding too many abstraction layers when it's not clear to me that you're fully aware of what browser features your demo is using right now. Targetting the DOM and using embedded canvases really doesn't have anything to do with whether or not you're writing Javascript, and it doesn't mean you can't use WebGL to render charts.
    - josephg 4y ago
      > You are using Canvas right now. WebGL uses Canvas to render to the screen. Ehhh sort of. Canvas is much older than webgl. When we talked about canvas a few years ago, that always referred to the canvas DOM element and its associated 2D shape based API. (fillRect / moveTo / lineTo / etc). When Webgl appeared, the nomenclature was "using canvas" vs "using webgl", since webgl's API is completely different from the 2d canvas API. But browser manufacturers didn't give webgl its own DOM element. Webgl also uses a canvas DOM element, then configures it differently. (getContext("webgl") instead of getContext("2d")). So if you're using webgl (which I think this app is doing), then you're "using canvas" in the sense that your HTML contains a <canvas> element. But you aren't using the canvas 2d API - which, as I said, is totally different from webgl and is also called "canvas" because it was here first. And it doesn't have another name. In defence of the GP comment, I think its pretty clear from context that "Canvas layers on top of WebGL/GPU draw calls" refers to the canvas API, not the canvas DOM element.

anderspitman 4y ago

Maybe I'm misunderstanding you, but shouldn't this be the GUI toolkit's job, ie choosing what to render as native HTML vs what needs to rendered to canvas? Can you recommend an alternative to egui that let's you write entirely in Rust and get the performance of egui but still maintain some level of accessibility?

danShumway 4y ago
> shouldn't this be the GUI toolkit's job Ideally yes. This is a failing of the egui toolkit, not necessarily a failing of the people using it. The whole point of a high-level toolkit like this is that you shouldn't have to worry about how things are being rendered. However, it is still important to counter claims that egui is making if the people using the toolkit are parroting those claims. > Can you recommend an alternative to egui that let's you write entirely in Rust and get the performance of egui but still maintain some level of accessibility? I don't think I've ever built a web GUI in Rust so I have no idea what the ecosystem looks like. To my detriment, I tend to build most of my UIs from scratch. I can promise that it is possible to have performant UIs with a lot of embedded charts (like this demo has) using the DOM, but I have no idea what Rust framework devs have been getting up to or publishing. There's also a little bit of a catch-22 here because yes, any time you introduce the DOM, you are going to lose some performance. That doesn't mean that you're going to lose so much performance that the demo above would be stuttery or wouldn't work. But if you're just measuring "how many quads can I refresh on the screen", that's kind of missing the point that you really shouldn't be refreshing thousands and thousands of quads of information in a UI at 60fps. That's not a thing that most interfaces should be doing in the first place. And if you are genuinely changing thousands of boxes of information on a screen at the same time, kind of by definition that isn't accessible period. There is no way to present that volume of informational change to a screen reader in a way that will be understandable. But a lot of people when building UIs tend to think they're in the rare cases where they need that level of performance and so they need to sacrifice that accessibility, when in reality what they actually have is a giant list of charts and aside from the actual chart rendering, there's nothing particularly expensive going on.