4 ms·

And scrolling is unavoidably all wrong: dragging two fingers from one end of my touchpad to the other should scroll by about four screenfuls, but only manages t

And scrolling is unavoidably all wrong: dragging two fingers from one end of my touchpad to the other should scroll by about four screenfuls, but only manages two thirds of one screenful; a quick swipe should send it dozens of screenfuls, with inertia, but manages even less than that two thirds of a screenful; and these limitations are fundamental due to the web not possessing the right primitives. (Then there are more fixable bugs like that, after focusing the page, you have to move the mouse before scrolling will work.) And input element behaviour is non-native, which will never be properly fixable since different UAs have different behaviours, some of which cannot be emulated even if you sniff which platform you’re on (e.g. scroll bar grab and drag behaviour when you move far outside the window). (Then there are also simple bugs: holding Left or Right while the range element is focused only bumps the value once, not according to key repeat. And Up and Down don’t work.) I have written about these things a number of times, fundamental limitations of the pure-canvas approach that make it unavoidably and permanently unsuitable. https://news.ycombinator.com/item?id=33861831 https://news.ycombinator.com/item?id=33861831 is probably the best thread. (And just for a bit of fun: the demo can have slightly negative utilisation, drawing outside the box!)

eslaught 4y ago
In my testing, I went through a bunch of Rust frameworks that sit on top of the core OS UI frameworks. My experience with all of those was that they all capped out at about 3k rectangles on a screen at a time at 30 fps (vs the about 80k rectangles that fit at usually 30-60 fps in my demo). I wonder if your experience is different? As best I've been able to tell, right now you have to make a choice between (a) fast and (b) native, and you only get to choose one. And in my application domain, fast is nonnegotiable.
- mtsr 4y ago
  Mixed UIs could probably work well here. Using a canvas and specific gpu rendering just for the part requiring high throughput makes sense.
- josephg 4y ago
  What domain are you working in? Smoothly rendering 3k rectangles isn't a common use case for UI libraries, so I'm not surprised they perform poorly. They generally aren't optimized for that. The normal answer for weird, high performance rendering like that is to just use opengl or something and program against the GPU directly. 3d rendering contexts can render millions of rectangles on modern hardware without breaking a sweat. This advice is the same on desktop, mobile or the web. I'm not surprised rust UI frameworks have the same limitation. Most UIs don't need thousands of rectangles. But they do need lots of custom widgets, and implementing all of those well is a higher priority job for anyone building a custom UI library.
  - eslaught 4y ago
    Like I said in my top comment, it's a profiler. I need to consume logs from distributed applications (on possibly tens of thousands of nodes) and render them in as coherent a fashion as possible. Going to 3d rendering via OpenGL / WebGL was one of the options I considered. But honestly, this is a boatload of complexity. Why do I need to pick up a full 3D rendering library just to draw rectangles on the screen? Plus, consider: the slot viewer (where the rectangles are shown) still needs to draw some UI components (e.g., on hoverover). I still need that UI integration there, even if it's not as performance-sensitive. So I end up needing something along the lines of egui anyway. I definitely understand that not all UI libraries are going to be optimized for this. But I was surprised that it wasn't easy to e.g. get a high-performance canvas dropped into libraries that specifically advertised themselves as high-performance.
    - IX-103 4y ago
      The reason to pick up a "a full 3D rendering library" is that it is actually a lower level interface to the hardware. Almost all 2D operations are now implemented on top of 3D hardware primitives. By limiting yourself to a 2D engine you're ensuring there are more layers of abstraction between you and the hardware. While # layers of abstraction doesn't always correlate with overhead, it does place limits on performance.
      - 4y ago
- danShumway 4y ago
  Should you be changing 3K rectangles of your interface at 30fps? This seems like a very narrow use case that should be restricted to using Canvas with very specific parts of the application. Maybe you have graphs/charts that need that kind of rendering effect. Great, use Canvas for them, they're not accessible anyway. But if something like your main user controls are running into that problem, I feel like something has probably gone wrong with your user interface. In the demo you linked, it makes a lot of sense to render to Canvas for the individual charts, flame charts, breakdowns. It does not make sense to me to render to Canvas for the sidebar. It doesn't make sense to me that the list element itself holding the charts is going through Canvas. That's something in your UI that honestly shouldn't be updating at the speed you're talking about.
  - eslaught 4y ago
    I don't know if you noticed, but my test has on the order of 10 billion rectangles. Only about 80k are drawn on the screen at once because I do very aggressive culling. To be honest, I'm not sure Canvas can handle either part of this: either drawing 80k rectangles at 60 fps or culling so that the other 10 billion don't need to waste CPU cycles. It requires a very tight integration with the UI library to make sure we do this fast. Yes, it's a very narrow use case. That's why I went with a specialized UI library.
    - danShumway 4y ago
      > I'm not sure Canvas can handle either part of this To be clear, WebGL/WebGPU is still using Canvas. You're not using the 2D api stuff, but that's just a high-level API on top of Canvas. You're still rendering to Canvas. > I don't know if you noticed, but my test has on the order of 10 billion rectangles Why? I've built apps like this before. Your choice is not "abandon the DOM" or "have zero performance." You can have most of your interface in the DOM and use Canvas (again WebGL/GPU is still Canvas) for specifically the parts of your app that wouldn't be accessible anyway (mainly charts). It does not require tight integration with the GPU to render the sliders on the left hand side of the screen in this demo. No one is saying that you need to render your charts in DOM. We're wondering why you got rid of `ul` elements.
rkagerer 4y ago
Well, in fairness web-based applications (including Electron ones running locally) break most of the inspectability benefits that came before, at least on the Windows platform. eg. I used to be able to programmatically grab hWnd's (like Spy++ does), and manipulate the forms of an app. Apps used to show up as one (or two) simple processes with rich instrumentation metrics in tools like Resource Monitor. Chrome abstracted all that away in an effort to 'displace' the OS. I think canvas-only technologies could eventually do the same (the endless cycle continues) and look forward to some of the benefits (eg. more deterministic and straightforward positioning and layout). But I strongly agree they need to accommodate most of the features you mentioned and until they do it's two steps backward. Do appreciate your thoughtful comment and the concise summary of some fundamental limitations that are presently hard barriers.
- horsawlarway 4y ago
  > Well, in fairness web-based applications (including Electron ones running locally) break most of the inspectability benefits that came before, at least on the Windows platform. eg. I used to be able to programmatically grab hWnd's (like Spy++ does), and manipulate the forms of an app. Apps used to show up as one (or two) simple processes with rich instrumentation metrics in tools like Resource Monitor. --- I disagree pretty hard with this sentiment. Yes, it's harder to programmatically interact with some of the browser features from outside of the browser. Programmatic interaction with basically any application that isn't yours is getting harder across every operating system as security becomes more important. That doesn't really change that introspection of the source that makes up a displayed site was easy and approachable. I can look at the css/html/js. Further, all of those resources are contained in files intended to be opened and looked at by humans (they're human readable text). This is my biggest issue with WASM. You're literally back at reading assembly again. And while that's possible, and there is tooling that can help - it's just not the same as being able to right-click on something, alter a couple of attributes in human readable words, hit enter and see that change.
  - josephg 4y ago
    > This is my biggest issue with WASM. You're literally back at reading assembly again. And while that's possible, and there is tooling that can help - it's just not the same as being able to right-click on something, alter a couple of attributes in human readable words, hit enter and see that change. WASM on its own isn't the problem. WASM just lets you use another language to replace javascript. When you right click on something in a browser and alter human-readable attributes, you're usually messing with CSS and HTML. Any half decent web UI framework should pass HTML and CSS to the browser in order to render the UI. Thats just as true with wasm frameworks as it is with javascript frameworks. The problem in this example is that the wasm code is bundling its own UI library that is trying (badly) to replace the browser's layout engine. All the browser can see is a bunch of drawing calls to an html canvas. Thats not the fault of wasm. Its just this UI library, which isn't really designed for the web.

eslaught 4y ago

In my testing, I went through a bunch of Rust frameworks that sit on top of the core OS UI frameworks. My experience with all of those was that they all capped out at about 3k rectangles on a screen at a time at 30 fps (vs the about 80k rectangles that fit at usually 30-60 fps in my demo). I wonder if your experience is different? As best I've been able to tell, right now you have to make a choice between (a) fast and (b) native, and you only get to choose one. And in my application domain, fast is nonnegotiable.

mtsr 4y ago
Mixed UIs could probably work well here. Using a canvas and specific gpu rendering just for the part requiring high throughput makes sense.
josephg 4y ago
What domain are you working in? Smoothly rendering 3k rectangles isn't a common use case for UI libraries, so I'm not surprised they perform poorly. They generally aren't optimized for that. The normal answer for weird, high performance rendering like that is to just use opengl or something and program against the GPU directly. 3d rendering contexts can render millions of rectangles on modern hardware without breaking a sweat. This advice is the same on desktop, mobile or the web. I'm not surprised rust UI frameworks have the same limitation. Most UIs don't need thousands of rectangles. But they do need lots of custom widgets, and implementing all of those well is a higher priority job for anyone building a custom UI library.
- eslaught 4y ago
  Like I said in my top comment, it's a profiler. I need to consume logs from distributed applications (on possibly tens of thousands of nodes) and render them in as coherent a fashion as possible. Going to 3d rendering via OpenGL / WebGL was one of the options I considered. But honestly, this is a boatload of complexity. Why do I need to pick up a full 3D rendering library just to draw rectangles on the screen? Plus, consider: the slot viewer (where the rectangles are shown) still needs to draw some UI components (e.g., on hoverover). I still need that UI integration there, even if it's not as performance-sensitive. So I end up needing something along the lines of egui anyway. I definitely understand that not all UI libraries are going to be optimized for this. But I was surprised that it wasn't easy to e.g. get a high-performance canvas dropped into libraries that specifically advertised themselves as high-performance.
  - IX-103 4y ago
    The reason to pick up a "a full 3D rendering library" is that it is actually a lower level interface to the hardware. Almost all 2D operations are now implemented on top of 3D hardware primitives. By limiting yourself to a 2D engine you're ensuring there are more layers of abstraction between you and the hardware. While # layers of abstraction doesn't always correlate with overhead, it does place limits on performance.
    - 4y ago
danShumway 4y ago
Should you be changing 3K rectangles of your interface at 30fps? This seems like a very narrow use case that should be restricted to using Canvas with very specific parts of the application. Maybe you have graphs/charts that need that kind of rendering effect. Great, use Canvas for them, they're not accessible anyway. But if something like your main user controls are running into that problem, I feel like something has probably gone wrong with your user interface. In the demo you linked, it makes a lot of sense to render to Canvas for the individual charts, flame charts, breakdowns. It does not make sense to me to render to Canvas for the sidebar. It doesn't make sense to me that the list element itself holding the charts is going through Canvas. That's something in your UI that honestly shouldn't be updating at the speed you're talking about.
- eslaught 4y ago
  I don't know if you noticed, but my test has on the order of 10 billion rectangles. Only about 80k are drawn on the screen at once because I do very aggressive culling. To be honest, I'm not sure Canvas can handle either part of this: either drawing 80k rectangles at 60 fps or culling so that the other 10 billion don't need to waste CPU cycles. It requires a very tight integration with the UI library to make sure we do this fast. Yes, it's a very narrow use case. That's why I went with a specialized UI library.
  - danShumway 4y ago
    > I'm not sure Canvas can handle either part of this To be clear, WebGL/WebGPU is still using Canvas. You're not using the 2D api stuff, but that's just a high-level API on top of Canvas. You're still rendering to Canvas. > I don't know if you noticed, but my test has on the order of 10 billion rectangles Why? I've built apps like this before. Your choice is not "abandon the DOM" or "have zero performance." You can have most of your interface in the DOM and use Canvas (again WebGL/GPU is still Canvas) for specifically the parts of your app that wouldn't be accessible anyway (mainly charts). It does not require tight integration with the GPU to render the sliders on the left hand side of the screen in this demo. No one is saying that you need to render your charts in DOM. We're wondering why you got rid of `ul` elements.

rkagerer 4y ago

Well, in fairness web-based applications (including Electron ones running locally) break most of the inspectability benefits that came before, at least on the Windows platform. eg. I used to be able to programmatically grab hWnd's (like Spy++ does), and manipulate the forms of an app. Apps used to show up as one (or two) simple processes with rich instrumentation metrics in tools like Resource Monitor. Chrome abstracted all that away in an effort to 'displace' the OS. I think canvas-only technologies could eventually do the same (the endless cycle continues) and look forward to some of the benefits (eg. more deterministic and straightforward positioning and layout). But I strongly agree they need to accommodate most of the features you mentioned and until they do it's two steps backward. Do appreciate your thoughtful comment and the concise summary of some fundamental limitations that are presently hard barriers.

horsawlarway 4y ago
> Well, in fairness web-based applications (including Electron ones running locally) break most of the inspectability benefits that came before, at least on the Windows platform. eg. I used to be able to programmatically grab hWnd's (like Spy++ does), and manipulate the forms of an app. Apps used to show up as one (or two) simple processes with rich instrumentation metrics in tools like Resource Monitor. --- I disagree pretty hard with this sentiment. Yes, it's harder to programmatically interact with some of the browser features from outside of the browser. Programmatic interaction with basically any application that isn't yours is getting harder across every operating system as security becomes more important. That doesn't really change that introspection of the source that makes up a displayed site was easy and approachable. I can look at the css/html/js. Further, all of those resources are contained in files intended to be opened and looked at by humans (they're human readable text). This is my biggest issue with WASM. You're literally back at reading assembly again. And while that's possible, and there is tooling that can help - it's just not the same as being able to right-click on something, alter a couple of attributes in human readable words, hit enter and see that change.
- josephg 4y ago
  > This is my biggest issue with WASM. You're literally back at reading assembly again. And while that's possible, and there is tooling that can help - it's just not the same as being able to right-click on something, alter a couple of attributes in human readable words, hit enter and see that change. WASM on its own isn't the problem. WASM just lets you use another language to replace javascript. When you right click on something in a browser and alter human-readable attributes, you're usually messing with CSS and HTML. Any half decent web UI framework should pass HTML and CSS to the browser in order to render the UI. Thats just as true with wasm frameworks as it is with javascript frameworks. The problem in this example is that the wasm code is bundling its own UI library that is trying (badly) to replace the browser's layout engine. All the browser can see is a bunch of drawing calls to an html canvas. Thats not the fault of wasm. Its just this UI library, which isn't really designed for the web.