5 ms·

Cool demo, but by the looks of things the whole site is rendered into a canvas element. As a result, the browser's dev tools are useless, it won't work with scr

Cool demo, but by the looks of things the whole site is rendered into a canvas element. As a result, the browser's dev tools are useless, it won't work with screen readers, and you can't copy+paste text. And any text rendering and input elements will be non-native. (So, no nice iphone controls. No native / system configurable keyboard shortcuts. And so on.) I sincerely hope the web as a whole doesn't move in this direction. The inspectibility of the DOM is one of the web's greatest strengths. Lets not throw that away.

deleted 4y ago
[deleted]
chrismorgan 4y ago
And scrolling is unavoidably all wrong: dragging two fingers from one end of my touchpad to the other should scroll by about four screenfuls, but only manages two thirds of one screenful; a quick swipe should send it dozens of screenfuls, with inertia, but manages even less than that two thirds of a screenful; and these limitations are fundamental due to the web not possessing the right primitives. (Then there are more fixable bugs like that, after focusing the page, you have to move the mouse before scrolling will work.) And input element behaviour is non-native, which will never be properly fixable since different UAs have different behaviours, some of which cannot be emulated even if you sniff which platform you’re on (e.g. scroll bar grab and drag behaviour when you move far outside the window). (Then there are also simple bugs: holding Left or Right while the range element is focused only bumps the value once, not according to key repeat. And Up and Down don’t work.) I have written about these things a number of times, fundamental limitations of the pure-canvas approach that make it unavoidably and permanently unsuitable. https://news.ycombinator.com/item?id=33861831 https://news.ycombinator.com/item?id=33861831 is probably the best thread. (And just for a bit of fun: the demo can have slightly negative utilisation, drawing outside the box!)
- eslaught 4y ago
  In my testing, I went through a bunch of Rust frameworks that sit on top of the core OS UI frameworks. My experience with all of those was that they all capped out at about 3k rectangles on a screen at a time at 30 fps (vs the about 80k rectangles that fit at usually 30-60 fps in my demo). I wonder if your experience is different? As best I've been able to tell, right now you have to make a choice between (a) fast and (b) native, and you only get to choose one. And in my application domain, fast is nonnegotiable.
  - mtsr 4y ago
    Mixed UIs could probably work well here. Using a canvas and specific gpu rendering just for the part requiring high throughput makes sense.
  - josephg 4y ago
    What domain are you working in? Smoothly rendering 3k rectangles isn't a common use case for UI libraries, so I'm not surprised they perform poorly. They generally aren't optimized for that. The normal answer for weird, high performance rendering like that is to just use opengl or something and program against the GPU directly. 3d rendering contexts can render millions of rectangles on modern hardware without breaking a sweat. This advice is the same on desktop, mobile or the web. I'm not surprised rust UI frameworks have the same limitation. Most UIs don't need thousands of rectangles. But they do need lots of custom widgets, and implementing all of those well is a higher priority job for anyone building a custom UI library.
    - eslaught 4y ago
      Like I said in my top comment, it's a profiler. I need to consume logs from distributed applications (on possibly tens of thousands of nodes) and render them in as coherent a fashion as possible. Going to 3d rendering via OpenGL / WebGL was one of the options I considered. But honestly, this is a boatload of complexity. Why do I need to pick up a full 3D rendering library just to draw rectangles on the screen? Plus, consider: the slot viewer (where the rectangles are shown) still needs to draw some UI components (e.g., on hoverover). I still need that UI integration there, even if it's not as performance-sensitive. So I end up needing something along the lines of egui anyway. I definitely understand that not all UI libraries are going to be optimized for this. But I was surprised that it wasn't easy to e.g. get a high-performance canvas dropped into libraries that specifically advertised themselves as high-performance.
      - IX-103 4y ago
        The reason to pick up a "a full 3D rendering library" is that it is actually a lower level interface to the hardware. Almost all 2D operations are now implemented on top of 3D hardware primitives. By limiting yourself to a 2D engine you're ensuring there are more layers of abstraction between you and the hardware. While # layers of abstraction doesn't always correlate with overhead, it does place limits on performance.
- rkagerer 4y ago
  Well, in fairness web-based applications (including Electron ones running locally) break most of the inspectability benefits that came before, at least on the Windows platform. eg. I used to be able to programmatically grab hWnd's (like Spy++ does), and manipulate the forms of an app. Apps used to show up as one (or two) simple processes with rich instrumentation metrics in tools like Resource Monitor. Chrome abstracted all that away in an effort to 'displace' the OS. I think canvas-only technologies could eventually do the same (the endless cycle continues) and look forward to some of the benefits (eg. more deterministic and straightforward positioning and layout). But I strongly agree they need to accommodate most of the features you mentioned and until they do it's two steps backward. Do appreciate your thoughtful comment and the concise summary of some fundamental limitations that are presently hard barriers.
  - horsawlarway 4y ago
    > Well, in fairness web-based applications (including Electron ones running locally) break most of the inspectability benefits that came before, at least on the Windows platform. eg. I used to be able to programmatically grab hWnd's (like Spy++ does), and manipulate the forms of an app. Apps used to show up as one (or two) simple processes with rich instrumentation metrics in tools like Resource Monitor. --- I disagree pretty hard with this sentiment. Yes, it's harder to programmatically interact with some of the browser features from outside of the browser. Programmatic interaction with basically any application that isn't yours is getting harder across every operating system as security becomes more important. That doesn't really change that introspection of the source that makes up a displayed site was easy and approachable. I can look at the css/html/js. Further, all of those resources are contained in files intended to be opened and looked at by humans (they're human readable text). This is my biggest issue with WASM. You're literally back at reading assembly again. And while that's possible, and there is tooling that can help - it's just not the same as being able to right-click on something, alter a couple of attributes in human readable words, hit enter and see that change.
    - josephg 4y ago
      > This is my biggest issue with WASM. You're literally back at reading assembly again. And while that's possible, and there is tooling that can help - it's just not the same as being able to right-click on something, alter a couple of attributes in human readable words, hit enter and see that change. WASM on its own isn't the problem. WASM just lets you use another language to replace javascript. When you right click on something in a browser and alter human-readable attributes, you're usually messing with CSS and HTML. Any half decent web UI framework should pass HTML and CSS to the browser in order to render the UI. Thats just as true with wasm frameworks as it is with javascript frameworks. The problem in this example is that the wasm code is bundling its own UI library that is trying (badly) to replace the browser's layout engine. All the browser can see is a bunch of drawing calls to an html canvas. Thats not the fault of wasm. Its just this UI library, which isn't really designed for the web.
eslaught 4y ago
I have no beef with deferred mode GUIs except that I have yet to see one with the performance characteristics I need. For what it's worth, egui does have an accessibility layer. I haven't gone to any particular effort to wire it up yet (besides what you get by default): https://github.com/emilk/egui/issues/167 https://github.com/emilk/egui/issues/167
- danShumway 4y ago
  This is what I don't get: if you have an accessibility layer that is fully usable, that means you have the capability to describe your user interface using only speech/text. So what are you doing that couldn't be performant in the DOM? Because any kind of fast updates that you're doing with a lot of triangle that are too fast to render into an XML tree are also going to be happening too fast to describe with a screen reader. The DOM is asking you to be able to describe your interface -- performantly -- in entirely pure text. If you can't do that, then there's no way you can be fully accessible, because the screen reader needs to be able to describe your interface -- performantly -- in entirely pure text.
  - danShumway 4y ago
    People always get upset about this kind of thing, but it's very obviously true. If you are updating thousands of rectangles in your UI at 30/60fps, it is impossible for you to fully convey the same amount of information that is changing visually to a screen reader. Either your interface can be described using pure text, or it can't. Either I could use your interface by closing my eyes and having someone describe it to me, or I can't.
keyle 4y ago
That's nothing new with immediate rendering... You can't have great performance and great usability. Not saying I prefer that way, just pointing out the obvious.
flohofwoe 4y ago
If the web API zoo wouldn't be so random it would all be easier. Pretty much all problems on the web (including the Javascript framework churn) are caused by the DOM being too high level, too inflexible and generally too 'black-boxy' to do anything interesting. Accessibility could be fixed easily if there would be a web accessibility API. There's basically an entire 'middle layer' of web APIs missing, current APIs are either too low level (like WebGL/WebGPU) or too high level (like the DOM).
- zkldi 4y ago
  you're almost describing flash i mean, it sucked, but it was kind of that middle layer.
- danShumway 4y ago
  > Accessibility could be fixed easily if there would be a web accessibility API It does have one. The DOM is the web accessibility API. A really big point of the DOM is that you don't get to make multiple interfaces, you have to make one interface that works for everyone, which forces you to provide some degree of feature parity between those interfaces. The web accessibility API is "you have to describe your interface to us in pure text, because that format is more likely to be accessible."
amelius 4y ago
We could build a "DOM" inspector that runs inside the canvas too ;)
kps 4y ago
> I sincerely hope the web as a whole doesn't move in this direction. I'm not happy about it, but I think it will. > The inspectibility of the DOM is one of the web's greatest strengths. Not if you're pushing ads.

deleted 4y ago

[deleted]

chrismorgan 4y ago

And scrolling is unavoidably all wrong: dragging two fingers from one end of my touchpad to the other should scroll by about four screenfuls, but only manages two thirds of one screenful; a quick swipe should send it dozens of screenfuls, with inertia, but manages even less than that two thirds of a screenful; and these limitations are fundamental due to the web not possessing the right primitives. (Then there are more fixable bugs like that, after focusing the page, you have to move the mouse before scrolling will work.) And input element behaviour is non-native, which will never be properly fixable since different UAs have different behaviours, some of which cannot be emulated even if you sniff which platform you’re on (e.g. scroll bar grab and drag behaviour when you move far outside the window). (Then there are also simple bugs: holding Left or Right while the range element is focused only bumps the value once, not according to key repeat. And Up and Down don’t work.) I have written about these things a number of times, fundamental limitations of the pure-canvas approach that make it unavoidably and permanently unsuitable. https://news.ycombinator.com/item?id=33861831 https://news.ycombinator.com/item?id=33861831 is probably the best thread. (And just for a bit of fun: the demo can have slightly negative utilisation, drawing outside the box!)

eslaught 4y ago
In my testing, I went through a bunch of Rust frameworks that sit on top of the core OS UI frameworks. My experience with all of those was that they all capped out at about 3k rectangles on a screen at a time at 30 fps (vs the about 80k rectangles that fit at usually 30-60 fps in my demo). I wonder if your experience is different? As best I've been able to tell, right now you have to make a choice between (a) fast and (b) native, and you only get to choose one. And in my application domain, fast is nonnegotiable.
- mtsr 4y ago
  Mixed UIs could probably work well here. Using a canvas and specific gpu rendering just for the part requiring high throughput makes sense.
- josephg 4y ago
  What domain are you working in? Smoothly rendering 3k rectangles isn't a common use case for UI libraries, so I'm not surprised they perform poorly. They generally aren't optimized for that. The normal answer for weird, high performance rendering like that is to just use opengl or something and program against the GPU directly. 3d rendering contexts can render millions of rectangles on modern hardware without breaking a sweat. This advice is the same on desktop, mobile or the web. I'm not surprised rust UI frameworks have the same limitation. Most UIs don't need thousands of rectangles. But they do need lots of custom widgets, and implementing all of those well is a higher priority job for anyone building a custom UI library.
  - eslaught 4y ago
    Like I said in my top comment, it's a profiler. I need to consume logs from distributed applications (on possibly tens of thousands of nodes) and render them in as coherent a fashion as possible. Going to 3d rendering via OpenGL / WebGL was one of the options I considered. But honestly, this is a boatload of complexity. Why do I need to pick up a full 3D rendering library just to draw rectangles on the screen? Plus, consider: the slot viewer (where the rectangles are shown) still needs to draw some UI components (e.g., on hoverover). I still need that UI integration there, even if it's not as performance-sensitive. So I end up needing something along the lines of egui anyway. I definitely understand that not all UI libraries are going to be optimized for this. But I was surprised that it wasn't easy to e.g. get a high-performance canvas dropped into libraries that specifically advertised themselves as high-performance.
    - IX-103 4y ago
      The reason to pick up a "a full 3D rendering library" is that it is actually a lower level interface to the hardware. Almost all 2D operations are now implemented on top of 3D hardware primitives. By limiting yourself to a 2D engine you're ensuring there are more layers of abstraction between you and the hardware. While # layers of abstraction doesn't always correlate with overhead, it does place limits on performance.
rkagerer 4y ago
Well, in fairness web-based applications (including Electron ones running locally) break most of the inspectability benefits that came before, at least on the Windows platform. eg. I used to be able to programmatically grab hWnd's (like Spy++ does), and manipulate the forms of an app. Apps used to show up as one (or two) simple processes with rich instrumentation metrics in tools like Resource Monitor. Chrome abstracted all that away in an effort to 'displace' the OS. I think canvas-only technologies could eventually do the same (the endless cycle continues) and look forward to some of the benefits (eg. more deterministic and straightforward positioning and layout). But I strongly agree they need to accommodate most of the features you mentioned and until they do it's two steps backward. Do appreciate your thoughtful comment and the concise summary of some fundamental limitations that are presently hard barriers.
- horsawlarway 4y ago
  > Well, in fairness web-based applications (including Electron ones running locally) break most of the inspectability benefits that came before, at least on the Windows platform. eg. I used to be able to programmatically grab hWnd's (like Spy++ does), and manipulate the forms of an app. Apps used to show up as one (or two) simple processes with rich instrumentation metrics in tools like Resource Monitor. --- I disagree pretty hard with this sentiment. Yes, it's harder to programmatically interact with some of the browser features from outside of the browser. Programmatic interaction with basically any application that isn't yours is getting harder across every operating system as security becomes more important. That doesn't really change that introspection of the source that makes up a displayed site was easy and approachable. I can look at the css/html/js. Further, all of those resources are contained in files intended to be opened and looked at by humans (they're human readable text). This is my biggest issue with WASM. You're literally back at reading assembly again. And while that's possible, and there is tooling that can help - it's just not the same as being able to right-click on something, alter a couple of attributes in human readable words, hit enter and see that change.
  - josephg 4y ago
    > This is my biggest issue with WASM. You're literally back at reading assembly again. And while that's possible, and there is tooling that can help - it's just not the same as being able to right-click on something, alter a couple of attributes in human readable words, hit enter and see that change. WASM on its own isn't the problem. WASM just lets you use another language to replace javascript. When you right click on something in a browser and alter human-readable attributes, you're usually messing with CSS and HTML. Any half decent web UI framework should pass HTML and CSS to the browser in order to render the UI. Thats just as true with wasm frameworks as it is with javascript frameworks. The problem in this example is that the wasm code is bundling its own UI library that is trying (badly) to replace the browser's layout engine. All the browser can see is a bunch of drawing calls to an html canvas. Thats not the fault of wasm. Its just this UI library, which isn't really designed for the web.

eslaught 4y ago

I have no beef with deferred mode GUIs except that I have yet to see one with the performance characteristics I need. For what it's worth, egui does have an accessibility layer. I haven't gone to any particular effort to wire it up yet (besides what you get by default): https://github.com/emilk/egui/issues/167 https://github.com/emilk/egui/issues/167

danShumway 4y ago
This is what I don't get: if you have an accessibility layer that is fully usable, that means you have the capability to describe your user interface using only speech/text. So what are you doing that couldn't be performant in the DOM? Because any kind of fast updates that you're doing with a lot of triangle that are too fast to render into an XML tree are also going to be happening too fast to describe with a screen reader. The DOM is asking you to be able to describe your interface -- performantly -- in entirely pure text. If you can't do that, then there's no way you can be fully accessible, because the screen reader needs to be able to describe your interface -- performantly -- in entirely pure text.
- danShumway 4y ago
  People always get upset about this kind of thing, but it's very obviously true. If you are updating thousands of rectangles in your UI at 30/60fps, it is impossible for you to fully convey the same amount of information that is changing visually to a screen reader. Either your interface can be described using pure text, or it can't. Either I could use your interface by closing my eyes and having someone describe it to me, or I can't.

keyle 4y ago

That's nothing new with immediate rendering... You can't have great performance and great usability. Not saying I prefer that way, just pointing out the obvious.

flohofwoe 4y ago

If the web API zoo wouldn't be so random it would all be easier. Pretty much all problems on the web (including the Javascript framework churn) are caused by the DOM being too high level, too inflexible and generally too 'black-boxy' to do anything interesting. Accessibility could be fixed easily if there would be a web accessibility API. There's basically an entire 'middle layer' of web APIs missing, current APIs are either too low level (like WebGL/WebGPU) or too high level (like the DOM).

zkldi 4y ago
you're almost describing flash i mean, it sucked, but it was kind of that middle layer.
danShumway 4y ago
> Accessibility could be fixed easily if there would be a web accessibility API It does have one. The DOM is the web accessibility API. A really big point of the DOM is that you don't get to make multiple interfaces, you have to make one interface that works for everyone, which forces you to provide some degree of feature parity between those interfaces. The web accessibility API is "you have to describe your interface to us in pure text, because that format is more likely to be accessible."

amelius 4y ago

We could build a "DOM" inspector that runs inside the canvas too ;)

kps 4y ago

> I sincerely hope the web as a whole doesn't move in this direction. I'm not happy about it, but I think it will. > The inspectibility of the DOM is one of the web's greatest strengths. Not if you're pushing ads.